Compacting a VirtualBox VM Centos disk

At work I used Centos 6.4 to convert a large Subversion repo to Git. This required a VM with a 100Gb disk, more than my old iMac could deal with, to the point where I could not update to newer OS version or even run iTunes to update apps.

[dchwalis@imac ~]$ df -h
Filesystem      Size   Used  Avail Capacity   iused   ifree %iused  Mounted on
/dev/disk0s2   931Gi  915Gi   16Gi    99% 239830061 4150681   98%   /

This article “How can I reduce the size of a VM on disk” started me off. Then I found https://forums.virtualbox.org/viewtopic.php?p=34636&sid=5bbe19387a6797d599beb35ec2e8b96c#p34636. Looks like since I used a large portion of the expandable virtual disk, there was data in it, and could not be reclaimed. I needed to zero out the now-deleted files and then pack the virtual disk image.

The tool to zero out the unused storage is zerofree and I needed to build that. Luckily my VM had all the dev tools installed, so it was a simple make. I grabbed the latest version, 1.0.3.

[dchwalis@centos ~]$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_vmxxfscv62979-lv_home
                      242G   74G  156G  33% /home

First bust the system down to runlevel 1 to prevent other processes from keeping the disk busy.

sudo telinit 1

Now how to run a tool on a mount that was being used. This is why you should ALWAYS create new disks and mount them instead of expanding the system disk.

(Mount command line missing. It disappeared into runlevel 1)
Something like

# mkdir /fred
# mount -o remount,rw /dev/mapper/vg_vmxxfscv62979-lv_home /fred

I ran zerofree on the device. That took a long time and my iMac kept sleeping. Not good.

# /usr/bin/zerofree /dev/mapper/vg_vmxxfscv62979-lv_home

Then to have fsck run when the VM started up, I did per “Force fsck on reboot

# touch /forcefsck
# reboot

Then back to to have

[dchwalis@imac ~]$ VBoxManage modifyhd --compact centos-64-x64.vdi
0%...10%...

This step took a while.

[dchwalis@imac ~]$ df -h
Filesystem Size Used Avail Capacity iused ifree %iused Mounted on
/dev/disk0s2 931Gi 780Gi 151Gi 84% 204440272 39540470 84% /

So 16Gb free to 151Gb free. Looks like it worked. Lets start up the VM… Yay!

[1] http://askubuntu.com/questions/175739/how-do-i-remount-a-filesystem-as-read-write
[2] http://linux.die.net/man/8/mount
[3] http://ianozsvald.com/2012/07/26/using-zerofree-to-shrink-a-virtualbox-linux-image/
[4] http://frippery.org/uml/index.html
[5] “How can I reduce the size of a VM on disk”
[6] http://unix.stackexchange.com/questions/11100/how-to-make-ext4-filesystem-sparse

Building Git 1.8 on Centos 6

We use Centos as a VM for development. Intellij IDEA was complaining about the 1.7.1 version from the Centos distro, requiring 1.7.1.1. Picky.

So I grabbed the git source:
$ git clone https://github.com/git/git
$ cd git
$ make
...
http-push.c:920: error: ‘xml_cdata’ undeclared (first use in this function)
http-push.c:921: warning: implicit declaration of function ‘XML_Parse’
http-push.c:926: warning: implicit declaration of function ‘XML_ErrorString’
http-push.c:927: warning: implicit declaration of function ‘XML_GetErrorCode’
http-push.c:930: warning: implicit declaration of function ‘XML_ParserFree’
http-push.c: In function ‘remote_ls’:
http-push.c:1154: error: ‘XML_Parser’ undeclared (first use in this function)
http-push.c:1154: error: expected ‘;’ before ‘parser’
http-push.c:1161: error: ‘parser’ undeclared (first use in this function)
http-push.c:1164: error: ‘xml_cdata’ undeclared (first use in this function)
http-push.c: In function ‘locking_available’:
http-push.c:1228: error: ‘XML_Parser’ undeclared (first use in this function)
http-push.c:1228: error: expected ‘;’ before ‘parser’
http-push.c:1235: error: ‘parser’ undeclared (first use in this function)

Oh man, compile errors. Bad.

The key seemed to be:
expat.h: No such file or directory

A bit o Googling: missing library headers.
$ sudo yum install expat-devel

Restart the build. Works.

$ make install

Whee!
$ git --version
git version 1.8.3.2.768.g911011a

ps. Dont forget the
export PATH=~/bin:$PATH

Building KDiff3 on Centos 6

My goto diff tool on Windows is the open-source WinMerge. On Linux, Ubuntu has an installable KDiff3, a member of the Qt-based KDE family of tools. At work we are using Centos 6 as our dev VM, and no installable KDiff3 is available.

So the README says to

$ ./configure kde4

and it should build and install. Our VMs had Qt installed already, but it failed getting the kdelibs. The instructions specified:

   - libkde4-devel (KDE4 Header files and development libraries)

So get them. Right?

$ yum install libkde4-devel
...
updates                                                  | 3.4 kB     00:00     
Setting up Install Process
No package libkde4-devel available.
Error: Nothing to do

I am not a lib guru, but have done work on Qt (but Windows and Mac) and some Linux devs. After struggling from one to another I found:

$ yum list | grep kde
kde-filesystem.noarch                    4-30.1.el6                     @base   
kde-settings.noarch                      4.3.1-1.el6.centos             @base   
kdebase.x86_64                           6:4.3.4-6.el6                  @base   
kdebase-libs.x86_64                      6:4.3.4-6.el6                  @base   
kdebase-runtime.x86_64                   4.3.4-9.el6                    @base   
kdebase-runtime-libs.x86_64              4.3.4-9.el6                    @base   
kdebase-workspace.x86_64                 4.3.4-24.el6                   @base   
kdebase-workspace-libs.x86_64            4.3.4-24.el6                   @base   
kdebase-workspace-wallpapers.noarch      4.3.4-24.el6                   @base   
kdelibs.x86_64                           6:4.3.4-19.el6                 @base   
kdelibs-common.x86_64                    6:4.3.4-19.el6                 @base   
kdelibs-devel.x86_64                     6:4.3.4-19.el6                 @base   
kdelibs-experimental.x86_64              4.3.4-3.el6                    @base   
kdepimlibs.x86_64                        4.3.4-4.el6                    @base   
...

Aha! In Centos/RedHat/Fedora, the package is different — or maybe the project itself changed the lib names, and the doc is stale…

So this was the happy path for me:

$ sudo yum install kdelibs-devel
$ sudo yum install cmake
[releaseKde]$ kde4-config --prefix
/usr
$ cmake .. -DCMAKE_INSTALL_PREFIX=/usr
$ make
$ sudo make install

A colleague for whom I was trying to get this to work installed the correct lib, and then the $ ./configure kde4 worked fine. Probably totally obvious to a Linux guru, but a good exercise for a lib-newbie.