VirtualBox + FreeNAS … an optimistic “maybe”

When I first heard that VirtualBox could be ran on a FreeNAS server, I was ecstatic. The idea of consolidating many of the little services onto one single machine was very enticing – less machines to manage UPS shutdown/power-on for, less network traffic and freeing-up of more computers to play with in my home lab!

Right after the release, I got cracking. Starting on my Dell 2950 a.k.a. the jet, I’ve installed plain FreeNAS and gave it a pair of SAS disks to home jails and VMs. First one to install was the good old Ubuntu 12.04. The host has 32GB of RAM in total, so I gave it generous 7GB of RAM and a couple of CPUs. phpVirtualBox worked great at a first glance, and all seemed OK.

After a few moments of use, I’ve discovered a few issues that needed working around:

1. some changes in the UI crash the VBoxHeadless process – namely make all your virtual screen settings in advance of powering the host on
2. it only supported 3.5GB of memory per host – any attempt of the OS to access anything beyond that mark froze the virtual process
3. virtualisation features of the slightly dated CPU were not detected correctly

All the above may have been a bit annoying, however, they did not pose any large problems in general use. I was not going to use the 2950 as the final production machine, so I moved it to its final home – recent Xeon 1230v2-equipped host.

There I started to see another, more serious problem. After a few days / hours of use (highly variable), the VM would just lock-up. I’ve spent quite a bit of time trying to figure out the issue, and it all seemed to boil down to IO problems. At some point, the virtual controller started acting up, causing data loss for the guest and ultimately leading to quite unpleasant lockup:

22:13:17.752113 AIOMgr: I/O manager 0x00000808e12420 encountered a critical error (rc=VERR_FILE_AIO_NO_REQUEST) during operation. Falling back to failsafe mode. Expect reduced performance
22:13:17.767509 AIOMgr: Error happened in /wrkdirs/usr/ports/emulators/virtualbox-ose/work/VirtualBox-4.3.12/src/VBox/VMM/VMMR3/PDMAsyncCompletionFileNormal.cpp:(1664){int pdmacFileAioMgrNormal(RTTHREADINT*, void*)}
22:13:17.767523 AIOMgr: Please contact the product vendor
22:13:48.120506 AHCI#0: Port 0 reset
22:13:48.120560 AHCI#0P0: Cancelled task 0
22:13:48.120586 AHCI#0P0: Cancelled task 1
22:13:48.120597 AHCI#0P0: Cancelled task 2
22:13:48.120605 AHCI#0P0: Cancelled task 3
22:13:48.120613 AHCI#0P0: Cancelled task 4

I’ve found similar issue filed on the VB issue tracker, which led me to believe that it is indeed related to the host IO cache settings for each virtual drive. As I’ve ruled out performance issues (machine was idle when the freezes happened), I concluded that the most likely solution would be to try out the host IO cache enabled.

With host IO cache enabled for each of the drives, I have now been running 2 VMs simultaneously, using 8GB of RAM combined. I am now approx. 2 weeks in testing but the change seems to be positive – I have not seen a a freeze or a repeat of the error message in the logs.

My conclusion is therefore a firm, yet optimistic “maybe”. Looks like it’s working now but it was not straightforward to get it working – and everybody should keep in mind that it’s an unsupported system that may require some persuasion to work reliably in any sense.