Converting KVM virtual machines to VirtualBox

Recently the requirement came up to take a KVM based virtual machine and move it over to a VirtualBox image. Which turned out to be a fairly simple endeavour, and was fairly painless. The longest part was transferring over the 40 GB image from one machine to the other where the conversion could take place. The machine the image was coming from was only on a 100 Mbit/sec connection so that took a good hour.

Converting from KVM to VirtualBox for a FreeBSD image was pretty simple, the VBoxManage command has a convertdd command that allows you to convert from raw disk .img format to .vdi format.

VBoxManage convertdd KVM-image.img VB-image.vdi

After this, unfortunately, there is no way to to automatically convert over the settings that the virtual machine had, such as the network cards, the memory allocations and hard drive settings. You will have to go to VirtualBox and create a new virtual machine and replicate all of the settings. Once that is done make sure to select the same type of disk controller (SATA or IDE) so that the drive will hopefully be assigned the same name in the device tree so that you don't need to alter your /etc/fstab.

Hopefully everything boots without any issues. If not try creating a new virtual machine, attach the converted image as a secondary drive and see if you can mount the converted image within your new install. If so maybe transferring the data using rsync or dump/restore would be an option.

Generating stylesheets for pygments

For this blog I am using Python markdown with the Pygments plugin that is available. Pygments does all of the code highlighting and outputs the required span tags and inputs them within the page.

If you were to look at the <div><pre></pre></div> block that is generated you'd see something like this:

<div class="codehilite"><pre><span class="k">def</span> <span class="nf">function</span><span class="p">(</span><span class="n">var</span><span class="o">=</span><span class="bp">None</span><span class="p">):</span>
<span class="k">if</span> <span class="n">var</span> <span class="ow">is</span> <span class="ow">not</span> <span class="bp">None</span><span class="p">:</span>
    <span class="k">print</span> <span class="n">var</span>
</pre></div>

which is generated from the following snippet of code:

def function(var=None):
    if var is not None:
        print var

As you see the top css class is codehilite, and then underneath that there are a bunch of other classes such as k, n, o, and so forth. Those are all classes that can be styled. Now one could write such a stylesheet by hand or we can simply use one of the stylesheets that Pygments ships with.

The documentation for Pygments tells you how to get built-in styles, which is as simple as:

>>> from pygments.styles import get_all_styles
>>> styles = list(get_all_styles())
>>> styles
['monokai', 'manni', 'perldoc', 'borland', 'colorful', 'default', ...]

These are all the styles that are available to you the user, now how do we get the style into an actual .css file that we can link to in our sites HTML file? We use the pygmentize command line tool:

pygmentize -f html -S trac -a .codehilite > pygments.css

Feel free to replace trac with any of the styles you have installed on your system. -f tells it what formatter to use, since we want it to output the css we ask it to use the HTML formatter, -S tells pygmentize what style to use, and -a prepends the .codehilite class before every rule (this way we don't pollute the whole CSS namespace with all the aforementioned class's such as k, o, ow and others).

NAT with PF on an interface with multiple IP addresses

If you want to do NAT for your currently running jail instances on FreeBSD so that they can have outgoing connections you could try the following pf.conf to set up NAT on the interface that the jails have their IP addresses on.

ext_if="em0"    # The network card your default gateway is on
jail_if="lo1"   # The interface that your jails have IP addresses on

# Set some options
set optimization aggressive
set block-policy drop
set skip on lo

# NAT on the external interface when coming from the jail interface
nat on $ext_if from $jail_if:network:0 to any -> ($ext_if)

# We just pass everything
pass quick all

What I have done is create IP addresses within the private network range (10/8, 172.16/12, 192.168/16) on the lo1 interface. These can't be in the 127/8 range because those addresses can't be NAT'ed (not sure if this is a limitation in pf or if this is a FreeBSD limitation), which is a shame because using 127.1.0.1/24 would be pretty awesome in my opinion.

What I did find though is that the above will not work correctly if your main network card (the one your default gateway is on, in the example em0) contains multiple IP addresses. At that point the syntax ($ext_if) does not function correctly and will cause packet loss/drop. ¹

So instead of using the syntax above we simply replace ($ext_if) with the actual IP address of the interface. This takes care of the issue and will let your jails have proper internet access without issues.

ext_if="em0"
jail_if="lo1"

set optimization aggressive
set block-policy drop
set skip on lo

nat on $ext_if from $jail_if:network:0 to any -> 192.168.1.2 # Your IP!

pass quick all

The other way you can solve this problem off course is to provide your jails with IP addresses within the same range that go to your default gateway thereby solving the problem of needing to NAT in the first place.

FreeBSD running within KVM: Serial Console

Instead of using virtualisation technology such as VMWare of VirtualBox I decided to give the new KVM technology that is included with Linux. It is fast, and has been extremely reliable. However I wanted to be able to access the console on the guest FreeBSD instances.

I used virt-manager, a graphical tool, to set up everything, including creating the virtual machine, I didn't want to have to deal with XML configuration files and whatnot. After installing FreeBSD there are a few changes I made to get a virtual console working, this allows virsh console <domain> to function as expected.

First we add a new file to the root file system (which should be the first slice on the drive):

echo "-Dh" > /boot.config

See man 5 boot.config for more information. Basically -D says that FreeBSD should boot with a dual console configuration (built-in over VGA) and serial console, and -h says to force the serial console to on.

If you were to reboot now and use the virsh command:

virsh console FreeBSD

you would see the system go through the boot process and show output starting from when the boot block gains control, at this point however you still have no way to login to this new console since we never specified that we wanted any ttys on the serial console, for that we need to edit /etc/ttys.

We are looking for the ttyu0 entry, we want to make sure to change dialup to vt100, and off to on, so that it should look like this:

ttyu0   "/usr/libexec/getty std.9600"   vt100   on secure

Now when we reboot once more we will get the usual login display that we have come to expect of console on FreeBSD, and we can login over the serial console. This way we don't have to open up the virtual machine manager to get a console, and can continue to do almost everything from a terminal window.

PostgreSQL setup for external connections

PostgreSQL is a fantastic open-source RDBMS which is extremely powerful, once installed on FreeBSD getting it set up is pretty simple. We do want to change some of the defaults though. By default the system will be set up to trust any user using the local socket to connect as any user to any database.

By setting up the initdb flags we can change how PostgreSQL generates the default configuration files. Adding -A md5 tells it to use the md5 authentication mechanism, which requires that passwords sent over the wire are md5 hashed.

Add the following to /etc/rc.conf for the initdb flags, and enable PostgreSQL as a service:

postgresql_initdb_flags="-D /usr/local/pgsql/data -W -A md5"
postgresql_enable="YES"

And then run:

/usr/local/etc/rc.d/postgresql initdb

This will initialise and create the configuration as well as setting up some of the default tables. You will want to navigate to the folder specified in the initdb_flags above:

cd /usr/local/pgsql/data

this is where you will find the files that we are going to modify next to get PostgreSQL set up to listen to connections coming in on a private network (192.168.1.1/24) and where we allow connections coming in from servers on that network. This is strictly for setting up PostgreSQL in a situation where you have multiple servers accessing it as a client on a private network.

After this we need to modify a few files to get it to listen to TCP/IPv4 connections, first we start with postgresql.conf and add:

listen_addresses = '127.0.0.1,192.168.1.1'

Then in pg_hba.conf we add a new line:

host    all     all     192.168.1.1/24      md5

This allows anyone in the 192.168.1.1/24 range to connect to our PostgreSQL instance and to use the databases. This is exactly what we want to allow, now we start PostgreSQL using the rc script:

/usr/local/etc/rc.d/postgresql start