FreeBSD running within KVM: Serial Console

Instead of using virtualisation technology such as VMWare of VirtualBox I decided to give the new KVM technology that is included with Linux. It is fast, and has been extremely reliable. However I wanted to be able to access the console on the guest FreeBSD instances.

I used virt-manager, a graphical tool, to set up everything, including creating the virtual machine, I didn't want to have to deal with XML configuration files and whatnot. After installing FreeBSD there are a few changes I made to get a virtual console working, this allows virsh console <domain> to function as expected.

First we add a new file to the root file system (which should be the first slice on the drive):

echo "-Dh" > /boot.config

See man 5 boot.config for more information. Basically -D says that FreeBSD should boot with a dual console configuration (built-in over VGA) and serial console, and -h says to force the serial console to on.

If you were to reboot now and use the virsh command:

virsh console FreeBSD

you would see the system go through the boot process and show output starting from when the boot block gains control, at this point however you still have no way to login to this new console since we never specified that we wanted any ttys on the serial console, for that we need to edit /etc/ttys.

We are looking for the ttyu0 entry, we want to make sure to change dialup to vt100, and off to on, so that it should look like this:

ttyu0   "/usr/libexec/getty std.9600"   vt100   on secure

Now when we reboot once more we will get the usual login display that we have come to expect of console on FreeBSD, and we can login over the serial console. This way we don't have to open up the virtual machine manager to get a console, and can continue to do almost everything from a terminal window.

PostgreSQL setup for external connections

PostgreSQL is a fantastic open-source RDBMS which is extremely powerful, once installed on FreeBSD getting it set up is pretty simple. We do want to change some of the defaults though. By default the system will be set up to trust any user using the local socket to connect as any user to any database.

By setting up the initdb flags we can change how PostgreSQL generates the default configuration files. Adding -A md5 tells it to use the md5 authentication mechanism, which requires that passwords sent over the wire are md5 hashed.

Add the following to /etc/rc.conf for the initdb flags, and enable PostgreSQL as a service:

postgresql_initdb_flags="-D /usr/local/pgsql/data -W -A md5"
postgresql_enable="YES"

And then run:

/usr/local/etc/rc.d/postgresql initdb

This will initialise and create the configuration as well as setting up some of the default tables. You will want to navigate to the folder specified in the initdb_flags above:

cd /usr/local/pgsql/data

this is where you will find the files that we are going to modify next to get PostgreSQL set up to listen to connections coming in on a private network (192.168.1.1/24) and where we allow connections coming in from servers on that network. This is strictly for setting up PostgreSQL in a situation where you have multiple servers accessing it as a client on a private network.

After this we need to modify a few files to get it to listen to TCP/IPv4 connections, first we start with postgresql.conf and add:

listen_addresses = '127.0.0.1,192.168.1.1'

Then in pg_hba.conf we add a new line:

host    all     all     192.168.1.1/24      md5

This allows anyone in the 192.168.1.1/24 range to connect to our PostgreSQL instance and to use the databases. This is exactly what we want to allow, now we start PostgreSQL using the rc script:

/usr/local/etc/rc.d/postgresql start

Apache mod_fastcgi and PHP with PHP-FPM

Recently I did a server migration from an older server to a newer server and in an attempt to help stability I wanted to see if there was a better way to do PHP FastCGI. In my research I came across running PHP using the FastCGI server that spins up a PHP on a TCP/IP port and allows the web server to connect to it. However this doesn't help with spawning or keeping track of instances or error recovery.

This is where PHP-FPM comes in handy. It does all of the hard work for us, it spawns the processes and has a bunch of really awesome features that help run PHP as various different users as required with different PHP ini files and memory limits. PHP-FPM's defaults, at least from a ports install, are extremely sane and I don't really suggest changing them. After setting up PHP FPM I had to set up Apache.

mod_fcgid doesn't allow for remote connections, and as such I was unable to use it for what I needed it for. mod_fastcgi provides the FastCGIExternalServer configuration key, which is exactly what I needed.

"The FastCgiExternalServer directive defines filename as an external FastCGI application. If filename does not begin with a slash (/) then it is assumed to be relative to the ServerRoot. The filename does not have to exist in the local filesystem. URIs that Apache resolves to this filename will be handled by this external FastCGI application."

What this documentation does not state is that the path up to the last part of it has to exist in the local file system. So in my first couple of attempts I pointed it at /usr/local/www/fastcgi/php5.fcgi without having an actual fastcgi directory located in /usr/local/www/. There are a lot of examples that require creating a new FastCGIExternalServer for each and every VirtualHost this is unacceptable to me, the reason they require it is because they set the FastCGIExternalServer path to the folder where they are going to be serving files from.

In the end I found that after creating the /usr/local/www/fastcgi directory (and reading the mod_fastcgi source code) that all it does is make Apache believe a file exists in a certain directory, much like Alias, except Alias allows full paths to be aliased, not so here.

The AddType and Action add custom types and what the action should be when such a type is encountered. In this case the action is to redirect the request to /php5.fcgi which will handle the rest of the request. This does not require another FastCGI section per VirtualHost as each PHP request will just get shuttled to the php handler.

Do take note that I have specifically disallowed Apache to serve anything from the /usr/local/www/fastcgi/ folder, except for a single file php5.fcgi which is our FastCGIExternalServer file.

<IfModule mod_fastcgi.c>
    Alias /php5.fcgi /usr/local/www/fastcgi/php5.fcgi
    FastCGIExternalServer /usr/local/www/fastcgi/php5.fcgi -flush -host 127.0.0.1:9000
    AddType application/x-httpd-fastphp5 .php
    Action application/x-httpd-fastphp5 /php5.fcgi

    <Directory "/usr/local/www/fastcgi/">
        Order deny,allow
        Deny from all
        <Files "php5.fcgi">
            Order allow,deny
            Allow from all
        </Files>
    </Directory>
</IfModule>

Note that even-though in my last post concerning mod_fastcgi I as moving away from it, I am now doing the opposite, instead of moving from mod_fastcgi to mod_fcgid I'm back to mod_fastcgi, only because mod_fcgid doesn't offer the same functionality.

So far this has provided far more stability, along with PHP-FPM doing all of the process management I can now use a single PHP instance that is running on a single port for the various web servers I am testing. At the moment I have both Lighttpd and Apache using the same PHP-FPM instance. It is faster, less memory is wasted and PHP-FPM is much better at process management than mod_fastcgi or mod_fcgid.

Using DJB's daemontools and netcat to bounce a request around

Lately I have been moving the data from one server to another server, located halfway across the globe, and I needed some way to forward all incoming requests from the old server to the new server. This had to be done so that DNS could take its time to update while everything was now already being served from the new location.

What I ended up doing was using daemontools along with netcat to pipe the request around the world. Here are the steps I took in doing so:

mkdir /usr/local/redirect/
cd /usr/local/redirect/
mkdir smtp smtp/env
cd smtp

Put the following in a file named run:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

#!/bin/sh
exec 2>&1 \
envdir ./env \
sh -c '
    case "$REMOTENAME" in h) H=;; p) H=p;; *) H=H;; esac
    case "$REMOTEINFO" in r) R=;; [0-9]*) R="t$REMOTEINFO";; *) R=R;; esac
    exec \
    /usr/local/bin/tcpserver \
        -vDU"$H$R" \
        ${LOCALNAME+"-l$LOCALNAME"} \
        ${BACKLOG+"-b$BACKLOG"} \
        ${CONCURRENCY+"-c$CONCURRENCY"} \
        -- "${IP-0}" "${PORT}" \
        /usr/bin/nc "${REMOTEHOST}" "${REMOTEPORT}"
'

Make it executable:

chmod +x run

Then we need to set up a few environment variables:

cd env
echo "example.net" > REMOTEHOST
echo "25" > REMOTEPORT
echo "25" > PORT
echo `hostname` > LOCALNAME
echo "200" > CONCURRENCY

I made REMOTEPORT and PORT be separate on purpose, in one case I now had more IP's than before, so instead of having SSL running on a separate port it was running on the default port, and I needed a clean way to forward that.

Now just add it to your services folder as a symlink and it will automatically be started, from there it will do it's job! I also suggest adding some simple logging, or discarding all off the output from tcpserver.