funcptr - operations

Mac OS X El Capitan Installer Removes Custom Group ID and Membership

2015-10-02T01:23:45+00:00

As always, after Apple releases their new operating system, my systems are upgraded. This time the upgrade was less of a surprise in terms of what it brings because I'd been beta testing the new release for the past couple of weeks, however I was still caught off guard.

On OS X, by default all user accounts start at ID 501 and count up, so if you have two accounts, you will have user ID 501 and 502 in use. For most people they will most likely never change this, and all is well. The default group ID for all new user accounts is staff</code> which has a group ID of 20. So if you have a single account named for example janedoe</code> her user ID would be 501 and her group ID would be 20 (staff</code>).

Coming from a FreeBSD world and running a lot of FreeBSD systems, user accounts start at 1001, and count up. When you create a new user account on FreeBSD, by default that user is also added to a group with the same name as the username, with the same ID. So you end up with an account with ID 1001 and default group ID 1001. Using the same example, a user named janedoe</code> would have a user ID of 1001, and a group ID of 1001 (janedoe</code>).

When I first installed OS X, and almost every single new installation since, I have followed these steps to change my user ID and group ID to match those on my FreeBSD systems:


Assumption is that you have a separate user account other than the one you
are about to modify that you can temporarily use that has administrator
privileges on the local Mac; I create an "Administrator" account for that
reason.</li>
System Preferences</li>
Users and Groups</li>
Click the +</code> (You may need to click the lock in the bottom left first)</li>
Change the dropdown to group</code></li>
Enter Full Name: janedoe</code></li>
Create group</li>
Right click on group (janedoe</code>)</li>
Advanced Options...</li>
Change the Group ID to 1001</li>
Okay</li>
Right click on user (janedoe</code>)</li>
Advanced Options...</li>
Change User ID from 501</code> to 1001</code></li>
Change Group from staff</code> to janedoe</code></li>
Okay</li>
Close System Preferences</li>
Open Terminal, become root user (sudo su)</li>
cd /Users/janedoe</code></li>
find . -uid 501 -print0 | xargs -0 chown 1001:1001</code></li>
</ol>
This allows me to have the same user ID and group ID on both my Mac OS X and on
FreeBSD, thereby making it easier to use tools like rsync that keeps ownership
and permissions, as well as using NFS. Other ways to do something similar is
using LDAP/Kerberos with shared directory service, but that is a little heavy
handed for a home network.</p>
This has worked for me without issues since OS X 10.8, even upgrading from 10.8 to
10.9 and then 10.10 did not change anything. However as soon as I did the
upgrade to El Capitan (10.11) I noticed that all of my ls -lah</code> output looked
like this:</p>
drwxr-xr-x+  13 xistence  1001   442B Oct  1 16:58 Desktop</span></span>
drwx------+  28 xistence  1001   952B Aug 31 12:17 Documents</span></span>
drwx------+  89 xistence  1001   3.0K Oct  1 15:56 Downloads</span></span>
drwx------@  72 xistence  1001   2.4K Oct  2 00:16 Library</span></span></code></pre>
and id</code> provided this valuable output:</p>
uid=1001(xistence) gid=20(xistence) groups=20(xistence),12(everyone),61(localaccounts),399(com.apple.access_ssh),402(com.apple.sharepoint.group.2),401(com.apple.sharepoint.group.1),100(_lpoperator)</span></span></code></pre>
Wait, what happened to the staff</code> group that I am supposed to be a member of,
and why is my xistence</code> group ID now stating it is 20 and not 1001 as I was
expecting.</p>
I wondered if the upgrade had messed up my group somehow, and it was
confirmed when I checked with dscl</code>.</p>
$ dscl . -read /Groups/xistence</span></span>
[...]</span></span>
Password: *</span></span>
PrimaryGroupID: 20</span></span>
RealName: xistence</span></span>
RecordName: xistence</span></span>
RecordType: dsRecTypeStandard:Groups</span></span></code></pre>
Do note that the group xistence</code> does not show up in System Preferences</em> ->
Users and Groups</em>, so we'll have to do some command line magic.</p>
Well, that's worrisome, why is this matching a built-in group's ID? Specifically
let's check the staff</code> group and make sure it still has the appropriate group
ID.</p>
$ dscl . -read /Groups/staff</span></span>
[...]</span></span>
GroupMembership: root</span></span>
Password: *</span></span>
PrimaryGroupID: 20</span></span>
RealName: Staff</span></span>
RecordName: staff BUILTIN\Users</span></span>
RecordType: dsRecTypeStandard:Groups</span></span></code></pre>
Next I had to check to see what my user account was set to as the default group
ID:</p>
$ dscl . -read /Users/xistence</span></span>
[...]</span></span>
NFSHomeDirectory: /Users/xistence</span></span>
Password: ********</span></span>
PrimaryGroupID: 20</span></span>
RealName:</span></span>
 Delta Regeer</span></span>
RecordName: xistence delta@regeer.org com.apple.idms.appleid.prd.53696d524c62372b48344a53755864634e4f374b32513d3d</span></span>
RecordType: dsRecTypeStandard:Users</span></span>
UniqueID: 1001</span></span>
UserShell: /bin/bash</span></span></code></pre>
Well, that is not entirely what I was expecting either, at last it didn't touch
my user ID. Time to fix things.</p>
First let's change the xistence</code> group's group ID to 1001, and then change the
Primary Group ID for the user xistence</code> to group ID 1001.</p>
dscl</span> .</span> -change</span> /Groups/xistence PrimaryGroupID</span> 20 1001</span></span>
dscl</span> .</span> -change</span> /Users/xistence PrimaryGroupID</span> 20 1001</span></span></code></pre>
After that id</code> looked a little bit more sane:</p>
uid=1001(xistence) gid=1001(xistence) groups=1001(xistence),12(everyone),61(localaccounts),399(com.apple.access_ssh),402(com.apple.sharepoint.group.2),401(com.apple.sharepoint.group.1),100(_lpoperator)</span></span></code></pre>
However now the group staff</code> is missing from the list of groups that the user
xistence</code> is a member of, which I don't think will hurt anything, but we still
want to be able to read/write any folders that are designated as staff</code>
elsewhere in the OS, and any other privileges that entails. So let's add the
user xistence</code> to the staff</code> group:</p>
dscl</span> .</span> -append</span> /Groups/staff GroupMembership xistence</span></span></code></pre>
Let's verify, and check id</code> again:</p>
uid=1001(xistence) gid=1001(xistence) groups=1001(xistence),12(everyone),20(staff),61(localaccounts),399(com.apple.access_ssh),402(com.apple.sharepoint.group.2),401(com.apple.sharepoint.group.1),100(_lpoperator)</span></span></code></pre>
For this to fully take effect, log out and log back in. This will make sure
that all new files have the correct user ID/group ID set.</p>
After the change to the Group ID, the group still doesn't show up in System
Preferences</em> -> Users and Groups</em>, which I find weird since it is not a
built-in group.</p>
Luckily everything is back to the way it was before the upgrade, and my backup
scripts and NFS shares work again without issues.</p>

Cobbler with CentOS 7 failure to boot/kickstart 2015-07-08T19:54:43+00:00 Over the past week I've been working on building out an instance of Cobbler and testing some of the provisioning that it is able to do. One of the operating systems that I wanted to deploy is CentOS 7. After I imported the system into cobbler, it correctly showed up in the pxelinux boot menu and it would happily load the kernel and the initrd, however after initial bootup it would throw the following error message: dracut-initqueue[867]: Warning: Could not boot. dracut-initqueue[867]: Warning: /dev/root does not exist Starting Dracut Emergency Shell... Warning: /dev/root does not exist Generating "/run/initramfs/rdsosreport.txt" Entering emergency mode. Exit the shell to continue. Type "journalctl" to view the system logs. You might want to save "/run/initramfs/rdsosreport.txt" to a USB stick or /boot after mounting them and attach it to a bug report</code></pre> After that it gives you a root shell. Some Google searching led me to an mailing list post for Cobbler</a> where someone mentioned that adding ksdevice=link</code> to the Cobbler profile allowed the system to boot without issues. However before I just implement a change I want to know why that fixes the issue, so I searched Google for "kickstart ksdevice" and found Red Hat's documentation on starting a kickstart</a>. Searching that page for "ksdevice" led me to this section: ksdevice=<device></code> The installation program uses this network device to connect to the network. You can specify the device in one of five ways: the device name of the interface, for example, eth0</code></li> the MAC address of the interface, for example, 00:12:34:56:78:9a</code></li> the keyword link</code>, which specifies the first interface with its link in the up state</li> the keyword bootif</code>, which uses the MAC address that pxelinux set in the BOOTIF variable. Set IPAPPEND 2 in your pxelinux.cfg file to have pxelinux set the BOOTIF variable.</li> the keyword ibft</code>, which uses the MAC address of the interface specified by iBFT</li> </ul> For example, consider a system connected to an NFS server through the eth1 device. To perform a kickstart installation on this system using a kickstart file from the NFS server, you would use the command ks=nfs:<server>:/<path> ksdevice=eth1</code> at the boot: prompt. </blockquote> While ksdevice=link</code> would work for some of the machines I am deploying, it wouldn't work for most since they have multiple interfaces and each one of those interfaces would have link, what I really wanted was ksdevice=bootif</code>, which is the most sensible default. So I modified the profile with ksdevice=link</code> just to test, and that worked without issues, so then I modified the profile and added ksdevice=link</code>, and this failed. I figured I should check the pxelinux.cfg/default</code> file that Cobbler generates upon issuing a cobbler sync</code> and verify that ksdevice=bootif</code> is actually listed correctly. What I found was this: LABEL CentOS-7.1-x86_64 kernel /images/CentOS-7.1-x86_64/vmlinuz MENU LABEL CentOS-7.1-x86_64 append initrd=/images/CentOS-7.1-x86_64/initrd.img ksdevice=${net0/mac} lang= kssendmac text ks=http://10.10.10.1/cblr/svc/op/ks/profile/CentOS-7.1-x86_64 ipappend 2</code></pre> This has a ksdevice=${net0/mac}</code> which is not what I had put in the profile, overwriting ksdevice</code> in the profile with ksdevice=link</code> did correctly put that into the pxelinux.cfg/default</code> file, so Cobbler was overwriting my change somehow. A quick search of ${net0/mac}</code> led me to a page about gPXE commandline items</a> that contained the same variable. At which point I remembered that in Cobbler you set up your profile to be gPXE enabled or not. The default when you import an image is to enable gPXE support. cobbler profile report --name=CentOS-7.1-x86_64 Name : CentOS-7.1-x86_64 TFTP Boot Files : {} Comment : DHCP Tag : default Distribution : CentOS-7.1-x86_64 Enable gPXE? : True Enable PXE Menu? : 1 [...]</code></pre> So let's modify the profile to disable gPXE support: cobbler profile edit --name=CentOS-7.1-x86_64 --enable-gpxe=False cobbler sync</code></pre> Verify that the change was made: cobbler profile report --name=CentOS-7.1-x86_64 [...] Enable gPXE? : False [...]</code></pre> Then let's take a look at our pxelinux.cfg/default</code> file and make sure that it looks correct: LABEL CentOS-7.1-x86_64 kernel /images/CentOS-7.1-x86_64/vmlinuz MENU LABEL CentOS-7.1-x86_64 append initrd=/images/CentOS-7.1-x86_64/initrd.img ksdevice=bootif lang= kssendmac text ks=http://10.10.10.1/cblr/svc/op/ks/profile/CentOS-7.1-x86_64 ipappend 2</code></pre> This time our ksdevice is correctly set. Upon rebooting my PXE booted server it picked up the correct interface, made a DHCP request and kickstarted the server using the provided kickstart file, and installation completed successfully. So unless you chain-boot gPXE from pxelinux by default, make sure that your profiles are not set to be gPXE enabled if you want to use them directly from the pxelinux menu. While researching more about this article, I found a blog post by Vlad Ionescu about PXE installing RHEL 7 from Cobbler</a> where he suggests disabling ksdevice</code> entirely and adding an extra inst.repo</code> variable to the kernel command line, however on older versions of CentOS 7 and Red Hat Enterprise Linux 7 there is a bug report</a> that shows that an empty ksdevice</code> could cause anaconda to crash, and setting a manual inst.repo</code> for every profile seems like overkill when just disabling gPXE for the profile also solves the problem. Neutron L3 agent with multiple provider networks 2014-09-29T18:11:00+00:00 Due to requirements outside of my control, there was a requirement to run multiple "provider" networks each with each providing their own floating address pool from a single network node, I wanted to do this as simply as possible using a single l3 agent rather than having to figure out how to get systemd to start multiple with different configuration files. Currently I've installed and configured an OpenStack instance that looks like this: +---------------------+ | | | +--+----+ | | | | +-----------+-+ +--+----------+ | | Compute | | Compute | | | 01 | | 02 | | +------+------+ +-----+-------+ | | | | | +----------+ | +------------+--+ | | | | | +-------------+ +-----+-------+ | | | Controller | | Network | | | | | | | +---+ Tenant Networks (vlan tagged) (vlan ID's 350 - 400) | +-----+----+--+ +------+----+-+ | | | | | | | | | +-----------+ Floating Networks (vlan tagged) (vlan ID's 340 - 349) | | | | | | | | +------------+--------------+----------------+ Management Network (10.5.2.0/25) | | +------------------------------------+ External API Network (10.5.2.128/25)</code></pre> There are two compute nodes, a controller node that runs all of the API services, and a network node that is strictly used for providing network functions (routers, load balancers, firewalls, all that fun stuff!). There are two flat networks that provide the following: External API access</li> A management network that OpenStack uses internally to communicate between instances and to manage it, which is not accessible from the other three networks.</li> </ol> The other two networks are both vlan tagged: Tenant networks, with the possibility of 50 vlan ID's</li> Floating networks, with existing vlan ID's for existing networks</li> </ol> Since the OpenStack Icehouse release, the l3 agent has supported the ability to use the Open vSwitch configuration to specify how traffic should be routed rather than statically defining that a single l3 agent routes certain traffic to a single Linux bridge. Setting this up is fairly simple if you follow the documentation, with one caveat, variables you think would be defined to no value, actually have a value and thus need to be explicitly zeroed out. On the network node</h2> First, we need to configure the l3 agent, so let's set some extra variables in /etc/neutron/l3-agent.ini</code>: gateway_external_network_id = external_network_bridge =</code></pre> It is important that these two are set, not left commented out, unfortunately when commented out they have some defaults set and it will fail to work, so explicitly setting them to blank will fix that issue. Next, we need to set up our Open vSwitch configuration. In /etc/neutron/plugin.ini</code> the following needs to be configured: bridge_mappings</code></li> network_vlan_ranges</code></li> </ul> Note, that these may already be configured, in which case there is nothing left to do. Mine currently looks like this: bridge_mappings = tenant1:br-tnt,provider1:br-ex</code></pre> This basically specifies that any networks created under "provider name" tenant1</code> are going to be mapped to the Open vSwitch bridge br-tnt</code> and any networks with "provider name" provider1</code> will be mapped to br-ex</code>. br-tnt</code> is mapped to my tenant network and on the switch has vlan ID's 350 - 400 assigned, and br-ex</code> has vlan ID's 340 - 349 assigned. Following the above knowledge, my network_vlan_ranges</code> is configured as such: network_vlan_ranges = tenant1:350:400,provider1:340:349</code></pre> Make sure to restart all neutron services: openstack-service restart neutron</code></pre>On the controller (where neutron-server</code> lives)</h2> On the controller we just need to make sure that our network_vlan_ranges</code> matches what is on the network node, with one exception, we do not list our provider1</code> vlan ranges since we don't want to make those available to accidentally be assigned when a regular tenant creates a new network. So our configuration should list: network_vlan_ranges = tenant1:350:400</code></pre> Make sure that all neutron services are restarted: openstack-service restart neutron</code></pre>Create the Neutron networks</h2> Now, as an administrative user we need to create the provider networks. source ~/keystonerc_admin neutron net-create "192.168.1.0/24-floating" \ --router:external True \ --provider:network_type vlan \ --provider:physical_network provider1 \ --provider:segmentation_id 340 neutron net-create "192.168.2.0/24-floating" \ --router:external True \ --provider:network_type vlan \ --provider:physical_network provider1 \ --provider:segmentation_id 341</code></pre> Notice how we've created two networks, given them each individual names (I like to use the name of the network they are going to be used for) and have been attached to the provider1</code>. Note that provider1</code> is completely administratively defined, and could just as well have been physnet1</code>, so long as it is consistent across all of the configuration files. Now let's create subnets on this network: neutron subnet-create "192.168.1.0/24-floating" 192.168.1.0/24 \ --allocation-pool start=192.168.1.4,end=192.168.1.254 \ --disable-dhcp --gateway 192.168.1.1 neutron subnet-create "192.168.2.0/24-floating" 192.168.2.0/24 \ --allocation-pool start=192.168.2.4,end=192.168.2.254 \ --disable-dhcp --gateway 192.168.2.1</code></pre> Now that these networks are defined, you should be able to have tenants create routers and set their gateways to either of these new networks by selecting from the drop-down in Horizon or by calling neutron router-gateway-set <router id> <network id></code> on the command line. The l3 agent will automatically configure and set up the router as required on the network node, and traffic will flow to either vlan 340 or vlan 341 as defined above depending on what floating network the user uses as a gateway. This drastically simplifies the configuration of multiple floating IP networks since no longer is there a requirement to start up and configure multiple l3 agents each with their own network ID configured. This makes configuration less brittle and easier to maintain over time. OpenStack resizing of instances 2014-09-29T17:10:00+00:00 One thing that is not always adequately explained in the OpenStack documentation is how exactly instance resizing works, and what is required, especially while using KVM as the virtualisation provider, with multiple compute nodes. You might find something similiar to the following in your logs, and no good documentation on how to fix it. ERROR nova.compute.manager [req-7cb1c029-beb4-4905-a9d9-62d488540eda f542d1b5afeb4908b8b132c4486f9fa8 c2bfab5ad24642359f43cdff9bb00047] [instance: 99736f90-db0f-4cba-8f44-a73a603eee0b] Setting instance vm_state to ERROR TRACE nova.compute.manager [instance: 99736f90-db0f-4cba-8f44-a73a603eee0b] Traceback (most recent call last): TRACE nova.compute.manager [instance: 99736f90-db0f-4cba-8f44-a73a603eee0b] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 5596, in _error_out_instance_on_exception TRACE nova.compute.manager [instance: 99736f90-db0f-4cba-8f44-a73a603eee0b] yield TRACE nova.compute.manager [instance: 99736f90-db0f-4cba-8f44-a73a603eee0b] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 3459, in resize_instance TRACE nova.compute.manager [instance: 99736f90-db0f-4cba-8f44-a73a603eee0b] block_device_info) TRACE nova.compute.manager [instance: 99736f90-db0f-4cba-8f44-a73a603eee0b] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4980, in migrate_disk_and_power_off TRACE nova.compute.manager [instance: 99736f90-db0f-4cba-8f44-a73a603eee0b] utils.execute('ssh', dest, 'mkdir', '-p', inst_base) TRACE nova.compute.manager [instance: 99736f90-db0f-4cba-8f44-a73a603eee0b] File "/usr/lib/python2.7/site-packages/nova/utils.py", line 165, in execute TRACE nova.compute.manager [instance: 99736f90-db0f-4cba-8f44-a73a603eee0b] return processutils.execute(*cmd, **kwargs) TRACE nova.compute.manager [instance: 99736f90-db0f-4cba-8f44-a73a603eee0b] File "/usr/lib/python2.7/site-packages/nova/openstack/common/processutils.py", line 193, in execute TRACE nova.compute.manager [instance: 99736f90-db0f-4cba-8f44-a73a603eee0b] cmd=' '.join(cmd)) TRACE nova.compute.manager [instance: 99736f90-db0f-4cba-8f44-a73a603eee0b] ProcessExecutionError: Unexpected error while running command. TRACE nova.compute.manager [instance: 99736f90-db0f-4cba-8f44-a73a603eee0b] Command: ssh 10.5.2.20 mkdir -p /var/lib/nova/instances/99736f90-db0f-4cba-8f44-a73a603eee0b TRACE nova.compute.manager [instance: 99736f90-db0f-4cba-8f44-a73a603eee0b] Exit code: 255 TRACE nova.compute.manager [instance: 99736f90-db0f-4cba-8f44-a73a603eee0b] Stdout: '' TRACE nova.compute.manager [instance: 99736f90-db0f-4cba-8f44-a73a603eee0b] Stderr: 'Host key verification failed.\r\n' TRACE nova.compute.manager [instance: 99736f90-db0f-4cba-8f44-a73a603eee0b] ERROR oslo.messaging.rpc.dispatcher [-] Exception during message handling: Unexpected error while running command. Command: ssh 10.5.2.20 mkdir -p /var/lib/nova/instances/99736f90-db0f-4cba-8f44-a73a603eee0b Exit code: 255 Stdout: '' Stderr: 'Host key verification failed.\r\n'</code></pre> When OpenStack's nova is instructed to resize an instance it will also change the host it is running on, almost never will it schedule the instance on the same host and do the resize on the same host it already exists. There is a configuration flag to change this, however in my case I would rather the scheduler be run again, especially if the instance size is changing drastically. During the resize process, the node where the instance is currently running will use SSH to connect to the instance where the resized instance will live, and copy over the instance and associated files. There are a couple of assumptions I will be making: Your nova</code>, and qemu</code> user both have the same UID on all compute nodes</li> The path for your instances is the same on all of your compute nodes</li> </ol> Configure the nova user</h2> First things first, let's make sure our nova</code> user has an appropriate shell set: cat /etc/passwd | grep nova</code></pre> Verify that the last entry is /bin/bash</code>. If not, let's modify the user and make it so: usermod -s /bin/bash nova</code></pre>Generate SSH key and configuration</h2> After doing this the next steps are all run as the nova</code> user. su - nova</code></pre> We need to generate an SSH key: ssh-keygen -t rsa</code></pre> Follow the directions, and save the key WITHOUT a passphrase. Next up we need to configure SSH to not do host key verification, unless you want to manually SSH to all compute nodes that exist and accept the key (and continue to do so for each new compute node you add). cat << EOF > ~/.ssh/config Host * StrictHostKeyChecking no UserKnownHostsFile=/dev/null EOF</code></pre> Next we need to make sure we copy the the contents of id_rsa.pub</code> to authorized_keys</code> and set the mode on it correctly. cat ~/.ssh/id_rsa.pub > .ssh/authorized_keys chmod 600 .ssh/authorized_keys</code></pre> This should be all the configuration for SSH you need to do. Now comes the import part, you will need to tar up and copy the ~nova/.ssh</code> directory to every single compute node you have provisioned. This way all compute nodes will be able to SSH to the remote host to run the commands required to copy an instance over, and resize it. Reset state on existing ERROR'ed instances</h2> If you have any instances that are currently in the ERROR</code> state due to a failed resize, you will be able to issue the following command to reset the state back to running and try again: nova reset-state --active <ID of instance></code></pre> This will start the instance, and you will be able to once again issue the resize command to resize the instance. Building custom ports with Poudriere and Portshaker 2013-12-11T00:10:11+00:00 Guest post by Scott Sturdivant</a>. Maintaining custom ports and integrating them into your build process doesn't need to be difficult. The documentation surrounding this process however is either non-existent, or lacking in its clarity. At the end of the day, it really is as simple as maintaining a repository whose structure matches the ports tree layout, then managing that repository and the standard ports tree with portshaker</a>, and finally handing the end result off to poudriere</a>. Your Custom Repository</h2> For this example, we'll assume a git</a> repo is used and that you're already familiar with how to build FreeBSD</a> ports</a>. We'll also assume that we have but a single port that we're maintaining and that it is called myport</code>. The hierarchy of your repo should simply be category/myport</code>. We'll refer to this repo simply as myrepo</code>. Portshaker</h2> Portshaker is the tool responsible for taking multiple ports sources and then merging them down into a single target. In our case, we have two sources: our git repo (myrepo</code>) containing myport</code>, and the standard FreeBSD ports tree. We aim to merge this down into a single ports tree that poudriere will then use for its builds. To configure portshaker, add the following to the /usr/local/etc/portshaker.conf</code> file: # vim:set syntax=sh: # $Id: portshaker.conf.sample 116 2008-09-30 16:15:02Z romain.tartiere $ #---[ Base directory for mirrored Ports Trees ]--- mirror_base_dir="/var/cache/portshaker" #---[ Directories where to merge ports ]--- ports_trees="default" use_zfs="no" poudriere_ports_mountpoint="/usr/local/poudriere/ports" default_poudriere_tree="default" default_merge_from="freebsd myrepo"</code></pre> Some key points here are that the two items listed in for the default_merge_from</code> argument need to have scripts present in the /usr/local/etc/portshaker.d</code> directory. Further more, the combination of the poudriere_ports_mountpoint</code> and default_poudriere_tree</code> needs to be a ports tree that is then registered with poudriere. Next, we need to tell portshaker how to go off and fetch our two types of ports trees, freebsd</code> and myrepo</code>. For the freebsd</code> ports tree, create /usr/local/etc/portshaker.d/freebsd</code> with the following contents and make it executable: #!/bin/sh . /usr/local/share/portshaker/portshaker.subr method="portsnap" run_portshaker_command $*</code></pre> Next, create a similar script to handle our repository containing our custom port. /usr/local/etc/portshaker.d/myrepo</code> should contain the following and similarly be executable: #!/bin/sh . /usr/local/share/portshaker/portshaker.subr method="git" git_clone_uri="http://github.com/scottsturdivant/packaging.git" git_branch="master" run_portshaker_command $*</code></pre> Obviously replace the git_clone_uri</code> and git_branch</code> variables to reflect your actual configuration. For more information about the values and what they can contain, consult man portshaker.d</code> Now, portshaker should be all set. Execute portshaker -U</code> to update your merge_from</code> ports trees (freebsd</code> and myrepo</code>). You'll see the standard portsnap</a> fetch and extract process as well as a git clone. After a good bit of time, these will both be present in the /var/cache/portshaker</code> directory. Go ahead and merge them together by executing portshaker -M</code>. Hooray! You now have /usr/local/poudriere/ports/default/ports</code> that is a combination of the normal ports tree and your custom one. We're effectively complete with configuring portshaker. Whenever your port is updated, just re-run portshaker -U</code> and portshaker -M</code> to grab the latest changes and perform the merge. Poudriere</h2> Poudriere is a good tool for building ports. We will use it to handle our merged directory. Begin by configuring poudriere (/usr/local/etc/poudriere.conf</code>): NO_ZFS=yes FREEBSD_HOST=ftp://ftp.freebsd.org RESOLV_CONF=/etc/resolv.conf BASEFS=/usr/local/poudriere USE_PORTLINT=no USE_TMPFS=yes DISTFILES_CACHE=/usr/ports/distfiles CHECK_CHANGED_OPTIONS=yes</code></pre> Really there's nothing here that is specific to the problem at hand, so feel free to consult the provided configuration file to tune it to your needs. Now, the step that is specific is to set poudriere up with a ports tree that it does not manage, specifically our resultant merged directory. If you consult man poudriere</code>, it specifies that for the ports</code> subcommand, there is a -m method</code> switch which controls the methodology used to create the ports tree. By default, it is portsnap. This is confusing as in our case, we do not want poudriere to actually do anything. We want it to just use an existing path. Fortunately, there is a way! The poudriere wiki has an entry for using the system ports tree</a>, so we adopt it for our needs by executing: poudriere ports -c -F -f none -M /usr/local/poudriere/ports/default \ -p default</code></pre> If you've consulted the poudriere manpage, you'll see that the -F</code> and -f</code> switches both reference ZFS in their help. As we're not using ZFS, it's not clear how they will behave. However, in conjunction with the custom mountpoint (-M /usr/local/poudriere/ports/default</code>), we ultimately wind up with what we want, a ports tree that poudriere can use, but does not manage: # poudriere ports -l PORTSTREE METHOD PATH default - /usr/local/poudriere/ports/default</code></pre> Note that this resulting PATH is the combination of the poudriere_ports_mountpoint</code> and default_poudriere_tree</code> variables present in our /usr/local/etc/portshaker.conf</code> configuration file. Building software from your custom ports tree</h2> Go ahead and create your jail(s) like you normally would (i.e. poudriere -c -j 92amd64 -V 9.2-RELEASE -a amd64</code>) and any other configuration you would like, and then go ahead and build myport</code> with poudriere bulk -j 92amd64 -p default category/myport</code>. Success! FreeBSD 8.2 to 9.0 update with ZFS on root (mirror) 2012-04-24T18:36:20+00:00 Upgrading from one version of FreeBSD to another has become much simpler with freebsd-update</code></a> than before, a full rebuild of the FreeBSD kernel/world source used to be required (other upgrade paths existed but were not widely touted). freebsd-update</code> works extremely well when using UFS as your file system, there are however some gotcha's that one needs to look out for when using FreeBSD Root on ZFS. Two machines needed upgrading, both are running with a ZFS root mirror on FreeBSD 8.2-RELEASE. I had followed the FreeBSD Root on ZFS (mirror)</a> wiki article and everything has been running smoothly ever since. pool: zroot state: ONLINE scrub: scrub completed after 0h35m with 0 errors on Mon Mar 5 10:58:19 2012 config: NAME STATE READ WRITE CKSUM zroot ONLINE 0 0 0 mirror ONLINE 0 0 0 gpt/disk0 ONLINE 0 0 0 gpt/disk1 ONLINE 0 0 0 errors: No known data errors</code></pre> Wanting to take advantage of some of the advancements made in FreeBSD 9.0-RELEASE</a> such as the updated ZFS, a bug that was fixed that stopped jails from owning a ZFS file system, improved dtrace, and better IPv6 support required that the plunge be taken and an upgrade was in order. There were a couple reports of people having done source upgrades and then having issues booting their system, but I found one mailling list post</a> that suggested that everything went well using freebsd-update</code>, so that is the route I went with. I read through the entirety of the FreeBSD 9.0-ERLEASE Release Notes</a> before moving forward to make sure I didn't miss any important changes that would cause my system to not restart correctly or would require changing system configuration files before attempting the upgrade. I would advise you to do the same. I ran freebsd-update -r 9.0-RELEASE upgrade</code> and after accepting the list of components that were installed, I got an error saying: # freebsd-update -r 9.0-RELEASE upgrade Looking up update.FreeBSD.org mirrors... 4 mirrors found. Fetching metadata signature for 8.2-RELEASE from update2.freebsd.org... done. Fetching metadata index... done. Inspecting system... done. The following components of FreeBSD seem to be installed: kernel/generic src/base src/bin src/cddl src/contrib src/crypto src/etc src/games src/gnu src/include src/krb5 src/lib src/libexec src/release src/rescue src/sbin src/secure src/share src/sys src/tools src/ubin src/usbin world/base world/catpages world/dict world/doc world/games world/info world/lib32 world/manpages The following components of FreeBSD do not seem to be installed: world/proflibs Does this look reasonable (y/n)? y Fetching metadata signature for 9.0-RELEASE from update2.freebsd.org... done. Fetching metadata index... done. The update metadata is correctly signed, but failed an integrity check. Cowardly refusing to proceed any further.</code></pre> Specifically the line stating "Cowardly refusing to proceed further", a quick Google</a> later I found the 9.0-RC1 mailling list post</a> that suggested the following one line fix: sed -i '' -e 's/=_/=%@_/' /usr/sbin/freebsd-update</code></pre> After that quick fix, a re-run of freebsd-update</code> functioned without issues and finished, showing me what it would change, asking me if it seemed acceptable and telling me what it was going to remove and what was going to be installed. Next we run freebsd-update</code> for the first time in this three step process: freebsd-update install</code></pre> Don't restart yet (although it tells you to)! We need to go back and make sure we update the boot records first. Going back to the mailling list post made by Magnus Strahlert</a>, I made note of the following: Had to upgrade the gpt bootcode as instructed when upgrading the zpool for the system to boot. </blockquote> This is the only comment that worried me, and nowhere in the email thread does the poster describe what commands he ran. In any case I figure he meant re-running the boot code commands from the FreeBSD Root on ZFS wiki article: gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da0 gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da1</code></pre> After running these commands, it told me that it had updated the boot code and then I ran: shutdown -r now</code></pre> After the machine came back online (it is in those moments when you have a small heartattack when the machine doesn't come up as fast as you had expected ...) you log back in and run: freebsd-update install</code></pre> This will install all of the updated binaries that came with FreeBSD 9.0, at this point FreeBSD update will tell you to upgrade all installed software and ports, and then run freebsd-update</code> one more time to remove all unnecessary shared libraries. If you want to have certain software continue functioning, yet you want freebsd-update</code> to remove any other remaining old 8.x files, you can install misc/compat8x</code></a> from the ports tree. This is the suggested route to go if you want to keep the 8.x libraries around for compatibility reasons. cd /usr/ports/misc/compat8x make install clean # or # portmaster misc/compat8x</code></pre> I use portmaster</code> and the following command will simply rebuild every port installed on the system: portmaster -aPf</code></pre> The -P</code> to portmaster</code> means look for package and if available use it, otherwise build from port. Run freebsd-update</code> one last time: freebsd-update install</code></pre> Since we are running ZFS we will most likely also want to upgrade our pool to the latest version of ZFS to take advantage of any new goodies (such as deduplication!), upgrading a zpool is easy: zpool upgrade zroot</code></pre> That upgrades to the latest pool version, now we will want to upgrade our existing file systems to the latest version of ZFS, do note that this will make them incompatible with older file system verisons (so if you use zfs send for backups, your other endpoint will need to be of the same ZFS version) zfs upgrade # List all ZFS file systems that need to be upgraded zfs upgrade -a # Upgrade all file systems</code></pre> After this one last reboot to make sure that all the processes come back online using the newer libraries and that you didn't accidentally miss something. shutdown -r now</code></pre> At this point the system has been upgraded from FreeBSD 8.0-RELEASE to FreeBSD 9.0-RELEASE and has all of the latest updates and enhancements. Converting KVM virtual machines to VirtualBox 2012-04-01T01:03:32+00:00 Recently the requirement came up to take a KVM</a> based virtual machine and move it over to a VirtualBox</a> image. Which turned out to be a fairly simple endeavour, and was fairly painless. The longest part was transferring over the 40 GB image from one machine to the other where the conversion could take place. The machine the image was coming from was only on a 100 Mbit/sec connection so that took a good hour. Converting from KVM to VirtualBox for a FreeBSD image was pretty simple, the VBoxManage</code> command has a convertdd</code> command that allows you to convert from raw disk .img</code> format to .vdi</code> format. VBoxManage convertdd KVM-image.img VB-image.vdi</code></pre> After this, unfortunately, there is no way to to automatically convert over the settings that the virtual machine had, such as the network cards, the memory allocations and hard drive settings. You will have to go to VirtualBox and create a new virtual machine and replicate all of the settings. Once that is done make sure to select the same type of disk controller (SATA or IDE) so that the drive will hopefully be assigned the same name in the device tree so that you don't need to alter your /etc/fstab</code>. Hopefully everything boots without any issues. If not try creating a new virtual machine, attach the converted image as a secondary drive and see if you can mount the converted image within your new install. If so maybe transferring the data using rsync</code> or dump</code>/restore</code> would be an option. NAT with PF on an interface with multiple IP addresses 2011-09-08T02:40:00+00:00 If you want to do NAT for your currently running jail instances on FreeBSD so that they can have outgoing connections you could try the following pf.conf</code> to set up NAT on the interface that the jails have their IP addresses on. ext_if="em0" # The network card your default gateway is on jail_if="lo1" # The interface that your jails have IP addresses on # Set some options set optimization aggressive set block-policy drop set skip on lo # NAT on the external interface when coming from the jail interface nat on $ext_if from $jail_if:network:0 to any -> ($ext_if) # We just pass everything pass quick all</code></pre> What I have done is create IP addresses within the private network range (10/8</code>, 172.16/12</code>, 192.168/16</code>) on the lo1</code> interface. These can't be in the 127/8</code> range because those addresses can't be NAT'ed (not sure if this is a limitation in pf or if this is a FreeBSD limitation), which is a shame because using 127.1.0.1/24</code> would be pretty awesome in my opinion. What I did find though is that the above will not work correctly if your main network card (the one your default gateway is on, in the example em0</code>) contains multiple IP addresses. At that point the syntax ($ext_if)</code> does not function correctly and will cause packet loss/drop. 1</a> So instead of using the syntax above we simply replace ($ext_if)</code> with the actual IP address of the interface. This takes care of the issue and will let your jails have proper internet access without issues. ext_if="em0" jail_if="lo1" set optimization aggressive set block-policy drop set skip on lo nat on $ext_if from $jail_if:network:0 to any -> 192.168.1.2 # Your IP! pass quick all</code></pre> The other way you can solve this problem off course is to provide your jails with IP addresses within the same range that go to your default gateway thereby solving the problem of needing to NAT in the first place. Do note that I am using FreeBSD 8.2, so this may be fixed in the new FreeBSD 9 that is due to be released soon. ↩</a> </li> </ol> </section> FreeBSD running within KVM: Serial Console 2011-08-21T18:53:52+00:00 Instead of using virtualisation technology such as VMWare of VirtualBox I decided to give the new KVM</a> technology that is included with Linux. It is fast, and has been extremely reliable. However I wanted to be able to access the console on the guest FreeBSD instances. I used virt-manager, a graphical tool, to set up everything, including creating the virtual machine, I didn't want to have to deal with XML configuration files and whatnot. After installing FreeBSD there are a few changes I made to get a virtual console working, this allows virsh console <domain></code> to function as expected. First we add a new file to the root file system (which should be the first slice on the drive): echo "-Dh" > /boot.config</code></pre> See man 5 boot.config</a> for more information. Basically -D says that FreeBSD should boot with a dual console configuration (built-in over VGA) and serial console, and -h says to force the serial console to on. If you were to reboot now and use the virsh command: virsh console FreeBSD</code></pre> you would see the system go through the boot process and show output starting from when the boot block gains control, at this point however you still have no way to login to this new console since we never specified that we wanted any ttys on the serial console, for that we need to edit /etc/ttys. We are looking for the ttyu0 entry, we want to make sure to change dialup to vt100, and off to on, so that it should look like this: ttyu0 "/usr/libexec/getty std.9600" vt100 on secure</code></pre> Now when we reboot once more we will get the usual login display that we have come to expect of console on FreeBSD, and we can login over the serial console. This way we don't have to open up the virtual machine manager to get a console, and can continue to do almost everything from a terminal window. PostgreSQL setup for external connections 2011-08-16T17:48:54+00:00 PostgreSQL is a fantastic open-source RDBMS which is extremely powerful, once installed on FreeBSD getting it set up is pretty simple. We do want to change some of the defaults though. By default the system will be set up to trust any user using the local socket to connect as any user to any database. By setting up the initdb</code></a> flags we can change how PostgreSQL generates the default configuration files. Adding -A md5</code> tells it to use the md5 authentication mechanism</a>, which requires that passwords sent over the wire are md5 hashed. Add the following to /etc/rc.conf</code></a> for the initdb flags, and enable PostgreSQL as a service: postgresql_initdb_flags="-D /usr/local/pgsql/data -W -A md5" postgresql_enable="YES"</code></pre> And then run: /usr/local/etc/rc.d/postgresql initdb</code></pre> This will initialise and create the configuration as well as setting up some of the default tables. You will want to navigate to the folder specified in the initdb_flags</code> above: cd /usr/local/pgsql/data</code></pre> this is where you will find the files that we are going to modify next to get PostgreSQL set up to listen to connections coming in on a private network (192.168.1.1/24</code>) and where we allow connections coming in from servers on that network. This is strictly for setting up PostgreSQL in a situation where you have multiple servers accessing it as a client on a private network. After this we need to modify a few files to get it to listen to TCP/IPv4 connections, first we start with postgresql.conf</code></a> and add: listen_addresses = '127.0.0.1,192.168.1.1'</code></pre> Then in pg_hba.conf</code></a> we add a new line: host all all 192.168.1.1/24 md5</code></pre> This allows anyone in the 192.168.1.1/24 range to connect to our PostgreSQL instance and to use the databases. This is exactly what we want to allow, now we start PostgreSQL using the rc script: /usr/local/etc/rc.d/postgresql start</code></pre> Setting up jails with multiple IPs and providing it with internet access 2011-08-15T15:40:04+00:00 I have multiple VMs up and running and generally I have an internal to the VMs only network set up, in the current project I am working on I have three completely different networks set up. DHCP assigned network that routes to the outside network on the host (192.168.0.1/24)</li> An internal to the VM's network interface (10.99.12.1/24)</li> A network that is tied to the host's wireless network (setup to be an AP) (10.99.11.1/24)</li> </ol> So within my FreeBSD VM I have the following: em0</code>, em1</code>, em2</code> as "network cards". In my FreeBSD VM, I am creating two jails that are going to contain different pieces of server software. The jails need IP's in the following ranges, 10.99.12.1/24 (to connect to another VM), 10.99.11.1/24 (Wifi network) and then it needs to have an IP address that is considered "local" and we can NAT on, so in addition to the above networks we also needed to add another network. The easiest way to create new interface with IP addresses is using a bridge. They can be assigned IP addresses, which is all we need. At this point inside of a jail it kinda looks like this: em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC> ether 52:54:00:6b:6a:49 media: Ethernet autoselect (1000baseT <full-duplex>) status: active em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC> ether 52:54:00:2f:a5:55 inet 10.99.12.4 netmask 0xffffffff broadcast 10.99.12.4 media: Ethernet autoselect (1000baseT <full-duplex>) status: active em2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC> ether 52:54:00:bb:b6:43 inet 10.99.11.14 netmask 0xffffffff broadcast 10.99.11.14 media: Ethernet autoselect (1000baseT <full-duplex>) status: active lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 options=3<RXCSUM,TXCSUM> bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 ether da:de:63:c9:55:6c inet 10.0.1.4 netmask 0xffffffff broadcast 10.0.1.4 id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 100 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0</code></pre> em0</code> doesn't have an address since that is our DHCP address, em1</code>, em2</code> and bridge0</code> all have IP addresses, in the jail configuration the bridge0</code> IP address is listed first, and thus when something binds or sends it sends from that IP address. The thing is that at this point the jail does not have internet access. Jails inherit the routing table from their host environment (yes, you can compile in more routing tables ... but I don't want to modify stock), any traffic that goes to the outside is going to go out over the default route, thing is you'd expect that traffic to show up on bridge0</code> where the IP address exists, instead using tcpdump</code> you see it show up on em0</code>, which is where the default route is. I would have expected having to do NAT on the bridge, not just the external interface. Here is the pf</code> configuration. ext_if="em0" jail_if="bridge0" set optimization aggressive set block-policy drop set skip on lo nat on $ext_if from $jail_if:network:0 to any -> ($ext_if) pass quick all</code></pre> And we've got internet within our jail. Apache mod_fastcgi and PHP with PHP-FPM 2010-11-14T23:37:27+00:00 Recently I did a server migration from an older server to a newer server and in an attempt to help stability I wanted to see if there was a better way to do PHP FastCGI. In my research I came across running PHP using the FastCGI server that spins up a PHP on a TCP/IP port and allows the web server to connect to it. However this doesn't help with spawning or keeping track of instances or error recovery. This is where PHP-FPM</a> comes in handy. It does all of the hard work for us, it spawns the processes and has a bunch of really awesome features that help run PHP as various different users as required with different PHP ini files and memory limits. PHP-FPM's defaults, at least from a ports install, are extremely sane and I don't really suggest changing them. After setting up PHP FPM I had to set up Apache. mod_fcgid</code> doesn't allow for remote connections, and as such I was unable to use it for what I needed it for. mod_fastcgi</code> provides the FastCGIExternalServer</a> configuration key, which is exactly what I needed. "The FastCgiExternalServer directive defines filename as an external FastCGI application. If filename does not begin with a slash (/</code>) then it is assumed to be relative to the ServerRoot</code>. The filename does not have to exist in the local filesystem. URIs that Apache resolves to this filename will be handled by this external FastCGI application." </blockquote> What this documentation does not state is that the path up to the last part of it has to exist in the local file system. So in my first couple of attempts I pointed it at /usr/local/www/fastcgi/php5.fcgi</code> without having an actual fastcgi directory located in /usr/local/www/</code>. There are a lot of examples that require creating a new FastCGIExternalServer</code> for each and every VirtualHost</code> this is unacceptable to me, the reason they require it is because they set the FastCGIExternalServer</code> path to the folder where they are going to be serving files from. In the end I found that after creating the /usr/local/www/fastcgi</code> directory (and reading the mod_fastcgi</code> source code) that all it does is make Apache believe a file exists in a certain directory, much like Alias</code>, except Alias</code> allows full paths to be aliased, not so here. The AddType</code> and Action</code> add custom types and what the action should be when such a type is encountered. In this case the action is to redirect the request to /php5.fcgi</code> which will handle the rest of the request. This does not require another FastCGI section per VirtualHost</code> as each PHP request will just get shuttled to the php handler. Do take note that I have specifically disallowed Apache to serve anything from the /usr/local/www/fastcgi/</code> folder, except for a single file php5.fcgi</code> which is our FastCGIExternalServer</code> file. <IfModule mod_fastcgi.c> Alias /php5.fcgi /usr/local/www/fastcgi/php5.fcgi FastCGIExternalServer /usr/local/www/fastcgi/php5.fcgi -flush -host 127.0.0.1:9000 AddType application/x-httpd-fastphp5 .php Action application/x-httpd-fastphp5 /php5.fcgi <Directory "/usr/local/www/fastcgi/"> Order deny,allow Deny from all <Files "php5.fcgi"> Order allow,deny Allow from all </Files> </Directory> </IfModule></code></pre> Note that even-though in my last post</a> concerning mod_fastcgi</code> I as moving away from it, I am now doing the opposite, instead of moving from mod_fastcgi</code> to mod_fcgid</code></a> I'm back to mod_fastcgi</code>, only because mod_fcgid</code> doesn't offer the same functionality. So far this has provided far more stability, along with PHP-FPM doing all of the process management I can now use a single PHP instance that is running on a single port for the various web servers I am testing. At the moment I have both Lighttpd and Apache using the same PHP-FPM instance. It is faster, less memory is wasted and PHP-FPM is much better at process management than mod_fastcgi</code> or mod_fcgid</code>. Moving from mod_fastcgi to mod_fcgid 2009-08-25T23:22:03+00:00 Update: I've since moved to a PHP-FPM with mod_fastcgi</code> based setup which works much better than mod_fcgid</code> could ever dream of, taking process management away from Apache and putting it where it actually makes sense has improved server load and improved website speed. See my new PHP-FPM with mod_fastcgi post</a> for more information. -- Errors, all they cause is trouble. The dreaded 500 error showed up when visiting my favourite tech website, the one I am the administrator for. This time I had enough, there was going to be no more playing around with PHP settings, attempting to figure out why PHP was suddenly dying and or why mod_fastcgi</code> refused to retry a select() when it failed due to a system signal. The server in question runs Apache in MPM worker mode since a threaded Apache is going to be faster than a pre-fork, besides this server does not have as much memory so it seemed to be better to have multiple threads rather than multiple processes, each of which would have their own memory segment. There however is the issue that this server also needs to run PHP, the accepted method to do so is to use mod_php</code> along with Apache; however PHP is not thread safe. The alternative is FastCGI, basically it spawns PHP processes as a separate stand-alone process, with their own memory space, much like Apache pre-fork, however now when one process grows to big, or when a process is no longer needed it can be cleaned up, keeping memory to a minimum. Also, FastCGI is perfectly thread safe, this means that with Apache running in MPM worker mode we could now still run our PHP scripts even when they were not thread safe. Setting up mod_fastcgi</code> is not that hard, it takes some httpd.conf configuration values, and off course the loadmodule is assumed here: # Set up mod_fastcgi <IfModule `mod_fastcgi`.c> FastCgiIpcDir /var/tmp/fcgi-ipc/ FastCgiConfig -autoUpdate -singleThreshold 100 -killInterval 300 -idle-timeout 240 -pass-header HTTP_AUTHORIZATION AddHandler fastcgi-script .fcgi .fcg .fpl Action application/x-httpd-php5 /fastcgi-bin/php5.fcgi AddType application/x-httpd-php5 .php .php5 </IfModule> # Set up the script alias, basically anything in this directory gets executed as a fastcgi script ScriptAlias /fastcgi-bin/ "/usr/local/www/fastcgi-bin/" <Location /fastcgi-bin/> Options ExecCGI SetHandler fastcgi-script Order allow,deny Allow from all </Location></code></pre> Because of errors that mod_fastcgi</code> was throwing out at me, I figured mod_fcgid</code> is worth a try. However all of the configurations I found required me to add an FCGIWrapper line into each of my VirtualHost blocks, which was an immediate no-no since there is well over 100 of those on the server, and I'd rather spend my time doing other things. With some trial and error, and some Googling I put the following together for mod_fcgid</code>, tested it out, and once it worked perfectly set Apache off running with the following configuration: # Set up mod_fcgid <IfModule `mod_fcgid`.c> AddHandler fcgid-script .fcgi .fcg .fpl IPCCommTimeout 60 SocketPath /var/tmp/fcgi-ipc/ Action application/x-httpd-php5 /fastcgi-bin/php5.fcgi AddType application/x-httpd-php5 .php .php5 </IfModule> # Set up the script alias, basically anything in this directory gets executed as a fastcgi script ScriptAlias /fastcgi-bin/ "/usr/local/www/fastcgi-bin/" <Location /fastcgi-bin/> Options ExecCGI SetHandler fcgid-script Order allow,deny Allow from all </Location></code></pre> Notice how similar they both are, that was the whole goal. mod_fcgid</code> was supposed to be a drop in replacement, and thankfully it was. PHP was served as it once was, with a twist. The php5.fcgi script is as follows for those of you trying to set this up as well: #!/bin/sh # To use your own php.ini, comment the next line and uncomment the following one #PHPRC="/usr/local/etc" #export PHPRC PHP_FCGI_CHILDREN=4 export PHP_FCGI_CHILDREN exec /usr/local/bin/php-cgi</code></pre> Some things that that intrigued me is that my php processes seemed to be capped now. Where mod_fastcgi</code> would spawn processes but never remove them or kill them when they were no longer needed, mod_fcgid</code> keeps the running php processes to a sane limit instantly starting new processes when required to handle the requests coming in. This means that the server now has more free memory for MySQL, or file caches which has helped speed up other types of transfers as well. Even with the relatively short time that mod_fcgid</code> has been in place it has been faster, more reliable and more sane than mod_fastcgi</code>. Whichever one you pick, make sure to benchmark your server and check your error logs to solve common issues, the issue may not be what FastCGI module you picked but rather PHP itself that is causing the errors! Hosting NFS shares from a Zone 2009-02-21T00:10:26+00:00 I am currently trying to get my web development environment all set up. While I like using vi, I really do prefer TextMate on my Mac OS X to do the real heavy lifting and all of the editing. As I was working on getting my homedir on my zone to be shared over NFS I was getting errors that NFS was not a valid protocol from sharemgr</code>, the program that is used to add new shares to the system. xistence@webdev.network.lan:~# sharemgr create -P nfs homedir Invalid protocol specified: nfs</code></pre> A little bit of Googling brought me to a mailing list message saying that some server packages may be missing. This is entirely possible since my Zone was created with a minimal set of packages, so I log into the global host where NFS sharing does work and run the following: xistence@Keyhole.network.lan:~# pkg list | grep nfs SUNWnfsc 0.5.11-0.101 installed ---- SUNWnfsckr 0.5.11-0.101 installed ---- SUNWnfss 0.5.11-0.101 installed ---- SUNWnfsskr 0.5.11-0.101 installed ----</code></pre> Ah, so those are probably not installed in the zone, quick pkg install</code> later: xistence@webdev.network.lan:~# pkg install SUNWnfsc SUNWnfsckr SUNWnfss SUNWnfsskr DOWNLOAD PKGS FILES XFER (MB) Completed 3/3 17/17 0.28/0.28 PHASE ACTIONS Install Phase 55/55 PHASE ITEMS Reading Existing Index 9/9 Indexing Packages 3/3</code></pre> Now, lets re-run the sharemgr</code> command again and see if this time we do indeed get our group created: xistence@webdev.network.lan:~# sharemgr create -P nfs homedir xistence@webdev.network.lan:~# sharemgr show -vp default nfs=() zfs homedir nfs=()</code></pre> Yes, there we go. Now I can add a share to that group. Until more errors start showing up: xistence@webdev.network.lan:~# sharemgr add-share -s /export/home/xistence/ homedir NFS: Cannot share remote filesystem: /export/home/xistence NFS: Service needs to be enabled by a privileged user Failed to enable share for "nfs": system error</code></pre> This time Google does not help out, instead it gives me the option to search without quotes around the search terms which gives me absolutely nothing. Well, lets see if the services are running in the first place: xistence@webdev.network.lan:~# svcs -a | grep nfs disabled 22:19:04 svc:/network/nfs/client:default disabled 22:24:26 svc:/network/nfs/server:default online 22:24:20 svc:/network/nfs/rquota:default online 22:24:20 svc:/network/nfs/status:default online 22:24:20 svc:/network/nfs/mapid:default online 22:24:20 svc:/network/nfs/cbd:default online 22:24:21 svc:/network/nfs/nlockmgr:default</code></pre> Nope, for some reason the nfs server keeps being disabled even though I did enable it using svcadmin</code>. Okay, lets take a look with svcs</code> what is going on: xistence@webdev.network.lan:/# svcs -l nfs/server fmri svc:/network/nfs/server:default name NFS server enabled false (temporary) state disabled next_state none state_time Fri Feb 20 22:25:26 2009 logfile /var/svc/log/network-nfs-server:default.log restarter svc:/system/svc/restarter:default contract_id dependency require_any/error svc:/milestone/network (online) dependency require_all/error svc:/network/nfs/nlockmgr (online) dependency optional_all/error svc:/network/nfs/mapid (online) dependency require_all/restart svc:/network/rpc/bind (online) dependency optional_all/none svc:/network/rpc/keyserv (disabled) dependency optional_all/none svc:/network/rpc/gss (absent) dependency optional_all/none svc:/network/shares/group (multiple) dependency require_all/error svc:/system/filesystem/local (online)</code></pre> Ah, it has a log file, lets see what that log file says: xistence@webdev.network.lan:/# cat /var/svc/log/network-nfs-server\:default.log [ Feb 20 22:11:07 Disabled. ] [ Feb 20 22:11:07 Rereading configuration. ] [ Feb 20 22:18:43 Enabled. ] [ Feb 20 22:19:04 Enabled. ] [ Feb 20 22:24:21 Executing start method ("/lib/svc/method/nfs-server start"). ] The NFS server is not supported in a local zone</code></pre> Eh? What? That seems weird, but it is of course plausible. Zones are new to Solaris and thus not everything may be virtualised yet. Taking that query to google brings me to a web page asking the same question I am asking: Local zone as NFS server. Taking a look it brings me to a bug report and ultimately to: Bug ID 4964859 RFE: Zones should be able to be NFS servers. Well, I guess that ends that little stunt. I will be sharing it from the global zone. While this is acceptable to me, I do not believe that it is acceptable to most other people using Zones, and while it is unfortunate but it does not look like there is a whole lot of activity on the bug, as apparently there are no resources available. I ended up doing the following: sharemgr create -P nfs webdev-homedir</code></pre> Then I followed that up with some servers that should be allowed access, and others that should not: sharemgr set -P nfs -S sys -p 'rw=10.10.10.226 none=*' webdev-homedir</code></pre> And then finally I added the share I wanted to share all along: sharemgr add-share -r "homedir" -d "xistence home dir" -s /storage/zones/dev-web/root/export/home/xistence webdev-homedir</code></pre> This creates a long NFS share to type in, since the full path now becomes: nfs://keyhole/storage/zones/dev-web/root/export/home/xistence</code></pre> Which is not that big of a problem for me, especially since I just save it in my favourites in Mac OS X and everything is peachy. It works like a charm. Upgrading an OpenSolaris ipkg zone 2009-01-18T00:10:02+00:00 Recently I was working on my OpenSolaris machine (file server, ZFS rocks, will write more on that later), and I have one non-global zone, one that use for web development, which is aptly called dev-web. So the following shows up when I run zoneadm: # zoneadm list -iv ID NAME STATUS PATH BRAND IP 0 global running / native shared 4 dev-web running /storage/zones/dev-web ipkg shared</code></pre> The thing is, I had upgraded the global zone with the latest update available for the version (snv_101b</code>): pkg image-update -v</code></pre> This had not upgraded my one none-global zone. And running pkg image-update</code> from within the zone itself is not possible, because you can't do an upgrade on a "live" system, mainly because the image-update</code> wants to create a new bootable environment, something that was already created when I upgraded the global zone. So what we have to do is mount the non-global zone to /mnt</code> and tell pkg</code> with -R</code> where to find it and upgrade it anyway! First we are going to halt the current zone, important is running on the test system, I just used: zoneadm -z dev-web halt</code></pre> However, the better way is to off course use the shutdown command: zlogin dev-web shutdown</code></pre> and then for good measure a halt! Next up, looking at what we are going to be mounting. # zfs list NAME USED AVAIL REFER MOUNTPOINT rpool 11.9G 42.7G 75K /rpool rpool/ROOT 2.93G 42.7G 18K legacy rpool/ROOT/opensolaris 7.34M 42.7G 2.74G / rpool/ROOT/opensolaris-1 2.92G 42.7G 2.74G / rpool/dump 1.87G 42.7G 1.87G - rpool/export 5.27G 42.7G 19K /export rpool/export/home 5.27G 42.7G 21K /export/home rpool/export/home/guest 5.24G 14.8G 5.24G /export/home/guest rpool/export/home/xistence 37.0M 42.7G 37.0M /export/home/xistence rpool/swap 1.87G 43.0G 1.51G - storage 493G 3.08T 35.1K /storage storage/media 398G 3.08T 398G /storage/media storage/virtualbox 2.10G 3.08T 2.10G /storage/virtualbox storage/xistence 91.7G 3.08T 91.7G /storage/xistence storage/zones 957M 99.1G 30.4K /storage/zones storage/zones/dev-web 957M 19.1G 32.0K /storage/zones/dev-web storage/zones/dev-web/ROOT 957M 19.1G 28.8K legacy storage/zones/dev-web/ROOT/zbe 1.60M 19.1G 936M legacy storage/zones/dev-web/ROOT/zbe-1 955M 19.1G 935M legacy</code></pre> When pkg image-update</code> was run on the global zone it created a new bootable environment named opensolaris-1, the cool thing is, that beadm</code> at the same time will also create a new bootable environment for your zones. That way you can upgrade your zones afterwards, and if stuff does not work, you can revert the ENTIRE machine back to the previous state (ZFS is cool like that), thereby also making sure that your zones are reverted so that there are no incompatibilities. So what we are looking for in this case is a zbe-1, this is the new root for the zone that we need to update, so we now need to mount it. mount -F zfs storage/zones/dev-web/ROOT/zbe-1 /mnt</code></pre> Note, that there is no /</code> in front of storage, this is because we are specifying a pool name, since there is no "real" path that is defined as /storage/zones/dev-web/ROOT/zbe-1</code>. Now that it is mounted, we are able to pass the -R flag to pkg, to get it to update our zone: # pkg -R /mnt image-update -v Creating Plan / Before evaluation: UNEVALUATED: +pkg:/entire@0.5.11,5.11-0.101:20081204T010954Z After evaluation: pkg:/entire@0.5.11,5.11-0.101:20081119T235706Z -> pkg:/entire@0.5.11,5.11-0.101:20081204T010954Z Actuators: None PHASE ACTIONS Update Phase 1/1 PHASE ITEMS Reading Existing Index 9/9 Indexing Packages 1/1 --------------------------------------------------------------------------- NOTE: Please review release notes posted at: http://opensolaris.org/os/project/indiana/resources/relnotes/200811/x86/ ---------------------------------------------------------------------------</code></pre> Voilá, and the deed is done. The last command is to off course unmount the zone, that we can then issue a zoneadm boot command to start it back up: umount /mnt zoneadm -z dev-web boot</code></pre> And then on the zone after we log into it (over SSH in my case): xistence@webdev.network.lan:~# pkg list -u No installed packages have available updates</code></pre> Which is exactly what we wanted! Your zone is now upgraded with the latest version available from the global zone. Using DJB's daemontools and netcat to bounce a request around 2008-11-29T17:56:05+00:00 Lately I have been moving the data from one server to another server, located halfway across the globe, and I needed some way to forward all incoming requests from the old server to the new server. This had to be done so that DNS could take its time to update while everything was now already being served from the new location. What I ended up doing was using daemontools along with netcat to pipe the request around the world. Here are the steps I took in doing so: mkdir /usr/local/redirect/ cd /usr/local/redirect/ mkdir smtp smtp/env cd smtp</code></pre> Put the following in a file named run: #!/bin/sh exec 2>&1 \ envdir ./env \ sh -c ' case "$REMOTENAME" in h) H=;; p) H=p;; *) H=H;; esac case "$REMOTEINFO" in r) R=;; [0-9]*) R="t$REMOTEINFO";; *) R=R;; esac exec \ /usr/local/bin/tcpserver \ -vDU"$H$R" \ ${LOCALNAME+"-l$LOCALNAME"} \ ${BACKLOG+"-b$BACKLOG"} \ ${CONCURRENCY+"-c$CONCURRENCY"} \ -- "${IP-0}" "${PORT}" \ /usr/bin/nc "${REMOTEHOST}" "${REMOTEPORT}" '</code></pre> Make it executable: chmod +x run</code> Then we need to set up a few environment variables: cd env echo "example.net" > REMOTEHOST echo "25" > REMOTEPORT echo "25" > PORT echo `hostname` > LOCALNAME echo "200" > CONCURRENCY</code></pre> I made REMOTEPORT</code> and PORT</code> be separate on purpose, in one case I now had more IP's than before, so instead of having SSL running on a separate port it was running on the default port, and I needed a clean way to forward that. Now just add it to your services folder as a symlink and it will automatically be started, from there it will do it's job! I also suggest adding some simple logging, or discarding all off the output from tcpserver. FreeBSD GEOM striping and concatenation 2006-12-13T06:44:20+00:00 Some of the hard drives in my FreeBSD box. Just a little back-story: /dev/ad0 - 20 GB /dev/ad1 - 40 GB /dev/ad2 - 100 GB /dev/ad3 - 60 GB /dev/da0 - 8 GB /dev/da1 - 8 GB</code></pre> I had already made a gstripe(8)</code> out of the two SCSI drives, da[1|2]</code>, however I wanted to see if I could do the same with ad[1|2|3]</code>. As I did not want to resort to gvinum, which is rather heavy, and I was unable to figure out how it worked. The entire point behind gstripe is that since it is striped, the writes/reads are faster since it writes/reads to all the drives at the same time. So first I tried: gstripe label -v st1 /dev/ad1 /dev/ad2 /dev/ad3</code></pre> That however complained that it would only use 40 GB's on all of the drives, since the smallest drive in the collection was 40 GB. I then read up on gconcat(8)</code>, which concatenates all the drives together to look like one big drive, without the speed improvement of striping. So I figured, since FreeBSD's gstripe is GEOM based, it should not care about what the underlying data sink is, just that it is a device, so just maybe this could work: gconcat label -v gb100 /dev/ad1 /dev/ad3</code></pre> So now I had created a 100 GB "hd" at /dev/concat/gb100</code>. I now did what I was planning to do: gstripe label -v st1 /dev/ad2 /dev/concat/gb100</code></pre> No problem. I now had 180.48 GB, of which only about 166 GB was usable. Which is still excellent. As it still came with a speed improvement. Followed up with the standard advice from the handbook</a> I did the following: bsdlabel -wB /dev/stripe/st1 newfs -U /dev/stripe/st1a mount /dev/stripe/st1a /mnt</code></pre> Which got me the following: /dev/stripe/st1a 180G 5.5G 161G 3% /mnt</code></pre> Full: Filesystem Size Used Avail Capacity Mounted on /dev/ad0s1a 496M 63M 393M 14% / devfs 1.0K 1.0K 0B 100% /dev /dev/ad0s1e 496M 230K 456M 0% /tmp /dev/ad0s1f 15G 2.2G 11G 17% /usr /dev/ad0s1d 1.4G 115M 1.2G 9% /var /dev/stripe/st0a 16G 4.0K 15G 0% /usr/scsi /dev/stripe/st1a 180G 5.5G 161G 3% /mnt</code></pre> Next up I might actually add gdbe(8)</code> or geli(8)</code> even in front of the stripe, so that I will have an encrypted stripe. Talk about stacking. Might give gvinum a try sometime. All working: keyhole# gconcat status Name Status Components concat/gb100 UP ad1 ad3 keyhole# gstripe status Name Status Components stripe/st0 UP da0 da1 stripe/st1 UP ad2 concat/gb100</code></pre> FastCGI quick and dirty 2006-09-13T00:19:31+00:00 In this quick and dirty guide for FreeBSD I will assume you have already compiled and installed PHP5 with fcgi mode enabled, which can be checked by running: /usr/local/bin/php-cgi --version</code></pre> I will also assume you have installed mod_fastcgi</code> from www/mod_fastcgi</code> in the ports. Create the fast cgi ipc temp dir, make sure you place this somewhere it will keep existing: cd /var/tmp mkdir -p /var/tmp/fcgi-ipc/dynamic chmod -R 0777 /var/tmp/fcgi-ipc</code></pre> Let's create the directory where we are going to place our PHP wrapper, this config is for server wide using the same FastCGI PHP config. This is on purpose, since it will allow server wide PHP usage. If you want you could then split it up into virtualhosts if one required a standalone php running for that. mkdir /usr/local/www/fastcgi-bin/ chmod 755 /usr/local/www/fastcgi-bin/</code></pre> Next up we create the very simple wrapper: cat << EOF > /usr/local/www/fastcgi-bin/php5.fcgi #!/bin/sh # To use your own php.ini, comment the next line and uncomment the following one #PHPRC="/usr/local/etc" export PHPRC PHP_FCGI_CHILDREN=8 export PHP_FCGI_CHILDREN exec /usr/local/bin/php-cgi EOF</code></pre> Set PHP_FCGI_CHILDREN</code> to whatever value you want, that is how many processes it will use. I suggest the standard which is 8, if you make it more, you use more ram, you make it less Apache will have to wait if all of the PHP processes are in use. Next up, make it executable: chmod +x /usr/local/www/fastcgi-bin/php5.fcgi</code></pre> Last but not least, edit httpd.conf</code> and make sure to add a comment in front of the mod_php5</code> line, and uncomment the mod_fastcgi</code> line. Then add this to the end of your Apache config: <IfModule mod_fastcgi.c> FastCgiIpcDir /var/tmp/fcgi-ipc/ FastCgiConfig -pass-header HTTP_AUTHORIZATION AddHandler fastcgi-script .fcgi .fcg .fpl Action application/x-httpd-php5 /fastcgi-bin/php5.fcgi AddType application/x-httpd-php5 .php .php5 </IfModule> ScriptAlias /fastcgi-bin/ "/usr/local/www/fastcgi-bin/" <Location /fastcgi-bin/> Options ExecCGI SetHandler fastcgi-script Order allow,deny Allow from all </Location></code></pre> That's it. Restart apache and watch as the first time you access a PHP script FastCGI will spawn several of them, and they start serving requests. Load usage may seem higher, that is because unlike Apache the standalone running PHP's handle the jobs as they come in with whatever one is first, Apache will try to put as much work in one of its preforked modules. As the PHP processes come and go through FastCGI the memory usage will go up and down. Mine has an average of about 300 MB, which is very good compared to the 800 MB Apache with mod_php</code> would rack up. Backup Script 2006-09-09T00:33:02+00:00 I was working on some backup scripts and I decided I might as well generalise almost all of it (I am sorry about the hard coded paths) so almost anyone can come grab it and use it on their servers. The shell script is pretty self explanatory and I am not very good at documenting code, so the script source itself is your best bet as to why I did something some way. This script is in production use on several servers I administrate, I hope someone can use it as well. Documentation on how to use it: A backup script which will mount /backup</code>, then use rsync to copy files from /usr/home</code> to /backup/yyyy-mm-dd</code>. After it is done, it will remove the oldest backup, 14 days ago. All paths are hardcoded. If a file .exclude</code> is in a user's homedir, it is looked at, and is used to exclude certain directories or files from the rsync. For example: In /usr/home/xistence/.exclude</code>: public_html/iso</code></pre> No trailing slash, this will cause the folder /usr/home/xistence/public_html/iso</code> to not be backed up. This might be for various reasons, but for this example it is because backing up GB sized files should not be something that is done, only because it takes ages and wastes valuable space. Thing to keep in mind is, if you want to backup a 40 GB HD, with 22 GB used, to keep even a 3 day backup of that would require 3 * 22 GB, or 66 GB of available space on the backup drive. It is therefore in one's best interest to exclude very large files, and back them up in a different method since they are most likely not going to change much across revisions. This script should be set to cron every 30 minutes, as rsync will do one full backup when the day changes, and then from there on do incremental backups. Maybe removing the --delete</code> option might help with customers that delete files and expect them to still be in the backup for that day, this way however if a file is removed it will still exist in the backup until it is deleted after 14 days. #!/usr/local/bin/bash # This file REQUIRES bash, and can not be run under /bin/sh! ### # Copyright 2006 Delta Regeer. All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # 1. Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # 2. Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in the # documentation and/or other materials provided with the distribution. # # THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND # ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE # ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS # OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) # HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT # LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY # OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF # SUCH DAMAGE. # # The views and conclusions contained in the software and documentation are # those of the authors and should not be interpreted as representing official # policies, either expressed or implied, of bsdPanel project. # ### for HOMEDIR in `/usr/bin/find /usr/home -type d -maxdepth 1 | grep -v '^.$'`; do if [ -f $HOMEDIR/.exclude ]; then echo "Exclude found for $HOMEDIR" while read line; do if [ "${line:0:1}" != "/" ]; then # They are missing a full path, we will be nice and add it TMPLINE="$HOMEDIR/$line" line=$TMPLINE fi echo " Excluding $line" echo $line >> /tmp/exclude.$$ done < $HOMEDIR/.exclude fi done mount | grep /backup if [ $? -ne 0 ]; then /sbin/mount /backup if [ $? -ne 0 ]; then logger -s -p user.err -t backup "Backup drive could not be mounted! Bailing!" /sbin/umount -f /backup exit 1 fi fi BACKUPDIR="/backup/`date "+%Y-%m-%d"`" OLDDIR="/backup/`date -v-14d "+%Y-%m-%d"`" if [ ! -d $BACKUPDIR ]; then mkdir $BACKUPDIR fi if [ $? -ne 0 ]; then logger -s -p user.err -t backup "Could not create backup folder $BACKUPDIR. Bailing!" /sbin/umount -f /backup exit 1 fi # We got this far, now we have to run rsync. Rsync is very verbose on purpose /usr/local/bin/rsync -av --relative --delete /usr/home/ $BACKUPDIR --exclude-from=/tmp/exclude.$$ if [ $? -ne 0 ]; then logger -s -p user.err -t backup "rsync failed. Do not trust backup ($BACKUPDIR)" /sbin/umount /backup exit 1 fi echo "Backup complete! Deleting oldest backup (14 days ago)" if [ -d $OLDDIR ]; then rm -rf $OLDDIR fi /sbin/umount /backup /bin/rm -f /tmp/exclude.$$ logger -s -p user.notice -t backup "Backup complete at $BACKUPDIR."</code></pre>