funcptr - shell

Using DJB's daemontools and netcat to bounce a request around

2008-11-29T17:56:05+00:00

Lately I have been moving the data from one server to another server, located halfway across the globe, and I needed some way to forward all incoming requests from the old server to the new server. This had to be done so that DNS could take its time to update while everything was now already being served from the new location.</p>

What I ended up doing was using daemontools along with netcat to pipe the request around the world. Here are the steps I took in doing so:</p>

mkdir</span> /usr/local/redirect/</span></span>
cd</span> /usr/local/redirect/</span></span>
mkdir</span> smtp smtp/env</span></span>
cd</span> smtp</span></span></code></pre>
Put the following in a file named run:</p>
#!/bin/sh</span></span>
exec</span> 2>&1</span> \</span></span>
envdir</span> ./env</span> \</span></span>
sh</span> -c</span> '</span></span>
    case "$REMOTENAME" in h) H=;; p) H=p;; *) H=H;; esac</span></span>
    case "$REMOTEINFO" in r) R=;; [0-9]*) R="t$REMOTEINFO";; *) R=R;; esac</span></span>
    exec \</span></span>
    /usr/local/bin/tcpserver \</span></span>
        -vDU"$H$R" \</span></span>
        ${LOCALNAME+"-l$LOCALNAME"} \</span></span>
        ${BACKLOG+"-b$BACKLOG"} \</span></span>
        ${CONCURRENCY+"-c$CONCURRENCY"} \</span></span>
        -- "${IP-0}" "${PORT}" \</span></span>
        /usr/bin/nc "${REMOTEHOST}" "${REMOTEPORT}"</span></span>
'</span></span></code></pre>
Make it executable: chmod +x run</code></p>
Then we need to set up a few environment variables:</p>
cd</span> env</span></span>
echo</span> "</span>example.net</span>"</span> ></span> REMOTEHOST</span></span>
echo</span> "</span>25</span>"</span> ></span> REMOTEPORT</span></span>
echo</span> "</span>25</span>"</span> ></span> PORT</span></span>
echo</span> `</span>hostname</span>`</span> ></span> LOCALNAME</span></span>
echo</span> "</span>200</span>"</span> ></span> CONCURRENCY</span></span></code></pre>
I made REMOTEPORT</code> and PORT</code> be separate on purpose, in one case I now had more
IP's than before, so instead of having SSL running on a separate port it was
running on the default port, and I needed a clean way to forward that.</p>
Now just add it to your services folder as a symlink and it will automatically
be started, from there it will do it's job! I also suggest adding some simple
logging, or discarding all off the output from tcpserver.</p>



Backup Script
2006-09-09T00:33:02+00:00
I was working on some backup scripts and I decided I might as well generalise
almost all of it (I am sorry about the hard coded paths) so almost anyone can
come grab it and use it on their servers. The shell script is pretty self
explanatory and I am not very good at documenting code, so the script source
itself is your best bet as to why I did something some way. This script is in
production use on several servers I administrate, I hope someone can use it as
well.</p>
Documentation on how to use it:</p>
A backup script which will mount /backup</code>, then use rsync to copy files from
/usr/home</code> to /backup/yyyy-mm-dd</code>. After it is done, it will remove the
oldest backup, 14 days ago. All paths are hardcoded.</p>
If a file .exclude</code> is in a user's homedir, it is looked at, and is used to
exclude certain directories or files from the rsync. For example:</p>
In /usr/home/xistence/.exclude</code>:</p>
public_html/iso</span></span></code></pre>
No trailing slash, this will cause the folder
/usr/home/xistence/public_html/iso</code> to not be backed up. This might be for
various reasons, but for this example it is because backing up GB sized files
should not be something that is done, only because it takes ages and wastes
valuable space.</p>
Thing to keep in mind is, if you want to backup a 40 GB HD, with 22 GB used,
to keep even a 3 day backup of that would require 3 * 22 GB, or 66 GB of
available space on the backup drive. It is therefore in one's best interest
to exclude very large files, and back them up in a different method since
they are most likely not going to change much across revisions.</p>
This script should be set to cron every 30 minutes, as rsync will do one full
backup when the day changes, and then from there on do incremental backups.</p>
Maybe removing the --delete</code> option might help with customers that delete
files and expect them to still be in the backup for that day, this way
however if a file is removed it will still exist in the backup until it is
deleted after 14 days.</p>
#!/usr/local/bin/bash</span></span>
</span>
# This file REQUIRES bash, and can not be run under /bin/sh!</span></span>
</span>
###</span></span>
 # Copyright 2006 Delta Regeer. All rights  reserved.</span></span>
 #</span></span>
 # Redistribution and use in source and binary forms, with or without</span></span>
 # modification, are permitted provided that the following conditions</span></span>
 # are met:</span></span>
 # 1. Redistributions of source code must retain the above copyright</span></span>
 #    notice, this list of conditions and the following disclaimer.</span></span>
 # 2. Redistributions in binary form must reproduce the above copyright</span></span>
 #    notice, this list of conditions and the following disclaimer in the</span></span>
 #    documentation and/or other materials provided with the distribution.</span></span>
 #</span></span>
 # THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND</span></span>
 # ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE</span></span>
 # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE</span></span>
 # ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE</span></span>
 # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL</span></span>
 # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS</span></span>
 # OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)</span></span>
 # HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT</span></span>
 # LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY</span></span>
 # OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF</span></span>
 # SUCH DAMAGE.</span></span>
 #</span></span>
 # The views and conclusions contained in the software and  documentation are</span></span>
 # those of the authors and should not be  interpreted as representing official</span></span>
 # policies, either expressed  or implied, of bsdPanel project.</span></span>
 #</span></span>
###</span></span>
</span>
for</span> HOMEDIR</span> in</span> `</span>/usr/bin/find</span> /usr/home</span> -type</span> d</span> -maxdepth</span> 1</span> |</span> grep</span> -v</span> '</span>^.$</span>'</span>`</span>;</span> do</span></span>
	if</span> [</span> -f</span> $HOMEDIR</span>/.exclude ];</span> then</span></span>
		echo</span> "</span>Exclude found for </span>$HOMEDIR</span>"</span></span>
		while</span> read</span> line</span>;</span> do</span></span>
			if</span> [</span> "</span>${</span>line</span>:</span>0</span>:</span>1</span>}</span>"</span> !=</span> "</span>/</span>"</span> ];</span> then</span></span>
				# They are missing a full path, we will be nice and add it</span></span>
				TMPLINE</span>=</span>"</span>$HOMEDIR</span>/</span>$line</span>"</span></span>
				line</span>=</span>$TMPLINE</span></span>
			fi</span></span>
			echo</span> "</span>	Excluding </span>$line</span>"</span></span>
			echo</span> $line</span> >></span> /tmp/exclude.</span>$$</span></span>
		done</span> <</span> $HOMEDIR</span>/.exclude</span></span>
	fi</span></span>
done</span></span>
</span>
mount</span> |</span> grep</span> /backup</span></span>
</span>
if</span> [</span> $?</span> -ne</span> 0</span> ];</span> then</span></span>
	/sbin/mount</span> /backup</span></span>
	if</span> [</span> $?</span> -ne</span> 0</span> ];</span> then</span></span>
		logger</span> -s -p</span> user.err</span> -t</span> backup</span> "</span>Backup drive could not be mounted! Bailing!</span>"</span></span>
		/sbin/umount</span> -f</span> /backup</span></span>
		exit</span> 1</span></span>
	fi</span></span>
fi</span></span>
</span>
BACKUPDIR</span>=</span>"</span>/backup/`</span>date</span> "</span>+%Y-%m-%d</span>"</span>`</span>"</span></span>
OLDDIR</span>=</span>"</span>/backup/`</span>date</span> -v-14d</span> "</span>+%Y-%m-%d</span>"</span>`</span>"</span></span>
</span>
if</span> [</span> ! -d</span> $BACKUPDIR</span> ];</span> then</span></span>
	mkdir</span> $BACKUPDIR</span></span>
fi</span></span>
</span>
if</span> [</span> $?</span> -ne</span> 0</span> ];</span> then</span></span>
	logger</span> -s -p</span> user.err</span> -t</span> backup</span> "</span>Could not create backup folder </span>$BACKUPDIR</span>. Bailing!</span>"</span></span>
	/sbin/umount</span> -f</span> /backup</span></span>
	exit</span> 1</span></span>
fi</span></span>
</span>
# We got this far, now we have to run rsync. Rsync is very verbose on purpose</span></span>
</span>
/usr/local/bin/rsync</span> -av --relative --delete</span> /usr/home/</span> $BACKUPDIR</span> --exclude-from=/tmp/exclude.</span>$$</span></span>
</span>
if</span> [</span> $?</span> -ne</span> 0</span> ];</span> then</span></span>
	logger</span> -s -p</span> user.err</span> -t</span> backup</span> "</span>rsync failed. Do not trust backup (</span>$BACKUPDIR</span>)</span>"</span></span>
	/sbin/umount</span> /backup</span></span>
	exit</span> 1</span></span>
fi</span></span>
</span>
echo</span> "</span>Backup complete! Deleting oldest backup (14 days ago)</span>"</span></span>
</span>
if</span> [</span> -d</span> $OLDDIR</span> ];</span> then</span></span>
	rm</span> -rf</span> $OLDDIR</span></span>
fi</span></span>
</span>
/sbin/umount</span> /backup</span></span>
/bin/rm</span> -f</span> /tmp/exclude.</span>$$</span></span>
</span>
logger</span> -s -p</span> user.notice</span> -t</span> backup</span> "</span>Backup complete at </span>$BACKUPDIR</span>.</span>"</span></span></code></pre>