Building custom ports with Poudriere and Portshaker
Guest post by Scott Sturdivant.
Maintaining custom ports and integrating them into your build process doesn't need to be difficult. The documentation surrounding this process however is either non-existent, or lacking in its clarity. At the end of the day, it really is as simple as maintaining a repository whose structure matches the ports tree layout, then managing that repository and the standard ports tree with portshaker, and finally handing the end result off to poudriere.
Your Custom Repository
For this example, we'll assume a git repo is used and that you're already
familiar with how to build FreeBSD ports. We'll also assume that we
have but a single port that we're maintaining and that it is called myport.
The hierarchy of your repo should simply be category/myport. We'll refer to
this repo simply as myrepo.
Portshaker
Portshaker is the tool responsible for taking multiple ports sources and then
merging them down into a single target. In our case, we have two sources: our
git repo (myrepo) containing myport, and the standard FreeBSD ports tree.
We aim to merge this down into a single ports tree that poudriere will then use
for its builds.
To configure portshaker, add the following to the
/usr/local/etc/portshaker.conf file:
# vim:set syntax=sh: # $Id: portshaker.conf.sample 116 2008-09-30 16:15:02Z romain.tartiere $ #---[ Base directory for mirrored Ports Trees ]--- mirror_base_dir="/var/cache/portshaker" #---[ Directories where to merge ports ]--- ports_trees="default" use_zfs="no" poudriere_ports_mountpoint="/usr/local/poudriere/ports" default_poudriere_tree="default" default_merge_from="freebsd myrepo"
Some key points here are that the two items listed in for the
default_merge_from argument need to have scripts present in the
/usr/local/etc/portshaker.d directory. Further more, the combination of the
poudriere_ports_mountpoint and default_poudriere_tree needs to be a ports
tree that is then registered with poudriere.
Next, we need to tell portshaker how to go off and fetch our two types of ports
trees, freebsd and myrepo. For the freebsd ports tree, create
/usr/local/etc/portshaker.d/freebsd with the following contents and make it
executable:
#!/bin/sh . /usr/local/share/portshaker/portshaker.subr method="portsnap" run_portshaker_command $*
Next, create a similar script to handle our repository containing our custom
port. /usr/local/etc/portshaker.d/myrepo should contain the following and
similarly be executable:
#!/bin/sh . /usr/local/share/portshaker/portshaker.subr method="git" git_clone_uri="http://github.com/scott.sturdivant/packaging.git" git_branch="master" run_portshaker_command $*
Obviously replace the git_clone_uri and git_branch variables to reflect
your actual configuration. For more information about the values and what they
can contain, consult man portshaker.d
Now, portshaker should be all set. Execute portshaker -U to update your
merge_from ports trees (freebsd and myrepo). You'll see the standard
portsnap fetch and extract process as well as a git clone. After a good
bit of time, these will both be present in the /var/cache/portshaker
directory. Go ahead and merge them together by executing portshaker -M.
Hooray! You now have /usr/local/poudriere/ports/default/ports that is a
combination of the normal ports tree and your custom one.
We're effectively complete with configuring portshaker. Whenever your port is
updated, just re-run portshaker -U and portshaker -M to grab the latest
changes and perform the merge.
Poudriere
Poudriere is a good tool for building ports. We will use it to handle our
merged directory. Begin by configuring poudriere
(/usr/local/etc/poudriere.conf):
NO_ZFS=yes FREEBSD_HOST=ftp://ftp.freebsd.org RESOLV_CONF=/etc/resolv.conf BASEFS=/usr/local/poudriere USE_PORTLINT=no USE_TMPFS=yes DISTFILES_CACHE=/usr/ports/distfiles CHECK_CHANGED_OPTIONS=yes
Really there's nothing here that is specific to the problem at hand, so feel free to consult the provided configuration file to tune it to your needs.
Now, the step that is specific is to set poudriere up with a ports tree that
it does not manage, specifically our resultant merged directory. If you
consult man poudriere, it specifies that for the ports subcommand, there is
a -m method switch which controls the methodology used to create the ports
tree. By default, it is portsnap. This is confusing as in our case, we do not
want poudriere to actually do anything. We want it to just use an existing
path. Fortunately, there is a way!
The poudriere wiki has an entry for using the system ports tree, so we adopt it for our needs by executing:
poudriere ports -c -F -f none -M /usr/local/poudriere/ports/default \ -p default
If you've consulted the poudriere manpage, you'll see that the -F and -f
switches both reference ZFS in their help. As we're not using ZFS, it's not
clear how they will behave. However, in conjunction with the custom mountpoint
(-M /usr/local/poudriere/ports/default), we ultimately wind up with what we
want, a ports tree that poudriere can use, but does not manage:
# poudriere ports -l PORTSTREE METHOD PATH default - /usr/local/poudriere/ports/default
Note that this resulting PATH is the combination of the
poudriere_ports_mountpoint and default_poudriere_tree variables present in
our /usr/local/etc/portshaker.conf configuration file.
Building software from your custom ports tree
Go ahead and create your jail(s) like you normally would (i.e.
poudriere -c -j 92amd64 -V 9.2-RELEASE -a amd64) and any other configuration
you would like, and then go ahead and build myport with
poudriere bulk -j 92amd64 -p default category/myport. Success!