funcptr - OpenIndiana

IPv6 -- getaddrinfo() and bind() ordering with V6ONLY

2012-08-07T22:39:56+00:00

Recently I ran into an issue that took me a while to sort out, and it is regarding inconsistent behaviour on various OS's with regards to IPv6 sockets (AF_INET6</code></a>1</a></sup>) and calling bind(2)</code></a> after getting the results back from getaddrinfo(3)</code></a>.</p>

A call to getaddrinfo()</code> with the hints set to AF_UNSPEC</code></a> in ai_family</code> and AI_PASSIVE</code></a> in ai_flags</code> will return to us 1 or more results that we can bind()</code> to. Sample code for that looks like this:</p>

struct</span> addrinfo hints,</span> *</span>addrlist;</span></span>
</span>
memset</span>(</span>&</span>hints</span>,</span> 0</span>,</span> sizeof</span>(hints));</span></span>
</span>
// Ask for TCP</span></span>
hints.ai_socktype </span>=</span> SOCK_STREAM;</span></span>
</span>
// Any family works for us ...</span></span>
hints.ai_family </span>=</span> AF_UNSPEC;</span></span>
</span>
// Set some hints</span></span>
hints.ai_flags </span>=</span> </span></span>
    AI_PASSIVE    </span>|</span> // We want to use this with bind</span></span>
    AI_ADDRCONFIG;</span>  // Only return IPv4 or IPv6 if they are configured</span></span>
</span>
int</span> rv;</span></span>
</span>
if</span> ((rv </span>=</span> getaddrinfo</span>(</span>0</span>,</span> "</span>7020</span>"</span>,</span> &</span>hints</span>,</span> &</span>addrlist</span>))</span> !=</span> 0</span>) {</span></span>
    fprintf</span>(stderr,</span> "</span>getaddrinfo: </span>%s</span>"</span>,</span> gai_strerror</span>(rv))</span>;</span></span>
    return</span> 1</span>;</span></span>
}</span></span>
</span>
// Use the list in *addrlist</span></span>
for</span> (addr </span>=</span> addrlist; addr </span>!=</span> 0</span>; addr </span>=</span> addr</span>-</span>></span>ai_next) {</span></span>
    // use *addr as appropriate</span></span>
}</span></span>
</span>
// Clean up the memory from getaddrinfo()</span></span>
freeaddrinfo</span>(addrlist);</span></span></code></pre>
On Linux there are two entries returned when the host it is run on has both
IPv4 and IPv6 enabled. An AF_INET</code> which was followed by an AF_INET6</code>. Now,
it is not said that you are required to use all of the results that are
returned, but if you want to listen on all address families it is off course
suggested.</p>
Following the steps below for each of the returned results should result in
having 1 or more different sockets that are bound to a single port.</p>

Create the socket()</li>
Set any socket options you want (SO_REUSEADDR</code> for example)</li>
Then bind() the socket</li>
After that call listen() (followed off course by accept() on the socket)</li>
</ol>
Only for some unknown reason (and errno</code> is no help) bind()</code> fails when you
get to the AF_INET6</code>, which was returned second. Searching online as to why
the bind would fail doesn't give you any good results and the thing that is
even worse is that if you run the same code on another platform such as
FreeBSD</a>, OpenIndiana</a> or Mac OS X</a> no such failure exists. However
I started suspecting something was up when I started looking at the output from
netstat -lan | grep 7020</code> on Mac OS X. Where 7020 is the port I passed into
getaddrinfo()</code>.</p>
tcp46      0      0  *.7020                 *.*  LISTEN     </span></span>
tcp4       0      0  *.7020                 *.*  LISTEN </span></span></code></pre>
Wait a minute ... one of the sockets is on both IPv4 and on IPv6. Some more
time spent searching the internet I came across RFC 3493 section 5.3</a>,
which is titled "IPV6_V6ONLY</code> option for AF_INET6</code> Sockets".</p>

As stated in section <3.7 Compatibility with IPv4 Nodes>, AF_INET6</code> sockets
may be used for both IPv4 and IPv6 communications.  Some applications may
want to restrict their use of an AF_INET6</code> socket to IPv6 communications
only.</p>
</blockquote>
This was going down the right route, so I changed my code so that in the steps
listed above in number 2 I added the following code if the socket type is
AF_INET6</code>:</p>
if</span> (</span>setsockopt</span>(sockfd, IPPROTO_IPV6, IPV6_V6ONLY,</span> &</span>yes</span>,</span> sizeof</span>(</span>int</span>))</span> ==</span> -</span>1</span>) {</span></span>
    close</span>(sockfd)</span>;</span></span>
    fprintf</span>(stderr,</span> "</span>setsockopt: </span>%s</span> IPV6_V6ONLY</span>\n</span>"</span>,</span> strerror</span>(errno))</span>;</span></span>
    continue</span>;</span></span>
}</span></span></code></pre>
The RFC 3493 section 5.3 also states that this option should be turned off by
default, which means that all IPv6 sockets can also communicate over IPv4. Thus
technically setting the option manually in code the best way to fix the issue.
FreeBSD has had this feature turned on (as in IPv6 sockets can only communicate
with IPv6 and NOT IPv4) since 5.x.</p>
The biggest issue is that the remaining operating systems (OS X and
OpenIndiana) don't have the same behaviour as Linux which makes troubleshooting
this issue more difficult than it should be. The issue is that the RFC doesn't
specify what exactly the operating should do when it encounters a request to
bind to the same port on IPv4 and IPv6. The only place where I have found this
documented is in "IPv6 Network Programming"</a> under "Tips in IPv6
Programming" chapter 4, section 4, appropriately titled "bind(2) Ordering and
Conflicts".</p>

If you get a bind()</code> error when attempting to bind to an AF_INET6</code> socket
please make sure that you set the socket option IPV6_V6ONLY</code> on the AF_INET6</code>
socket. The default as required by RFC 3493 is to have that option be off. The
default is wrong, and the RFC should have been more specific regarding what the
right behaviour is when attempting to bind on an AF_INET6</code> socket when already
bound on an AF_INET</code> while IPV6_V6ONLY</code> is set to false.</p>
The full code that I used for testing, along with a little bit more information
is available as a gist on github</a>.</p>



The old BSD style socket()</code></a> called for defines
starting with PF_</code> such as PF_INET</code> and PF_INET6</code> with the PF</code> standing for
protocol family. POSIX starts them with AF_</code>, and calls them an address
family. On almost every operating system PF_INET</code> is the same as AF_INET</code>. If
the define doesn't exist you can always create it. ↩</a></p>
</li>
</ol>
</section>



Hosting NFS shares from a Zone
2009-02-21T00:10:26+00:00
I am currently trying to get my web development environment all set up.
While I like using vi, I really do prefer TextMate on my Mac OS X to do
the real heavy lifting and all of the editing.</p>
As I was working on getting my homedir on my zone to be shared over NFS I
was getting errors that NFS was not a valid protocol from sharemgr</code>, the
program that is used to add new shares to the system.</p>
xistence@webdev.network.lan:~# sharemgr create -P nfs homedir  </span></span>
Invalid protocol specified: nfs</span></span></code></pre>
A little bit of Googling brought me to a mailing list message saying that
some server packages may be missing. This is entirely possible since my
Zone was created with a minimal set of packages, so I log into the global
host where NFS sharing does work and run the following:</p>
xistence@Keyhole.network.lan:~# pkg list | grep nfs</span></span>
SUNWnfsc                                      0.5.11-0.101    installed  ----</span></span>
SUNWnfsckr                                    0.5.11-0.101    installed  ----</span></span>
SUNWnfss                                      0.5.11-0.101    installed  ----</span></span>
SUNWnfsskr                                    0.5.11-0.101    installed  ----</span></span></code></pre>
Ah, so those are probably not installed in the zone, quick pkg install</code>
later:</p>
xistence@webdev.network.lan:~# pkg install SUNWnfsc SUNWnfsckr SUNWnfss SUNWnfsskr</span></span>
DOWNLOAD                                    PKGS       FILES     XFER (MB)</span></span>
Completed                                    3/3       17/17     0.28/0.28 </span></span>
</span>
PHASE                                        ACTIONS</span></span>
Install Phase                                  55/55 </span></span>
PHASE                                          ITEMS</span></span>
Reading Existing Index                           9/9 </span></span>
Indexing Packages                                3/3</span></span></code></pre>
Now, lets re-run the sharemgr</code> command again and see if this time we do
indeed get our group created:</p>
xistence@webdev.network.lan:~# sharemgr create -P nfs homedir</span></span>
xistence@webdev.network.lan:~# sharemgr show -vp</span></span>
default nfs=()</span></span>
zfs</span></span>
homedir nfs=()</span></span></code></pre>
Yes, there we go. Now I can add a share to that group. Until more errors
start showing up:</p>
xistence@webdev.network.lan:~# sharemgr add-share -s /export/home/xistence/ homedir</span></span>
NFS: Cannot share remote filesystem: /export/home/xistence</span></span>
NFS: Service needs to be enabled by a privileged user</span></span>
Failed to enable share for "nfs": system error</span></span></code></pre>
This time Google does not help out, instead it gives me the option to
search without quotes around the search terms which gives me absolutely
nothing.</p>
Well, lets see if the services are running in the first place:</p>
xistence@webdev.network.lan:~# svcs -a | grep nfs    </span></span>
disabled       22:19:04 svc:/network/nfs/client:default</span></span>
disabled       22:24:26 svc:/network/nfs/server:default</span></span>
online         22:24:20 svc:/network/nfs/rquota:default</span></span>
online         22:24:20 svc:/network/nfs/status:default</span></span>
online         22:24:20 svc:/network/nfs/mapid:default</span></span>
online         22:24:20 svc:/network/nfs/cbd:default</span></span>
online         22:24:21 svc:/network/nfs/nlockmgr:default</span></span></code></pre>
Nope, for some reason the nfs server keeps being disabled even though I
did enable it using svcadmin</code>.</p>
Okay, lets take a look with svcs</code> what is going on:</p>
xistence@webdev.network.lan:/# svcs -l nfs/server</span></span>
fmri         svc:/network/nfs/server:default</span></span>
name         NFS server</span></span>
enabled      false (temporary)</span></span>
state        disabled</span></span>
next_state   none</span></span>
state_time   Fri Feb 20 22:25:26 2009</span></span>
logfile      /var/svc/log/network-nfs-server:default.log</span></span>
restarter    svc:/system/svc/restarter:default</span></span>
contract_id  </span></span>
dependency   require_any/error svc:/milestone/network (online)</span></span>
dependency   require_all/error svc:/network/nfs/nlockmgr (online)</span></span>
dependency   optional_all/error svc:/network/nfs/mapid (online)</span></span>
dependency   require_all/restart svc:/network/rpc/bind (online)</span></span>
dependency   optional_all/none svc:/network/rpc/keyserv (disabled)</span></span>
dependency   optional_all/none svc:/network/rpc/gss (absent)</span></span>
dependency   optional_all/none svc:/network/shares/group (multiple)</span></span>
dependency   require_all/error svc:/system/filesystem/local (online)</span></span></code></pre>
Ah, it has a log file, lets see what that log file says:</p>
xistence@webdev.network.lan:/# cat /var/svc/log/network-nfs-server\:default.log </span></span>
[ Feb 20 22:11:07 Disabled. ]</span></span>
[ Feb 20 22:11:07 Rereading configuration. ]</span></span>
[ Feb 20 22:18:43 Enabled. ]</span></span>
[ Feb 20 22:19:04 Enabled. ]</span></span>
[ Feb 20 22:24:21 Executing start method ("/lib/svc/method/nfs-server start"). ]</span></span>
The NFS server is not supported in a local zone</span></span></code></pre>
Eh? What? That seems weird, but it is of course plausible. Zones are new
to Solaris and thus not everything may be virtualised yet. Taking that
query to google brings me to a web page asking the same question I am
asking: Local zone as NFS server. Taking a look it brings me to a bug
report and ultimately to: Bug ID 4964859 RFE: Zones should be able to be
NFS servers. Well, I guess that ends that little stunt. I will be sharing
it from the global zone. While this is acceptable to me, I do not believe
that it is acceptable to most other people using Zones, and while it is
unfortunate but it does not look like there is a whole lot of activity on
the bug, as apparently there are no resources available.</p>
I ended up doing the following:</p>
sharemgr create -P nfs webdev-homedir</span></span></code></pre>
Then I followed that up with some servers that should be allowed access,
and others that should not:</p>
sharemgr set -P nfs -S sys -p 'rw=10.10.10.226 none=*' webdev-homedir</span></span></code></pre>
And then finally I added the share I wanted to share all along:</p>
sharemgr add-share -r "homedir" -d "xistence home dir" -s /storage/zones/dev-web/root/export/home/xistence webdev-homedir</span></span></code></pre>
This creates a long NFS share to type in, since the full path now becomes:</p>
nfs://keyhole/storage/zones/dev-web/root/export/home/xistence</span></span></code></pre>
Which is not that big of a problem for me, especially since I just save
it in my favourites in Mac OS X and everything is peachy. It works like a
charm.</p>


Upgrading an OpenSolaris ipkg zone
2009-01-18T00:10:02+00:00
Recently I was working on my OpenSolaris machine (file server, ZFS rocks, will
write more on that later), and I have one non-global zone, one that use for web
development, which is aptly called dev-web.</p>
So the following shows up when I run zoneadm:</p>
# zoneadm list -iv</span></span>
  ID NAME             STATUS     PATH                           BRAND    IP    </span></span>
   0 global           running    /                              native   shared</span></span>
   4 dev-web          running    /storage/zones/dev-web         ipkg     shared</span></span></code></pre>
The thing is, I had upgraded the global zone with the latest update available for the version (snv_101b</code>):</p>
pkg</span> image-update</span> -v</span></span></code></pre>
This had not upgraded my one none-global zone. And running pkg image-update</code>
from within the zone itself is not possible, because you can't do an upgrade on
a "live" system, mainly because the image-update</code> wants to create a new bootable
environment, something that was already created when I upgraded the global
zone. So what we have to do is mount the non-global zone to /mnt</code> and tell pkg</code>
with -R</code> where to find it and upgrade it anyway!</p>
First we are going to halt the current zone, important is running on the test
system, I just used:</p>
zoneadm</span> -z</span> dev-web halt</span></span></code></pre>
However, the better way is to off course use the shutdown command:</p>
zlogin</span> dev-web shutdown</span></span></code></pre>
and then for good measure a halt!</p>
Next up, looking at what we are going to be mounting.</p>
# zfs list</span></span>
NAME                               USED  AVAIL  REFER  MOUNTPOINT</span></span>
rpool                             11.9G  42.7G    75K  /rpool</span></span>
rpool/ROOT                        2.93G  42.7G    18K  legacy</span></span>
rpool/ROOT/opensolaris            7.34M  42.7G  2.74G  /</span></span>
rpool/ROOT/opensolaris-1          2.92G  42.7G  2.74G  /</span></span>
rpool/dump                        1.87G  42.7G  1.87G  -</span></span>
rpool/export                      5.27G  42.7G    19K  /export</span></span>
rpool/export/home                 5.27G  42.7G    21K  /export/home</span></span>
rpool/export/home/guest           5.24G  14.8G  5.24G  /export/home/guest</span></span>
rpool/export/home/xistence        37.0M  42.7G  37.0M  /export/home/xistence</span></span>
rpool/swap                        1.87G  43.0G  1.51G  -</span></span>
storage                            493G  3.08T  35.1K  /storage</span></span>
storage/media                      398G  3.08T   398G  /storage/media</span></span>
storage/virtualbox                2.10G  3.08T  2.10G  /storage/virtualbox</span></span>
storage/xistence                  91.7G  3.08T  91.7G  /storage/xistence</span></span>
storage/zones                      957M  99.1G  30.4K  /storage/zones</span></span>
storage/zones/dev-web              957M  19.1G  32.0K  /storage/zones/dev-web</span></span>
storage/zones/dev-web/ROOT         957M  19.1G  28.8K  legacy</span></span>
storage/zones/dev-web/ROOT/zbe    1.60M  19.1G   936M  legacy</span></span>
storage/zones/dev-web/ROOT/zbe-1   955M  19.1G   935M  legacy</span></span></code></pre>
When pkg image-update</code> was run on the global zone it created a new bootable
environment named opensolaris-1, the cool thing is, that beadm</code> at the same time
will also create a new bootable environment for your zones. That way you can
upgrade your zones afterwards, and if stuff does not work, you can revert the
ENTIRE machine back to the previous state (ZFS is cool like that), thereby also
making sure that your zones are reverted so that there are no
incompatibilities.</p>
So what we are looking for in this case is a zbe-1, this is the new root for
the zone that we need to update, so we now need to mount it.</p>
mount</span> -F</span> zfs storage/zones/dev-web/ROOT/zbe-1 /mnt</span></span></code></pre>
Note, that there is no /</code> in front of storage, this is because we are specifying
a pool name, since there is no "real" path that is defined as
/storage/zones/dev-web/ROOT/zbe-1</code>. Now that it is mounted, we are able to pass
the -R flag to pkg, to get it to update our zone:</p>
# pkg -R /mnt image-update -v</span></span>
Creating Plan / Before evaluation:     </span></span>
UNEVALUATED:</span></span>
+pkg:/entire@0.5.11,5.11-0.101:20081204T010954Z</span></span>
</span>
After evaluation:</span></span>
pkg:/entire@0.5.11,5.11-0.101:20081119T235706Z -> pkg:/entire@0.5.11,5.11-0.101:20081204T010954Z</span></span>
Actuators:</span></span>
</span>
None</span></span>
PHASE                                        ACTIONS</span></span>
Update Phase                                     1/1 </span></span>
PHASE                                          ITEMS</span></span>
Reading Existing Index                           9/9 </span></span>
Indexing Packages                                1/1 </span></span>
</span>
---------------------------------------------------------------------------</span></span>
NOTE: Please review release notes posted at:</span></span>
   http://opensolaris.org/os/project/indiana/resources/relnotes/200811/x86/</span></span>
---------------------------------------------------------------------------</span></span></code></pre>
Voilá, and the deed is done. The last command is to off course unmount the
zone, that we can then issue a zoneadm boot command to start it back up:</p>
umount</span> /mnt</span></span>
zoneadm</span> -z</span> dev-web boot</span></span></code></pre>
And then on the zone after we log into it (over SSH in my case):</p>
xistence@webdev.network.lan:~# pkg list -u</span></span>
No installed packages have available updates</span></span></code></pre>
Which is exactly what we wanted! Your zone is now upgraded with the latest
version available from the global zone.</p>