<?xml version="1.0" encoding="UTF-8"?>
<feed xmlns="http://www.w3.org/2005/Atom" xml:lang="en">
    <title>funcptr - Linux</title>
    <subtitle>An engineer&#x27;s technical notebook</subtitle>
    <link rel="self" type="application/atom+xml" href="https://funcptr.net/os/linux/atom.xml"/>
    <link rel="alternate" type="text/html" href="https://funcptr.net/"/>
    <generator uri="https://www.getzola.org/">Zola</generator>
    <updated>2015-07-08T19:54:43+00:00</updated>
    <id>https://funcptr.net/os/linux/atom.xml</id>
    <entry xml:lang="en">
        <title>Cobbler with CentOS 7 failure to boot&#x2F;kickstart</title>
        <published>2015-07-08T19:54:43+00:00</published>
        <updated>2015-07-08T19:54:43+00:00</updated>
        
        <author>
          <name>
            
              Delta Regeer
            
          </name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://funcptr.net/2015/cobbler-with-centos-7-failure-to-boot/"/>
        <id>https://funcptr.net/2015/cobbler-with-centos-7-failure-to-boot/</id>
        
        <content type="html" xml:base="https://funcptr.net/2015/cobbler-with-centos-7-failure-to-boot/">&lt;p&gt;Over the past week I&#x27;ve been working on building out an instance of Cobbler and
testing some of the provisioning that it is able to do. One of the operating
systems that I wanted to deploy is CentOS 7.&lt;&#x2F;p&gt;
&lt;p&gt;After I imported the system into cobbler, it correctly showed up in the
pxelinux boot menu and it would happily load the kernel and the initrd, however
after initial bootup it would throw the following error message:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #D6DEEB; background-color: #011627;&quot;&gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;dracut-initqueue[867]: Warning: Could not boot.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;dracut-initqueue[867]: Warning: &#x2F;dev&#x2F;root does not exist&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;         Starting Dracut Emergency Shell...&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Warning: &#x2F;dev&#x2F;root does not exist&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Generating &amp;quot;&#x2F;run&#x2F;initramfs&#x2F;rdsosreport.txt&amp;quot;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Entering emergency mode. Exit the shell to continue.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Type &amp;quot;journalctl&amp;quot; to view the system logs.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;You might want to save &amp;quot;&#x2F;run&#x2F;initramfs&#x2F;rdsosreport.txt&amp;quot; to a USB stick or &#x2F;boot&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;after mounting them and attach it to a bug report&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;After that it gives you a root shell.&lt;&#x2F;p&gt;
&lt;p&gt;Some Google searching led me to an &lt;a rel=&quot;external&quot; href=&quot;https:&#x2F;&#x2F;web.archive.org&#x2F;web&#x2F;20160202155132&#x2F;https:&#x2F;&#x2F;lists.fedorahosted.org&#x2F;pipermail&#x2F;cobbler&#x2F;2014-August&#x2F;009647.html&quot;&gt;mailing list post for Cobbler&lt;&#x2F;a&gt; where
someone mentioned that adding &lt;code&gt;ksdevice=link&lt;&#x2F;code&gt; to the Cobbler profile allowed
the system to boot without issues.&lt;&#x2F;p&gt;
&lt;p&gt;However before I just implement a change I want to know why that fixes the
issue, so I searched Google for &quot;kickstart ksdevice&quot; and found &lt;a rel=&quot;external&quot; href=&quot;https:&#x2F;&#x2F;web.archive.org&#x2F;web&#x2F;20150923201114&#x2F;https:&#x2F;&#x2F;access.redhat.com&#x2F;documentation&#x2F;en-US&#x2F;Red_Hat_Enterprise_Linux&#x2F;6&#x2F;html&#x2F;Installation_Guide&#x2F;s1-kickstart2-startinginstall.html&quot;&gt;Red Hat&#x27;s
documentation on starting a kickstart&lt;&#x2F;a&gt;. Searching that page for &quot;ksdevice&quot;
led me to this section:&lt;&#x2F;p&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;code&gt;ksdevice=&amp;lt;device&amp;gt;&lt;&#x2F;code&gt;&lt;&#x2F;p&gt;
&lt;p&gt;The installation program uses this network device to connect to the network. You can specify the device in one of five ways:&lt;&#x2F;p&gt;
&lt;ul&gt;
&lt;li&gt;the device name of the interface, for example, &lt;code&gt;eth0&lt;&#x2F;code&gt;&lt;&#x2F;li&gt;
&lt;li&gt;the MAC address of the interface, for example, &lt;code&gt;00:12:34:56:78:9a&lt;&#x2F;code&gt;&lt;&#x2F;li&gt;
&lt;li&gt;the keyword &lt;code&gt;link&lt;&#x2F;code&gt;, which specifies the first interface with its link in the up state&lt;&#x2F;li&gt;
&lt;li&gt;the keyword &lt;code&gt;bootif&lt;&#x2F;code&gt;, which uses the MAC address that pxelinux set in the
BOOTIF variable. Set IPAPPEND 2 in your pxelinux.cfg file to have pxelinux
set the BOOTIF variable.&lt;&#x2F;li&gt;
&lt;li&gt;the keyword &lt;code&gt;ibft&lt;&#x2F;code&gt;, which uses the MAC address of the interface specified by iBFT&lt;&#x2F;li&gt;
&lt;&#x2F;ul&gt;
&lt;p&gt;For example, consider a system connected to an NFS server through the eth1
device. To perform a kickstart installation on this system using a kickstart
file from the NFS server, you would use the command &lt;code&gt;ks=nfs:&amp;lt;server&amp;gt;:&#x2F;&amp;lt;path&amp;gt; ksdevice=eth1&lt;&#x2F;code&gt; at the boot: prompt.&lt;&#x2F;p&gt;
&lt;&#x2F;blockquote&gt;
&lt;p&gt;While &lt;code&gt;ksdevice=link&lt;&#x2F;code&gt; would work for some of the machines I am deploying, it
wouldn&#x27;t work for most since they have multiple interfaces and each one of
those interfaces would have link, what I really wanted was &lt;code&gt;ksdevice=bootif&lt;&#x2F;code&gt;,
which is the most sensible default.&lt;&#x2F;p&gt;
&lt;p&gt;So I modified the profile with &lt;code&gt;ksdevice=link&lt;&#x2F;code&gt; just to test, and that worked
without issues, so then I modified the profile and added &lt;code&gt;ksdevice=link&lt;&#x2F;code&gt;, and
this failed.&lt;&#x2F;p&gt;
&lt;p&gt;I figured I should check the &lt;code&gt;pxelinux.cfg&#x2F;default&lt;&#x2F;code&gt; file that Cobbler generates
upon issuing a &lt;code&gt;cobbler sync&lt;&#x2F;code&gt; and verify that &lt;code&gt;ksdevice=bootif&lt;&#x2F;code&gt; is actually
listed correctly.&lt;&#x2F;p&gt;
&lt;p&gt;What I found was this:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #D6DEEB; background-color: #011627;&quot;&gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;LABEL CentOS-7.1-x86_64&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;        kernel &#x2F;images&#x2F;CentOS-7.1-x86_64&#x2F;vmlinuz&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;        MENU LABEL CentOS-7.1-x86_64&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;        append initrd=&#x2F;images&#x2F;CentOS-7.1-x86_64&#x2F;initrd.img ksdevice=${net0&#x2F;mac} lang=  kssendmac text  ks=http:&#x2F;&#x2F;10.10.10.1&#x2F;cblr&#x2F;svc&#x2F;op&#x2F;ks&#x2F;profile&#x2F;CentOS-7.1-x86_64&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;        ipappend 2&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;This has a &lt;code&gt;ksdevice=${net0&#x2F;mac}&lt;&#x2F;code&gt; which is not what I had put in the profile,
overwriting &lt;code&gt;ksdevice&lt;&#x2F;code&gt; in the profile with &lt;code&gt;ksdevice=link&lt;&#x2F;code&gt; did correctly put
that into the &lt;code&gt;pxelinux.cfg&#x2F;default&lt;&#x2F;code&gt; file, so Cobbler was overwriting my change
somehow.&lt;&#x2F;p&gt;
&lt;p&gt;A quick search of &lt;code&gt;${net0&#x2F;mac}&lt;&#x2F;code&gt; led me to a page about &lt;a rel=&quot;external&quot; href=&quot;http:&#x2F;&#x2F;etherboot.org&#x2F;wiki&#x2F;commandline&quot;&gt;gPXE commandline
items&lt;&#x2F;a&gt; that contained the same variable. At which point I remembered that in
Cobbler you set up your profile to be gPXE enabled or not. The default when you
import an image is to enable gPXE support.&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #D6DEEB; background-color: #011627;&quot;&gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;cobbler profile report  --name=CentOS-7.1-x86_64&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Name                           : CentOS-7.1-x86_64&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;TFTP Boot Files                : {}&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Comment                        : &lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;DHCP Tag                       : default&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Distribution                   : CentOS-7.1-x86_64&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Enable gPXE?                   : True&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Enable PXE Menu?               : 1&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;[...]&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;So let&#x27;s modify the profile to disable gPXE support:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #D6DEEB; background-color: #011627;&quot;&gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #82AAFF;font-style: italic;&quot;&gt;cobbler&lt;&#x2F;span&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt; profile edit&lt;&#x2F;span&gt;&lt;span style=&quot;color: #82AAFF;&quot;&gt; --name=CentOS-7.1-x86_64 --enable-gpxe=False&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #82AAFF;font-style: italic;&quot;&gt;cobbler&lt;&#x2F;span&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt; sync&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;Verify that the change was made:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #D6DEEB; background-color: #011627;&quot;&gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;cobbler profile report  --name=CentOS-7.1-x86_64&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;[...]&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Enable gPXE?                   : False&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;[...]&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;Then let&#x27;s take a look at our &lt;code&gt;pxelinux.cfg&#x2F;default&lt;&#x2F;code&gt; file and make sure that it
looks correct:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #D6DEEB; background-color: #011627;&quot;&gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;LABEL CentOS-7.1-x86_64&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;        kernel &#x2F;images&#x2F;CentOS-7.1-x86_64&#x2F;vmlinuz&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;        MENU LABEL CentOS-7.1-x86_64&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;        append initrd=&#x2F;images&#x2F;CentOS-7.1-x86_64&#x2F;initrd.img ksdevice=bootif lang=  kssendmac text  ks=http:&#x2F;&#x2F;10.10.10.1&#x2F;cblr&#x2F;svc&#x2F;op&#x2F;ks&#x2F;profile&#x2F;CentOS-7.1-x86_64&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;        ipappend 2&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;This time our ksdevice is correctly set. Upon rebooting my PXE booted server it
picked up the correct interface, made a DHCP request and kickstarted the server
using the provided kickstart file, and installation completed successfully.&lt;&#x2F;p&gt;
&lt;p&gt;So unless you chain-boot gPXE from pxelinux by default, make sure that your
profiles are not set to be gPXE enabled if you want to use them directly from
the pxelinux menu.&lt;&#x2F;p&gt;
&lt;p&gt;While researching more about this article, I found a blog post by &lt;a rel=&quot;external&quot; href=&quot;https:&#x2F;&#x2F;web.archive.org&#x2F;web&#x2F;20160507142852&#x2F;https:&#x2F;&#x2F;blog.vladionescu.com&#x2F;pxe-installing-rhel-7-from-cobbler-2-6&#x2F;&quot;&gt;Vlad Ionescu
about PXE installing RHEL 7 from Cobbler&lt;&#x2F;a&gt; where he suggests disabling
&lt;code&gt;ksdevice&lt;&#x2F;code&gt; entirely and adding an extra &lt;code&gt;inst.repo&lt;&#x2F;code&gt; variable to the kernel
command line, however on older versions of CentOS 7 and Red Hat Enterprise
Linux 7 there is a &lt;a rel=&quot;external&quot; href=&quot;https:&#x2F;&#x2F;bugzilla.redhat.com&#x2F;show_bug.cgi?id=1096846&quot;&gt;bug report&lt;&#x2F;a&gt; that shows that an empty &lt;code&gt;ksdevice&lt;&#x2F;code&gt; could
cause anaconda to crash, and setting a manual &lt;code&gt;inst.repo&lt;&#x2F;code&gt; for every profile
seems like overkill when just disabling gPXE for the profile also solves the
problem.&lt;&#x2F;p&gt;
</content>
        
    </entry>
    <entry xml:lang="en">
        <title>Neutron L3 agent with multiple provider networks</title>
        <published>2014-09-29T18:11:00+00:00</published>
        <updated>2014-09-29T18:11:00+00:00</updated>
        
        <author>
          <name>
            
              Delta Regeer
            
          </name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://funcptr.net/2014/neutron-l3-agent-multiple-provider-networks/"/>
        <id>https://funcptr.net/2014/neutron-l3-agent-multiple-provider-networks/</id>
        
        <content type="html" xml:base="https://funcptr.net/2014/neutron-l3-agent-multiple-provider-networks/">&lt;p&gt;Due to requirements outside of my control, there was a requirement to run
multiple &quot;provider&quot; networks each with each providing their own floating
address pool from a single network node, I wanted to do this as simply as
possible using a single l3 agent rather than having to figure out how to get
systemd to start multiple with different configuration files.&lt;&#x2F;p&gt;
&lt;p&gt;Currently I&#x27;ve installed and configured an OpenStack instance that looks like
this:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #D6DEEB; background-color: #011627;&quot;&gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;+---------------------+&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;|                     |&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;|                  +--+----+&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;|                  |       |&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;|      +-----------+-+  +--+----------+&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;|      | Compute     |  | Compute     |&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;|      |     01      |  |     02      |&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;|      +------+------+  +-----+-------+&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;|             |               |&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;|             |               +----------+&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;|             +------------+--+          |&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;|                          |             |&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;| +-------------+    +-----+-------+     |&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;| | Controller  |    |   Network   |     |&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;| |             |    |             |     +---+  Tenant Networks (vlan tagged) (vlan ID&amp;#39;s 350 - 400)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;| +-----+----+--+    +------+----+-+&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;|       |    |              |    |&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;|       |    |              |    +-----------+  Floating Networks (vlan tagged) (vlan ID&amp;#39;s 340 - 349)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;|       |    |              |&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;|       |    |              |&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;+------------+--------------+----------------+  Management Network (10.5.2.0&#x2F;25)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;        |&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;        |&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;        +------------------------------------+  External API Network (10.5.2.128&#x2F;25)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;There are two compute nodes, a controller node that runs all of the API
services, and a network node that is strictly used for providing network
functions (routers, load balancers, firewalls, all that fun stuff!).&lt;&#x2F;p&gt;
&lt;p&gt;There are two flat networks that provide the following:&lt;&#x2F;p&gt;
&lt;ol&gt;
&lt;li&gt;External API access&lt;&#x2F;li&gt;
&lt;li&gt;A management network that OpenStack uses internally to communicate between
instances and to manage it, which is not accessible from the other three
networks.&lt;&#x2F;li&gt;
&lt;&#x2F;ol&gt;
&lt;p&gt;The other two networks are both vlan tagged:&lt;&#x2F;p&gt;
&lt;ol&gt;
&lt;li&gt;Tenant networks, with the possibility of 50 vlan ID&#x27;s&lt;&#x2F;li&gt;
&lt;li&gt;Floating networks, with existing vlan ID&#x27;s for existing networks&lt;&#x2F;li&gt;
&lt;&#x2F;ol&gt;
&lt;p&gt;Since the OpenStack Icehouse release, the l3 agent has supported the ability to
use the Open vSwitch configuration to specify how traffic should be routed
rather than statically defining that a single l3 agent routes certain traffic
to a single Linux bridge. Setting this up is fairly simple if you follow the
documentation, with one caveat, variables you think would be defined to no
value, actually have a value and thus need to be explicitly zeroed out.&lt;&#x2F;p&gt;
&lt;h2 id=&quot;on-the-network-node&quot;&gt;On the network node&lt;&#x2F;h2&gt;
&lt;p&gt;First, we need to configure the l3 agent, so let&#x27;s set some extra variables in
&lt;code&gt;&#x2F;etc&#x2F;neutron&#x2F;l3-agent.ini&lt;&#x2F;code&gt;:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #D6DEEB; background-color: #011627;&quot;&gt;&lt;code data-lang=&quot;ini&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C792EA;&quot;&gt;gateway_external_network_id&lt;&#x2F;span&gt;&lt;span&gt; =&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C792EA;&quot;&gt;external_network_bridge&lt;&#x2F;span&gt;&lt;span&gt; =&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;It is important that these two are set, not left commented out, unfortunately
when commented out they have some defaults set and it will fail to work, so
explicitly setting them to blank will fix that issue.&lt;&#x2F;p&gt;
&lt;p&gt;Next, we need to set up our Open vSwitch configuration. In
&lt;code&gt;&#x2F;etc&#x2F;neutron&#x2F;plugin.ini&lt;&#x2F;code&gt; the following needs to be configured:&lt;&#x2F;p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;bridge_mappings&lt;&#x2F;code&gt;&lt;&#x2F;li&gt;
&lt;li&gt;&lt;code&gt;network_vlan_ranges&lt;&#x2F;code&gt;&lt;&#x2F;li&gt;
&lt;&#x2F;ul&gt;
&lt;p&gt;Note, that these may already be configured, in which case there is nothing left
to do. Mine currently looks like this:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #D6DEEB; background-color: #011627;&quot;&gt;&lt;code data-lang=&quot;ini&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C792EA;&quot;&gt;bridge_mappings&lt;&#x2F;span&gt;&lt;span&gt; = tenant1:br-tnt,provider1:br-ex&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;This basically specifies that any networks created under &quot;provider name&quot;
&lt;code&gt;tenant1&lt;&#x2F;code&gt; are going to be mapped to the Open vSwitch bridge &lt;code&gt;br-tnt&lt;&#x2F;code&gt; and any
networks with &quot;provider name&quot; &lt;code&gt;provider1&lt;&#x2F;code&gt; will be mapped to &lt;code&gt;br-ex&lt;&#x2F;code&gt;.&lt;&#x2F;p&gt;
&lt;p&gt;&lt;code&gt;br-tnt&lt;&#x2F;code&gt; is mapped to my tenant network and on the switch has vlan ID&#x27;s 350 -
400 assigned, and &lt;code&gt;br-ex&lt;&#x2F;code&gt; has vlan ID&#x27;s 340 - 349 assigned.&lt;&#x2F;p&gt;
&lt;p&gt;Following the above knowledge, my &lt;code&gt;network_vlan_ranges&lt;&#x2F;code&gt; is configured as such:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #D6DEEB; background-color: #011627;&quot;&gt;&lt;code data-lang=&quot;ini&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C792EA;&quot;&gt;network_vlan_ranges&lt;&#x2F;span&gt;&lt;span&gt; = tenant1:350:400,provider1:340:349&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;Make sure to restart all neutron services:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #D6DEEB; background-color: #011627;&quot;&gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #82AAFF;font-style: italic;&quot;&gt;openstack-service&lt;&#x2F;span&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt; restart neutron&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;&lt;h2 id=&quot;on-the-controller-where-neutron-server-lives&quot;&gt;On the controller (where &lt;code&gt;neutron-server&lt;&#x2F;code&gt; lives)&lt;&#x2F;h2&gt;
&lt;p&gt;On the controller we just need to make sure that our &lt;code&gt;network_vlan_ranges&lt;&#x2F;code&gt;
matches what is on the network node, with one exception, we do not list our
&lt;code&gt;provider1&lt;&#x2F;code&gt; vlan ranges since we don&#x27;t want to make those available to
accidentally be assigned when a regular tenant creates a new network.&lt;&#x2F;p&gt;
&lt;p&gt;So our configuration should list:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #D6DEEB; background-color: #011627;&quot;&gt;&lt;code data-lang=&quot;ini&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C792EA;&quot;&gt;network_vlan_ranges&lt;&#x2F;span&gt;&lt;span&gt; = tenant1:350:400&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;Make sure that all neutron services are restarted:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #D6DEEB; background-color: #011627;&quot;&gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #82AAFF;font-style: italic;&quot;&gt;openstack-service&lt;&#x2F;span&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt; restart neutron&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;&lt;h2 id=&quot;create-the-neutron-networks&quot;&gt;Create the Neutron networks&lt;&#x2F;h2&gt;
&lt;p&gt;Now, as an administrative user we need to create the provider networks.&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #D6DEEB; background-color: #011627;&quot;&gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C5E478;font-style: italic;&quot;&gt;source&lt;&#x2F;span&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt; ~&#x2F;keystonerc_admin&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #82AAFF;font-style: italic;&quot;&gt;neutron&lt;&#x2F;span&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt; net-create&lt;&#x2F;span&gt;&lt;span style=&quot;color: #D9F5DD;&quot;&gt; &amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt;192.168.1.0&#x2F;24-floating&lt;&#x2F;span&gt;&lt;span style=&quot;color: #D9F5DD;&quot;&gt;&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #F78C6C;&quot;&gt; \&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #82AAFF;&quot;&gt;    --router:external&lt;&#x2F;span&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt; True&lt;&#x2F;span&gt;&lt;span style=&quot;color: #F78C6C;&quot;&gt; \&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #82AAFF;&quot;&gt;    --provider:network_type&lt;&#x2F;span&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt; vlan&lt;&#x2F;span&gt;&lt;span style=&quot;color: #F78C6C;&quot;&gt; \&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #82AAFF;&quot;&gt;    --provider:physical_network&lt;&#x2F;span&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt; provider1&lt;&#x2F;span&gt;&lt;span style=&quot;color: #F78C6C;&quot;&gt; \&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #82AAFF;&quot;&gt;    --provider:segmentation_id&lt;&#x2F;span&gt;&lt;span style=&quot;color: #F78C6C;&quot;&gt; 340&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #82AAFF;font-style: italic;&quot;&gt;neutron&lt;&#x2F;span&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt; net-create&lt;&#x2F;span&gt;&lt;span style=&quot;color: #D9F5DD;&quot;&gt; &amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt;192.168.2.0&#x2F;24-floating&lt;&#x2F;span&gt;&lt;span style=&quot;color: #D9F5DD;&quot;&gt;&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #F78C6C;&quot;&gt; \&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #82AAFF;&quot;&gt;    --router:external&lt;&#x2F;span&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt; True&lt;&#x2F;span&gt;&lt;span style=&quot;color: #F78C6C;&quot;&gt; \&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #82AAFF;&quot;&gt;    --provider:network_type&lt;&#x2F;span&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt; vlan&lt;&#x2F;span&gt;&lt;span style=&quot;color: #F78C6C;&quot;&gt; \&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #82AAFF;&quot;&gt;    --provider:physical_network&lt;&#x2F;span&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt; provider1&lt;&#x2F;span&gt;&lt;span style=&quot;color: #F78C6C;&quot;&gt; \&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #82AAFF;&quot;&gt;    --provider:segmentation_id&lt;&#x2F;span&gt;&lt;span style=&quot;color: #F78C6C;&quot;&gt; 341&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;Notice how we&#x27;ve created two networks, given them each individual names (I like
to use the name of the network they are going to be used for) and have been
attached to the &lt;code&gt;provider1&lt;&#x2F;code&gt;. Note that &lt;code&gt;provider1&lt;&#x2F;code&gt; is completely
administratively defined, and could just as well have been &lt;code&gt;physnet1&lt;&#x2F;code&gt;, so long
as it is consistent across all of the configuration files.&lt;&#x2F;p&gt;
&lt;p&gt;Now let&#x27;s create subnets on this network:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #D6DEEB; background-color: #011627;&quot;&gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #82AAFF;font-style: italic;&quot;&gt;neutron&lt;&#x2F;span&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt; subnet-create&lt;&#x2F;span&gt;&lt;span style=&quot;color: #D9F5DD;&quot;&gt; &amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt;192.168.1.0&#x2F;24-floating&lt;&#x2F;span&gt;&lt;span style=&quot;color: #D9F5DD;&quot;&gt;&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt; 192.168.1.0&#x2F;24&lt;&#x2F;span&gt;&lt;span style=&quot;color: #F78C6C;&quot;&gt; \&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #82AAFF;&quot;&gt;    --allocation-pool&lt;&#x2F;span&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt; start=192.168.1.4,end=&lt;&#x2F;span&gt;&lt;span style=&quot;color: #F78C6C;&quot;&gt;192.168.1.254 \&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #82AAFF;&quot;&gt;    --disable-dhcp --gateway&lt;&#x2F;span&gt;&lt;span style=&quot;color: #F78C6C;&quot;&gt; 192.168.1.1&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #82AAFF;font-style: italic;&quot;&gt;neutron&lt;&#x2F;span&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt; subnet-create&lt;&#x2F;span&gt;&lt;span style=&quot;color: #D9F5DD;&quot;&gt; &amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt;192.168.2.0&#x2F;24-floating&lt;&#x2F;span&gt;&lt;span style=&quot;color: #D9F5DD;&quot;&gt;&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt; 192.168.2.0&#x2F;24&lt;&#x2F;span&gt;&lt;span style=&quot;color: #F78C6C;&quot;&gt; \&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #82AAFF;&quot;&gt;    --allocation-pool&lt;&#x2F;span&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt; start=192.168.2.4,end=&lt;&#x2F;span&gt;&lt;span style=&quot;color: #F78C6C;&quot;&gt;192.168.2.254 \&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #82AAFF;&quot;&gt;    --disable-dhcp --gateway&lt;&#x2F;span&gt;&lt;span style=&quot;color: #F78C6C;&quot;&gt; 192.168.2.1&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;Now that these networks are defined, you should be able to have tenants create
routers and set their gateways to either of these new networks by selecting
from the drop-down in Horizon or by calling &lt;code&gt;neutron router-gateway-set &amp;lt;router id&amp;gt; &amp;lt;network id&amp;gt;&lt;&#x2F;code&gt; on the command line.&lt;&#x2F;p&gt;
&lt;p&gt;The l3 agent will automatically configure and set up the router as required on
the network node, and traffic will flow to either vlan 340 or vlan 341 as
defined above depending on what floating network the user uses as a gateway.&lt;&#x2F;p&gt;
&lt;p&gt;This drastically simplifies the configuration of multiple floating IP networks
since no longer is there a requirement to start up and configure multiple l3
agents each with their own network ID configured. This makes configuration less
brittle and easier to maintain over time.&lt;&#x2F;p&gt;
</content>
        
    </entry>
    <entry xml:lang="en">
        <title>OpenStack resizing of instances</title>
        <published>2014-09-29T17:10:00+00:00</published>
        <updated>2014-09-29T17:10:00+00:00</updated>
        
        <author>
          <name>
            
              Delta Regeer
            
          </name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://funcptr.net/2014/openstack-resizing-instances/"/>
        <id>https://funcptr.net/2014/openstack-resizing-instances/</id>
        
        <content type="html" xml:base="https://funcptr.net/2014/openstack-resizing-instances/">&lt;p&gt;One thing that is not always adequately explained in the OpenStack
documentation is how exactly instance resizing works, and what is required,
especially while using KVM as the virtualisation provider, with multiple
compute nodes.&lt;&#x2F;p&gt;
&lt;p&gt;You might find something similiar to the following in your logs, and no good
documentation on how to fix it.&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #D6DEEB; background-color: #011627;&quot;&gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;ERROR nova.compute.manager [req-7cb1c029-beb4-4905-a9d9-62d488540eda f542d1b5afeb4908b8b132c4486f9fa8 c2bfab5ad24642359f43cdff9bb00047] [instance: 99736f90-db0f-4cba-8f44-a73a603eee0b] Setting instance vm_state to ERROR&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;TRACE nova.compute.manager [instance: 99736f90-db0f-4cba-8f44-a73a603eee0b] Traceback (most recent call last):&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;TRACE nova.compute.manager [instance: 99736f90-db0f-4cba-8f44-a73a603eee0b]   File &amp;quot;&#x2F;usr&#x2F;lib&#x2F;python2.7&#x2F;site-packages&#x2F;nova&#x2F;compute&#x2F;manager.py&amp;quot;, line 5596, in _error_out_instance_on_exception&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;TRACE nova.compute.manager [instance: 99736f90-db0f-4cba-8f44-a73a603eee0b]     yield&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;TRACE nova.compute.manager [instance: 99736f90-db0f-4cba-8f44-a73a603eee0b]   File &amp;quot;&#x2F;usr&#x2F;lib&#x2F;python2.7&#x2F;site-packages&#x2F;nova&#x2F;compute&#x2F;manager.py&amp;quot;, line 3459, in resize_instance&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;TRACE nova.compute.manager [instance: 99736f90-db0f-4cba-8f44-a73a603eee0b]     block_device_info)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;TRACE nova.compute.manager [instance: 99736f90-db0f-4cba-8f44-a73a603eee0b]   File &amp;quot;&#x2F;usr&#x2F;lib&#x2F;python2.7&#x2F;site-packages&#x2F;nova&#x2F;virt&#x2F;libvirt&#x2F;driver.py&amp;quot;, line 4980, in migrate_disk_and_power_off&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;TRACE nova.compute.manager [instance: 99736f90-db0f-4cba-8f44-a73a603eee0b]     utils.execute(&amp;#39;ssh&amp;#39;, dest, &amp;#39;mkdir&amp;#39;, &amp;#39;-p&amp;#39;, inst_base)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;TRACE nova.compute.manager [instance: 99736f90-db0f-4cba-8f44-a73a603eee0b]   File &amp;quot;&#x2F;usr&#x2F;lib&#x2F;python2.7&#x2F;site-packages&#x2F;nova&#x2F;utils.py&amp;quot;, line 165, in execute&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;TRACE nova.compute.manager [instance: 99736f90-db0f-4cba-8f44-a73a603eee0b]     return processutils.execute(*cmd, **kwargs)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;TRACE nova.compute.manager [instance: 99736f90-db0f-4cba-8f44-a73a603eee0b]   File &amp;quot;&#x2F;usr&#x2F;lib&#x2F;python2.7&#x2F;site-packages&#x2F;nova&#x2F;openstack&#x2F;common&#x2F;processutils.py&amp;quot;, line 193, in execute&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;TRACE nova.compute.manager [instance: 99736f90-db0f-4cba-8f44-a73a603eee0b]     cmd=&amp;#39; &amp;#39;.join(cmd))&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;TRACE nova.compute.manager [instance: 99736f90-db0f-4cba-8f44-a73a603eee0b] ProcessExecutionError: Unexpected error while running command.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;TRACE nova.compute.manager [instance: 99736f90-db0f-4cba-8f44-a73a603eee0b] Command: ssh 10.5.2.20 mkdir -p &#x2F;var&#x2F;lib&#x2F;nova&#x2F;instances&#x2F;99736f90-db0f-4cba-8f44-a73a603eee0b&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;TRACE nova.compute.manager [instance: 99736f90-db0f-4cba-8f44-a73a603eee0b] Exit code: 255&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;TRACE nova.compute.manager [instance: 99736f90-db0f-4cba-8f44-a73a603eee0b] Stdout: &amp;#39;&amp;#39;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;TRACE nova.compute.manager [instance: 99736f90-db0f-4cba-8f44-a73a603eee0b] Stderr: &amp;#39;Host key verification failed.\r\n&amp;#39;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;TRACE nova.compute.manager [instance: 99736f90-db0f-4cba-8f44-a73a603eee0b] &lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;ERROR oslo.messaging.rpc.dispatcher [-] Exception during message handling: Unexpected error while running command.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Command: ssh 10.5.2.20 mkdir -p &#x2F;var&#x2F;lib&#x2F;nova&#x2F;instances&#x2F;99736f90-db0f-4cba-8f44-a73a603eee0b&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Exit code: 255&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Stdout: &amp;#39;&amp;#39;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;Stderr: &amp;#39;Host key verification failed.\r\n&amp;#39;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;When OpenStack&#x27;s nova is instructed to resize an instance it will also change
the host it is running on, almost never will it schedule the instance on the
same host and do the resize on the same host it already exists. There is a
configuration flag to change this, however in my case I would rather the
scheduler be run again, especially if the instance size is changing
drastically. During the resize process, the node where the instance is
currently running will use SSH to connect to the instance where the resized
instance will live, and copy over the instance and associated files.&lt;&#x2F;p&gt;
&lt;p&gt;There are a couple of assumptions I will be making:&lt;&#x2F;p&gt;
&lt;ol&gt;
&lt;li&gt;Your &lt;code&gt;nova&lt;&#x2F;code&gt;, and &lt;code&gt;qemu&lt;&#x2F;code&gt; user both have the same UID on all compute nodes&lt;&#x2F;li&gt;
&lt;li&gt;The path for your instances is the same on all of your compute nodes&lt;&#x2F;li&gt;
&lt;&#x2F;ol&gt;
&lt;h2 id=&quot;configure-the-nova-user&quot;&gt;Configure the nova user&lt;&#x2F;h2&gt;
&lt;p&gt;First things first, let&#x27;s make sure our &lt;code&gt;nova&lt;&#x2F;code&gt; user has an appropriate shell set:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #D6DEEB; background-color: #011627;&quot;&gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #82AAFF;font-style: italic;&quot;&gt;cat&lt;&#x2F;span&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt; &#x2F;etc&#x2F;passwd&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7FDBCA;&quot;&gt; |&lt;&#x2F;span&gt;&lt;span style=&quot;color: #82AAFF;font-style: italic;&quot;&gt; grep&lt;&#x2F;span&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt; nova&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;Verify that the last entry is &lt;code&gt;&#x2F;bin&#x2F;bash&lt;&#x2F;code&gt;.&lt;&#x2F;p&gt;
&lt;p&gt;If not, let&#x27;s modify the user and make it so:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #D6DEEB; background-color: #011627;&quot;&gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #82AAFF;font-style: italic;&quot;&gt;usermod&lt;&#x2F;span&gt;&lt;span style=&quot;color: #82AAFF;&quot;&gt; -s&lt;&#x2F;span&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt; &#x2F;bin&#x2F;bash nova&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;&lt;h2 id=&quot;generate-ssh-key-and-configuration&quot;&gt;Generate SSH key and configuration&lt;&#x2F;h2&gt;
&lt;p&gt;After doing this the next steps are all run as the &lt;code&gt;nova&lt;&#x2F;code&gt; user.&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #D6DEEB; background-color: #011627;&quot;&gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #82AAFF;font-style: italic;&quot;&gt;su&lt;&#x2F;span&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt; - nova&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;We need to generate an SSH key:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #D6DEEB; background-color: #011627;&quot;&gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #82AAFF;font-style: italic;&quot;&gt;ssh-keygen&lt;&#x2F;span&gt;&lt;span style=&quot;color: #82AAFF;&quot;&gt; -t&lt;&#x2F;span&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt; rsa&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;Follow the directions, and save the key WITHOUT a passphrase.&lt;&#x2F;p&gt;
&lt;p&gt;Next up we need to configure SSH to not do host key verification, unless you
want to manually SSH to all compute nodes that exist and accept the key (and
continue to do so for each new compute node you add).&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #D6DEEB; background-color: #011627;&quot;&gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #82AAFF;font-style: italic;&quot;&gt;cat&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7FDBCA;&quot;&gt; &amp;lt;&amp;lt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #D9F5DD;&quot;&gt; EOF&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7FDBCA;&quot;&gt; &amp;gt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt; ~&#x2F;.ssh&#x2F;config&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt;Host *&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt;    StrictHostKeyChecking no&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt;    UserKnownHostsFile=&#x2F;dev&#x2F;null&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #D9F5DD;&quot;&gt;EOF&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;Next we need to make sure we copy the the contents of &lt;code&gt;id_rsa.pub&lt;&#x2F;code&gt; to
&lt;code&gt;authorized_keys&lt;&#x2F;code&gt; and set the mode on it correctly.&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #D6DEEB; background-color: #011627;&quot;&gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #82AAFF;font-style: italic;&quot;&gt;cat&lt;&#x2F;span&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt; ~&#x2F;.ssh&#x2F;id_rsa.pub&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7FDBCA;&quot;&gt; &amp;gt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt; .ssh&#x2F;authorized_keys&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #82AAFF;font-style: italic;&quot;&gt;chmod&lt;&#x2F;span&gt;&lt;span style=&quot;color: #F78C6C;&quot;&gt; 600&lt;&#x2F;span&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt; .ssh&#x2F;authorized_keys&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;This should be all the configuration for SSH you need to do. Now comes the
import part, you will need to tar up and copy the &lt;code&gt;~nova&#x2F;.ssh&lt;&#x2F;code&gt; directory to
every single compute node you have provisioned. This way all compute nodes will
be able to SSH to the remote host to run the commands required to copy an
instance over, and resize it.&lt;&#x2F;p&gt;
&lt;h2 id=&quot;reset-state-on-existing-error-ed-instances&quot;&gt;Reset state on existing ERROR&#x27;ed instances&lt;&#x2F;h2&gt;
&lt;p&gt;If you have any instances that are currently in the &lt;code&gt;ERROR&lt;&#x2F;code&gt; state due to a
failed resize, you will be able to issue the following command to reset the
state back to running and try again:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #D6DEEB; background-color: #011627;&quot;&gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #82AAFF;font-style: italic;&quot;&gt;nova&lt;&#x2F;span&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt; reset-state&lt;&#x2F;span&gt;&lt;span style=&quot;color: #82AAFF;&quot;&gt; --active&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7FDBCA;&quot;&gt; &amp;lt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt;ID of instanc&lt;&#x2F;span&gt;&lt;span&gt;e&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7FDBCA;&quot;&gt;&amp;gt;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;This will start the instance, and you will be able to once again issue the
resize command to resize the instance.&lt;&#x2F;p&gt;
</content>
        
    </entry>
    <entry xml:lang="en">
        <title>IPv6 -- getaddrinfo() and bind() ordering with V6ONLY</title>
        <published>2012-08-07T22:39:56+00:00</published>
        <updated>2012-08-07T22:39:56+00:00</updated>
        
        <author>
          <name>
            
              Delta Regeer
            
          </name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://funcptr.net/2012/ipv6-and-getaddrinfo/"/>
        <id>https://funcptr.net/2012/ipv6-and-getaddrinfo/</id>
        
        <content type="html" xml:base="https://funcptr.net/2012/ipv6-and-getaddrinfo/">&lt;p&gt;Recently I ran into an issue that took me a while to sort out, and it is
regarding inconsistent behaviour on various OS&#x27;s with regards to IPv6 sockets
(&lt;a rel=&quot;external&quot; href=&quot;http:&#x2F;&#x2F;pubs.opengroup.org&#x2F;onlinepubs&#x2F;9699919799&#x2F;basedefs&#x2F;sys_socket.h.html&quot;&gt;&lt;code&gt;AF_INET6&lt;&#x2F;code&gt;&lt;&#x2F;a&gt;&lt;sup class=&quot;footnote-reference&quot; id=&quot;fr-20120807afvspf-1&quot;&gt;&lt;a href=&quot;#fn-20120807afvspf&quot;&gt;1&lt;&#x2F;a&gt;&lt;&#x2F;sup&gt;) and calling &lt;a rel=&quot;external&quot; href=&quot;http:&#x2F;&#x2F;www.freebsd.org&#x2F;cgi&#x2F;man.cgi?query=bind&amp;amp;apropos=0&amp;amp;sektion=2&amp;amp;manpath=FreeBSD+9.0-RELEASE&amp;amp;arch=default&amp;amp;format=html&quot;&gt;&lt;code&gt;bind(2)&lt;&#x2F;code&gt;&lt;&#x2F;a&gt; after getting the
results back from &lt;a rel=&quot;external&quot; href=&quot;http:&#x2F;&#x2F;www.freebsd.org&#x2F;cgi&#x2F;man.cgi?query=getaddrinfo&amp;amp;apropos=0&amp;amp;sektion=3&amp;amp;manpath=FreeBSD+9.0-RELEASE&amp;amp;arch=default&amp;amp;format=html&quot;&gt;&lt;code&gt;getaddrinfo(3)&lt;&#x2F;code&gt;&lt;&#x2F;a&gt;.&lt;&#x2F;p&gt;
&lt;p&gt;A call to &lt;code&gt;getaddrinfo()&lt;&#x2F;code&gt; with the hints set to &lt;a rel=&quot;external&quot; href=&quot;http:&#x2F;&#x2F;pubs.opengroup.org&#x2F;onlinepubs&#x2F;9699919799&#x2F;basedefs&#x2F;sys_socket.h.html&quot;&gt;&lt;code&gt;AF_UNSPEC&lt;&#x2F;code&gt;&lt;&#x2F;a&gt; in &lt;code&gt;ai_family&lt;&#x2F;code&gt;
and &lt;a rel=&quot;external&quot; href=&quot;http:&#x2F;&#x2F;www.freebsd.org&#x2F;cgi&#x2F;man.cgi?query=getaddrinfo&amp;amp;apropos=0&amp;amp;sektion=3&amp;amp;manpath=FreeBSD+9.0-RELEASE&amp;amp;arch=default&amp;amp;format=html&quot;&gt;&lt;code&gt;AI_PASSIVE&lt;&#x2F;code&gt;&lt;&#x2F;a&gt; in &lt;code&gt;ai_flags&lt;&#x2F;code&gt; will return to us 1 or more results that we
can &lt;code&gt;bind()&lt;&#x2F;code&gt; to. Sample code for that looks like this:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #D6DEEB; background-color: #011627;&quot;&gt;&lt;code data-lang=&quot;c&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C792EA;&quot;&gt;struct&lt;&#x2F;span&gt;&lt;span&gt; addrinfo hints,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7FDBCA;&quot;&gt; *&lt;&#x2F;span&gt;&lt;span&gt;addrlist;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #82AAFF;font-style: italic;&quot;&gt;memset&lt;&#x2F;span&gt;&lt;span&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7FDBCA;&quot;&gt;&amp;amp;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #D7DBE0;&quot;&gt;hints&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #F78C6C;&quot;&gt; 0&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7FDBCA;&quot;&gt; sizeof&lt;&#x2F;span&gt;&lt;span&gt;(hints));&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #637777;font-style: italic;&quot;&gt;&#x2F;&#x2F; Ask for TCP&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;hints.ai_socktype &lt;&#x2F;span&gt;&lt;span style=&quot;color: #C792EA;&quot;&gt;=&lt;&#x2F;span&gt;&lt;span&gt; SOCK_STREAM;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #637777;font-style: italic;&quot;&gt;&#x2F;&#x2F; Any family works for us ...&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;hints.ai_family &lt;&#x2F;span&gt;&lt;span style=&quot;color: #C792EA;&quot;&gt;=&lt;&#x2F;span&gt;&lt;span&gt; AF_UNSPEC;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #637777;font-style: italic;&quot;&gt;&#x2F;&#x2F; Set some hints&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;hints.ai_flags &lt;&#x2F;span&gt;&lt;span style=&quot;color: #C792EA;&quot;&gt;=&lt;&#x2F;span&gt;&lt;span&gt; &lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    AI_PASSIVE    &lt;&#x2F;span&gt;&lt;span style=&quot;color: #7FDBCA;&quot;&gt;|&lt;&#x2F;span&gt;&lt;span style=&quot;color: #637777;font-style: italic;&quot;&gt; &#x2F;&#x2F; We want to use this with bind&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    AI_ADDRCONFIG;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #637777;font-style: italic;&quot;&gt;  &#x2F;&#x2F; Only return IPv4 or IPv6 if they are configured&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C792EA;&quot;&gt;int&lt;&#x2F;span&gt;&lt;span&gt; rv;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C792EA;font-style: italic;&quot;&gt;if&lt;&#x2F;span&gt;&lt;span&gt; ((rv &lt;&#x2F;span&gt;&lt;span style=&quot;color: #C792EA;&quot;&gt;=&lt;&#x2F;span&gt;&lt;span style=&quot;color: #82AAFF;font-style: italic;&quot;&gt; getaddrinfo&lt;&#x2F;span&gt;&lt;span&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #F78C6C;&quot;&gt;0&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #D9F5DD;&quot;&gt; &amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt;7020&lt;&#x2F;span&gt;&lt;span style=&quot;color: #D9F5DD;&quot;&gt;&amp;quot;&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7FDBCA;&quot;&gt; &amp;amp;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #D7DBE0;&quot;&gt;hints&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7FDBCA;&quot;&gt; &amp;amp;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #D7DBE0;&quot;&gt;addrlist&lt;&#x2F;span&gt;&lt;span&gt;))&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C792EA;&quot;&gt; !=&lt;&#x2F;span&gt;&lt;span style=&quot;color: #F78C6C;&quot;&gt; 0&lt;&#x2F;span&gt;&lt;span&gt;) {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #82AAFF;font-style: italic;&quot;&gt;    fprintf&lt;&#x2F;span&gt;&lt;span style=&quot;color: #82AAFF;&quot;&gt;(stderr,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #D9F5DD;&quot;&gt; &amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt;getaddrinfo: &lt;&#x2F;span&gt;&lt;span style=&quot;color: #82AAFF;&quot;&gt;%s&lt;&#x2F;span&gt;&lt;span style=&quot;color: #D9F5DD;&quot;&gt;&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #82AAFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #82AAFF;font-style: italic;&quot;&gt; gai_strerror&lt;&#x2F;span&gt;&lt;span style=&quot;color: #82AAFF;&quot;&gt;(rv))&lt;&#x2F;span&gt;&lt;span&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C792EA;font-style: italic;&quot;&gt;    return&lt;&#x2F;span&gt;&lt;span style=&quot;color: #F78C6C;&quot;&gt; 1&lt;&#x2F;span&gt;&lt;span&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;}&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #637777;font-style: italic;&quot;&gt;&#x2F;&#x2F; Use the list in *addrlist&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C792EA;font-style: italic;&quot;&gt;for&lt;&#x2F;span&gt;&lt;span&gt; (addr &lt;&#x2F;span&gt;&lt;span style=&quot;color: #C792EA;&quot;&gt;=&lt;&#x2F;span&gt;&lt;span&gt; addrlist; addr &lt;&#x2F;span&gt;&lt;span style=&quot;color: #C792EA;&quot;&gt;!=&lt;&#x2F;span&gt;&lt;span style=&quot;color: #F78C6C;&quot;&gt; 0&lt;&#x2F;span&gt;&lt;span&gt;; addr &lt;&#x2F;span&gt;&lt;span style=&quot;color: #C792EA;&quot;&gt;=&lt;&#x2F;span&gt;&lt;span&gt; addr&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7FDBCA;&quot;&gt;-&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C792EA;&quot;&gt;&amp;gt;&lt;&#x2F;span&gt;&lt;span&gt;ai_next) {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #637777;font-style: italic;&quot;&gt;    &#x2F;&#x2F; use *addr as appropriate&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;}&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #637777;font-style: italic;&quot;&gt;&#x2F;&#x2F; Clean up the memory from getaddrinfo()&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #82AAFF;font-style: italic;&quot;&gt;freeaddrinfo&lt;&#x2F;span&gt;&lt;span&gt;(addrlist);&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;On Linux there are two entries returned when the host it is run on has both
IPv4 and IPv6 enabled. An &lt;code&gt;AF_INET&lt;&#x2F;code&gt; which was followed by an &lt;code&gt;AF_INET6&lt;&#x2F;code&gt;. Now,
it is not said that you are required to use all of the results that are
returned, but if you want to listen on all address families it is off course
suggested.&lt;&#x2F;p&gt;
&lt;p&gt;Following the steps below for each of the returned results should result in
having 1 or more different sockets that are bound to a single port.&lt;&#x2F;p&gt;
&lt;ol&gt;
&lt;li&gt;Create the socket()&lt;&#x2F;li&gt;
&lt;li&gt;Set any socket options you want (&lt;code&gt;SO_REUSEADDR&lt;&#x2F;code&gt; for example)&lt;&#x2F;li&gt;
&lt;li&gt;Then bind() the socket&lt;&#x2F;li&gt;
&lt;li&gt;After that call listen() (followed off course by accept() on the socket)&lt;&#x2F;li&gt;
&lt;&#x2F;ol&gt;
&lt;p&gt;Only for some unknown reason (and &lt;code&gt;errno&lt;&#x2F;code&gt; is no help) &lt;code&gt;bind()&lt;&#x2F;code&gt; fails when you
get to the &lt;code&gt;AF_INET6&lt;&#x2F;code&gt;, which was returned second. Searching online as to why
the bind would fail doesn&#x27;t give you any good results and the thing that is
even worse is that if you run the same code on another platform such as
&lt;a rel=&quot;external&quot; href=&quot;http:&#x2F;&#x2F;www.freebsd.org&quot;&gt;FreeBSD&lt;&#x2F;a&gt;, &lt;a rel=&quot;external&quot; href=&quot;http:&#x2F;&#x2F;openindiana.org&quot;&gt;OpenIndiana&lt;&#x2F;a&gt; or &lt;a rel=&quot;external&quot; href=&quot;https:&#x2F;&#x2F;web.archive.org&#x2F;web&#x2F;20120808044613&#x2F;http:&#x2F;&#x2F;www.apple.com&#x2F;osx&#x2F;&quot;&gt;Mac OS X&lt;&#x2F;a&gt; no such failure exists. However
I started suspecting something was up when I started looking at the output from
&lt;code&gt;netstat -lan | grep 7020&lt;&#x2F;code&gt; on Mac OS X. Where 7020 is the port I passed into
&lt;code&gt;getaddrinfo()&lt;&#x2F;code&gt;.&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #D6DEEB; background-color: #011627;&quot;&gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;tcp46      0      0  *.7020                 *.*  LISTEN     &lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;tcp4       0      0  *.7020                 *.*  LISTEN &lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;Wait a minute ... one of the sockets is on both IPv4 and on IPv6. Some more
time spent searching the internet I came across &lt;a rel=&quot;external&quot; href=&quot;http:&#x2F;&#x2F;tools.ietf.org&#x2F;html&#x2F;rfc3493#page-22&quot;&gt;RFC 3493 section 5.3&lt;&#x2F;a&gt;,
which is titled &quot;&lt;code&gt;IPV6_V6ONLY&lt;&#x2F;code&gt; option for &lt;code&gt;AF_INET6&lt;&#x2F;code&gt; Sockets&quot;.&lt;&#x2F;p&gt;
&lt;blockquote&gt;
&lt;p&gt;As stated in section &amp;lt;3.7 Compatibility with IPv4 Nodes&amp;gt;, &lt;code&gt;AF_INET6&lt;&#x2F;code&gt; sockets
may be used for both IPv4 and IPv6 communications.  Some applications may
want to restrict their use of an &lt;code&gt;AF_INET6&lt;&#x2F;code&gt; socket to IPv6 communications
only.&lt;&#x2F;p&gt;
&lt;&#x2F;blockquote&gt;
&lt;p&gt;This was going down the right route, so I changed my code so that in the steps
listed above in number 2 I added the following code if the socket type is
&lt;code&gt;AF_INET6&lt;&#x2F;code&gt;:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #D6DEEB; background-color: #011627;&quot;&gt;&lt;code data-lang=&quot;c&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C792EA;font-style: italic;&quot;&gt;if&lt;&#x2F;span&gt;&lt;span&gt; (&lt;&#x2F;span&gt;&lt;span style=&quot;color: #82AAFF;font-style: italic;&quot;&gt;setsockopt&lt;&#x2F;span&gt;&lt;span&gt;(sockfd, IPPROTO_IPV6, IPV6_V6ONLY,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7FDBCA;&quot;&gt; &amp;amp;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #D7DBE0;&quot;&gt;yes&lt;&#x2F;span&gt;&lt;span&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7FDBCA;&quot;&gt; sizeof&lt;&#x2F;span&gt;&lt;span&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C792EA;&quot;&gt;int&lt;&#x2F;span&gt;&lt;span&gt;))&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C792EA;&quot;&gt; ==&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7FDBCA;&quot;&gt; -&lt;&#x2F;span&gt;&lt;span style=&quot;color: #F78C6C;&quot;&gt;1&lt;&#x2F;span&gt;&lt;span&gt;) {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #82AAFF;font-style: italic;&quot;&gt;    close&lt;&#x2F;span&gt;&lt;span style=&quot;color: #82AAFF;&quot;&gt;(sockfd)&lt;&#x2F;span&gt;&lt;span&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #82AAFF;font-style: italic;&quot;&gt;    fprintf&lt;&#x2F;span&gt;&lt;span style=&quot;color: #82AAFF;&quot;&gt;(stderr,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #D9F5DD;&quot;&gt; &amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt;setsockopt: &lt;&#x2F;span&gt;&lt;span style=&quot;color: #82AAFF;&quot;&gt;%s&lt;&#x2F;span&gt;&lt;span style=&quot;color: #ECC48D;&quot;&gt; IPV6_V6ONLY&lt;&#x2F;span&gt;&lt;span style=&quot;color: #F78C6C;&quot;&gt;\n&lt;&#x2F;span&gt;&lt;span style=&quot;color: #D9F5DD;&quot;&gt;&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #82AAFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #82AAFF;font-style: italic;&quot;&gt; strerror&lt;&#x2F;span&gt;&lt;span style=&quot;color: #82AAFF;&quot;&gt;(errno))&lt;&#x2F;span&gt;&lt;span&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C792EA;font-style: italic;&quot;&gt;    continue&lt;&#x2F;span&gt;&lt;span&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;}&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;The RFC 3493 section 5.3 also states that this option should be turned off by
default, which means that all IPv6 sockets can also communicate over IPv4. Thus
technically setting the option manually in code the best way to fix the issue.
FreeBSD has had this feature turned on (as in IPv6 sockets can only communicate
with IPv6 and NOT IPv4) since 5.x.&lt;&#x2F;p&gt;
&lt;p&gt;The biggest issue is that the remaining operating systems (OS X and
OpenIndiana) don&#x27;t have the same behaviour as Linux which makes troubleshooting
this issue more difficult than it should be. The issue is that the RFC doesn&#x27;t
specify what exactly the operating should do when it encounters a request to
bind to the same port on IPv4 and IPv6. The only place where I have found this
documented is in &lt;a rel=&quot;external&quot; href=&quot;http:&#x2F;&#x2F;www.amazon.com&#x2F;Network-Programming-Jun-ichiro-itojun-Hagino&#x2F;dp&#x2F;1555583180&quot;&gt;&quot;IPv6 Network Programming&quot;&lt;&#x2F;a&gt; under &quot;Tips in IPv6
Programming&quot; chapter 4, section 4, appropriately titled &quot;bind(2) Ordering and
Conflicts&quot;.&lt;&#x2F;p&gt;
&lt;hr &#x2F;&gt;
&lt;p&gt;If you get a &lt;code&gt;bind()&lt;&#x2F;code&gt; error when attempting to bind to an &lt;code&gt;AF_INET6&lt;&#x2F;code&gt; socket
please make sure that you set the socket option &lt;code&gt;IPV6_V6ONLY&lt;&#x2F;code&gt; on the &lt;code&gt;AF_INET6&lt;&#x2F;code&gt;
socket. The default as required by RFC 3493 is to have that option be off. The
default is wrong, and the RFC should have been more specific regarding what the
right behaviour is when attempting to bind on an &lt;code&gt;AF_INET6&lt;&#x2F;code&gt; socket when already
bound on an &lt;code&gt;AF_INET&lt;&#x2F;code&gt; while &lt;code&gt;IPV6_V6ONLY&lt;&#x2F;code&gt; is set to false.&lt;&#x2F;p&gt;
&lt;p&gt;The full code that I used for testing, along with a little bit more information
is available as a &lt;a rel=&quot;external&quot; href=&quot;https:&#x2F;&#x2F;gist.github.com&#x2F;3231892&quot;&gt;gist on github&lt;&#x2F;a&gt;.&lt;&#x2F;p&gt;
&lt;section class=&quot;footnotes&quot;&gt;
&lt;ol class=&quot;footnotes-list&quot;&gt;
&lt;li id=&quot;fn-20120807afvspf&quot;&gt;
&lt;p&gt;The old BSD style &lt;a rel=&quot;external&quot; href=&quot;http:&#x2F;&#x2F;www.freebsd.org&#x2F;cgi&#x2F;man.cgi?query=socket&amp;amp;apropos=0&amp;amp;sektion=2&amp;amp;manpath=FreeBSD+9.0-RELEASE&amp;amp;arch=default&amp;amp;format=html&quot;&gt;&lt;code&gt;socket()&lt;&#x2F;code&gt;&lt;&#x2F;a&gt; called for defines
starting with &lt;code&gt;PF_&lt;&#x2F;code&gt; such as &lt;code&gt;PF_INET&lt;&#x2F;code&gt; and &lt;code&gt;PF_INET6&lt;&#x2F;code&gt; with the &lt;code&gt;PF&lt;&#x2F;code&gt; standing for
protocol family. POSIX starts them with &lt;code&gt;AF_&lt;&#x2F;code&gt;, and calls them an address
family. On almost every operating system &lt;code&gt;PF_INET&lt;&#x2F;code&gt; is the same as &lt;code&gt;AF_INET&lt;&#x2F;code&gt;. If
the define doesn&#x27;t exist you can always create it. &lt;a href=&quot;#fr-20120807afvspf-1&quot;&gt;↩&lt;&#x2F;a&gt;&lt;&#x2F;p&gt;
&lt;&#x2F;li&gt;
&lt;&#x2F;ol&gt;
&lt;&#x2F;section&gt;
</content>
        
    </entry>
</feed>
