Add IPv6 support

[kpfleming]

This patch adds substantial support for use of IPv6 in the installer,
and in the installed system. It supports static configuration, although
most IPv6 networks use automatic (and dynamic) configuration. This
patch does not provide DHCPv6 support as there is no suitable DHCPv6
client available to use in the installer's environment, but it does
provide support for Router Advertisements, SLAAC, and RDNSS.

In order to simplify management of DNS resolver information in the
installed system (since there are multiple potential sources of this
information), this patch adds the 'resolvconf' package to the installed
system. If IPv6 DNS information is gathered from RDNSS, the 'rdnssd'
package will also be added to the installed system.

For discovery of Router Advertisment-provided information during the
installation, the patch also adds tools from the 'ndisc6' package to
the installer.

Finally, to provide consistent syntax in the installer configuration
file, this patch changes the 'ip_*' options to use an 'ip4_' prefix
instead, and also uses 'prefix length' instead of 'netmask' for
IPv4 networks. It supports prior configuration files, though, and
will automatically convert options in the old format.

[kpfleming]

I have tested this branch with IPv4 disabled, but was unable to complete an installation in that configuration. This is not due to problems in the installer itself, but rather in various tools used during the installation (including cURL), which try to contact servers over IPv4 and don't retry when that fails. There are possible ways to address this, but I wanted to get this merged first as it represents the bulk of the work and IPv6-only networks (without NAT64/DNS64 for access to IPv4 sites) are rare.

[Mausy5043]

Impressive 👍

[Mausy5043]

These two spaces force a newline.

[Mausy5043]

Please re-look at L1019 and L1027. I see no change.

[kpfleming]

Ahh yes, it's amazing what happens when you modify two files but only git add one of them :-)

[Mausy5043]

👍

[Mausy5043]

@diederikdehaas PR is good to go as far as I'm concerned.

[Mausy5043]

@diederikdehaas bump

[diederikdehaas]

I plan to (finally) work on this the coming week.

[diederikdehaas]

I finally got around to testing this, sorry for that :-(

The apt-get update step near the end of the installation fails as dns doesn't seem to be working and as a consequence so does the installation of the firmware and kernel.
Is that expected?
Once the system is rebooted, dns does work though.

[diederikdehaas]

The reason this PR got assigned to me was because I was triggered by something you said somewhere else and that was that for a certain feature some big changes were needed in the installer.
(as you can probably tell, I can't find that statement atm)

What were those changes you had in mind?

[kpfleming]

I'm fairly certain this is the feature that I was referring to, and the other changes were the additions of some IPv6-specific tools to the installer (which have now been incorporated into this PR instead of the separate PRs they were in previously).

[kpfleming]

What was the network configuration in your test where DNS was failing? IPv4 only, IPv4+6, DHCP, ?

[diederikdehaas]

IPv4 only, DHCP

My initial guess is that resolvconf isn't working yet when it chroots into the target system

[diederikdehaas]

The issue is only visible by the FAILED word in the log file, but if you apply the following patch, you'll get full details:

diederik@bagend:~/dev/raspberry-pi/raspbian-ua-netinst$ git diff
diff --git a/scripts/etc/init.d/rcS b/scripts/etc/init.d/rcS
index 8f82c20..96d0488 100755
--- a/scripts/etc/init.d/rcS
+++ b/scripts/etc/init.d/rcS
@@ -1253,13 +1253,13 @@ cd $old_dir || fail
 
 
 echo -n "Updating package lists... "
-chroot /rootfs /usr/bin/apt-get update &>/dev/null
+chroot /rootfs /usr/bin/apt-get update 
 echo "OK"
 
 # firmware package can't be installed during cdebootstrap phase, so do so now
 if [ "$kernel_module" = true ] ; then
   echo -n "Installing bootloader package (=firmware)... "
-  chroot /rootfs /usr/bin/apt-get -y install raspberrypi-bootloader-nokernel &>/dev/null
+  chroot /rootfs /usr/bin/apt-get -y install raspberrypi-bootloader-nokernel 
   if [ $? -eq 0 ]; then
     echo "OK"
   else
@@ -1269,7 +1269,7 @@ fi
 
 # install libraspberrypi-bin to provide vcgencmd functionality
 echo -n "Installing libraspberrypi-bin package (=vcgencmd,raspistill,raspivid,etc)... "
-chroot /rootfs /usr/bin/apt-get -y install libraspberrypi-bin &>/dev/null
+chroot /rootfs /usr/bin/apt-get -y install libraspberrypi-bin 
 if [ $? -eq 0 ]; then
     echo "OK"
 else

[kpfleming]

Thanks for the details, will investigate soon.

[diederikdehaas]

I will too. I asked first as I wasn't sure if this was an expected issue or not.

The resolvconf package installs a service and cdebootstrap-helper-rc.d prevents it from starting.
IIRC there was a way around that, but I'll have to dig to find it again.

[kpfleming]

I suspect we'll need to just copy the /etc/resolv.conf from the installer into the rootfs for use by the chroot-ed steps, then it will be overwritten by resolvconf when the system is booted the first time.

[kpfleming]

OK, I'm 99% certain that is indeed the cause of this problem, although looking through rcS in the v1.1.x branch I don't actually understand how it's working now, since the installer's resolv.conf is only copied into the rootfs when DHCP Is not being used. When it is, the DHCP client has populated /etc/resolv.conf with information it got from the DHCP server, but that doesn't get copied into the rootfs... so the apt-get update which is run inside the rootfs should always fail.

I'll add a commit to this branch to ensure that the installer's resolver configuration is copied into the rootfs for use before the first boot.

[kpfleming]

I thought I could test this using QEMU, but it turns out that's a lot harder than it should be... I don't have any more spare Pis, so I just ordered one (my Pi1 could use an upgrade to a Pi3 anyway). Hopefully next weekend I'll be able to pick this up again.

[kpfleming]

Bug confirmed and resolved. I didn't notice this before because any failure of apt-get update still reports 'OK' in the log, but I should still have noticed the two package installation failures after that. Oops!

[diederikdehaas]

Awesome, thanks @kpfleming 👍

[diederikdehaas]

I value all contributions, but the addition of IPv6 support in a net(work) installer is probably one of my favorites \o/

(and sorry it took so long)

[diederikdehaas]

The reason this PR got assigned to me was because I was triggered by something you said somewhere else and that was that for a certain feature some big changes were needed in the installer.

Found it!
In #347 (comment) you said:

I've submitted PR #425 to add substantial IPv6 support (although not complete, as it does not support DHCPv6, since that would require substantial additions to the installer).

If you (still) know what those additions are, feel free to share them 👍

[kpfleming]

Ahh... that. Supporting DHCPv6 would require using a full-blown DHCP client in the installer (probably dhclient, the same one that is used in the installed system) instead of udhcpc. While I haven't tried it, this would likely increase the size of the installer image quite noticeably, so as a result it's probably best to wait until the community expresses a real need for DHCPv6 support.

[mutability]

Counterintutively, this PR actually broke IPv6 on an install that was previously working fine (install over IPv4, afterwards IPv6 works fine).

The cause is that the PR disables IPv6 entirely on the target system if IPv6 is not enabled at install time.

[kpfleming]

Yes, that was intentional, since we've had reports (both related to this installer, and in general) that people have broken IPv6 on their networks and don't know it. When the installer did not care about IPv6, the installation would work fine, but then the resulting system would exhibit odd breakage which was difficult to diagnose because the user did not even know that there was IPv6 available on their network.

I wouldn't have any objection to changing the IPv6 default in the installer to 'auto', allowing it to be used if it's available, and then this would fall automatically into the installed system as well. If a user then reports that the installation fails and it's determined that broken IPv6 networking on their LAN is the cause, they can set it to 'disable' in their installer-config.txt file and use IPv4 only. It just seemed safer to continue ignoring IPv6 completely since that was the installer's previous behavior (although not an intentional choice).

[mutability]

Well, I just undo the damage in post-install.txt now, so it's not a big deal, but it's a regression from how it behaved before in my environment.

[mutability]

I have a slightly unusual environment in that I end up with v4 external connectivity and use v6 internally for a VPN. So the external connectivity doesn't matter - but globally disabling v6 breaks the VPN because it prevents setting up the tunnel device.

[kpfleming]

For what it's worth, the README in the v1.1.x branch does make it clear that IPv6 support will be disabled by default, although it might be worth extending that statement to indicate that it will be disabled both during the installation and in the installed system.