problems with DHCP opt 61 on Raspi3

[mam59]

Before I begin: english is not my natural language, it maybe a bit hard for me the rather complex problem I have run into recently.

The problem could briefly be titled "NOOBs on a raspberry 3 cannot get a dhcp lease from a hardened microsoft dhcp server" (other vendors may be affected too, but are not tested)
Let us take a look into the request packet the NOOBs (1.9) sends out at boot time:

We see that raspy with the ethernet mac address of ... AC-39-B0 sends out a request with an client id of (combined) "ebac39b0br27ebf96ce5", the first part is taken from the LAN card (Raspberry vendor stripped), the last part is the complete MAC from the WLAN card onboard.
This is a very nice and valid clientid for IPV6!, but it kills V4 if the server is configured to run in a mode with either reservations defined or with a mac filter.
If the microsoft server finds an option 61 in a packet it takes this id in favor to the mac address the packet originated from!
And, in the V4 part of the Server you have no chance to define a filter with such an id, only plain mac addresses are available!
So the admin has no chance to add this raspi to the list of allowed clients.
To let him in, he has to disable any filter mode of the server, then option 61 is ignored and the client is given a lease.
But even if he lets him in this way and then wants to freeze the given lease into a permanent reservation, it fails because the invalid (for V4) client id.

It seems to me like somebody has written the docs too deeply, NOOBs could safely drop this option 61 and everything will be fine again..

Sometimes doing less is giving more...

[procount]

My guess is that changing the duid option to clientid in /etc/dhcpcd.conf (the same as in Raspbian) will fix it, but I've not tested it yet. Any comments @maxnet, @XECDesign, @lurch?

Does NOOBS actually define a dhcpcd.conf file, or does it just use a default file? I couldn't find it. So where best to put one - my guess is in package/recovery and add a line in recovery.mk such as:
$(INSTALL) -D -m 0664 package/recovery/dhcpcd.conf $(TARGET_DIR)/etc/dhcpcd.conf ???

[mam59]

Nice Idea but I'm afraid, it wont help much. DUIDs are also defined for V6 only.
The clientid for V4 should simply be Hardware-Type + Mac Address.
I still not have found the source for the DHClient program to look up, what it does internally to construct the client id. Isn't it within the NOOBs master tree somewhere?

Funny Joke: if NOOBs would send out this kind of "wrong" packet here in our LAN, just on a V6 socket, it would succeed and find a nice Server that would reply to it. :-)

[maxnet]

We see that raspy with the ethernet mac address of ... AC-39-B0 sends out a request with an client id
of (combined) "ebac39b0br27ebf96ce5", the first part is taken from the LAN card (Raspberry vendor
stripped), the last part is the complete MAC from the WLAN card onboard.

Yes, that is the correct behavior per RFC 3315.
Should pick one MAC as DUID to uniquely identify the client, and it should not vary by interface used.

   The choice of network interface can be completely arbitrary, as long
   as that interface provides a globally unique link-layer address for
   the link type, and the same DUID-LLT SHOULD be used in configuring
   all network interfaces connected to the device, regardless of which
   interface's link-layer address was used to generate the DUID-LLT.

We could tell dhcpcd to not send the duid to work around the short comings of your DHCP server, but keep in mind that doing so would only make sense if the OS'es we install like Raspbian make the same change...

[mam59]

Yes, that is the correct behavior per RFC 3315.
Should pick one MAC as DUID to uniquely identify the client, and it should not vary by interface used.
Cough, have you READ that RFC?
It describes DHCPV6! not valid for V4!
Read the title again: "Dynamic Host Configuration Protocol for IPv6 (DHCPv6)"
And now please reconsider...

[maxnet]

It describes DHCPV6! not valid for V4!

https://tools.ietf.org/html/rfc4361#section-6.1

A device that is normally configured by both a DHCPv4 and DHCPv6 client 
SHOULD automatically use the same DUID for DHCPv4 and DHCPv6 without 
any operator intervention.

DHCPv4 clients that support more than one network interface SHOULD
use the same DUID on every interface.  DHCPv4 clients that support
more than one network interface SHOULD use a different IAID on each
interface.

[mam59]

you simply ignore the fact that NOOBs is not V6 at all!
It is far away of beeing "normally configured by both..."
so that RFC does not apply here too!

But ok, we dont need to fight about RFCs here, I just tell you that your current configuration breaks existing implementations. Better consider to use a more defensive and safe approach.

[maxnet]

Keep in mind that we are just using the default dhcpcd configuration.
That does support IPv6, and we are not in a position to complain to the author that the default should be changed when it is actually the right thing according to the standards...

Yes, we could deviate from the default configuration, and write our own.
Then you indeed would be able to use NOOBS lite with your specific configuration.
However after you are done installing an OS with NOOBS, it would reboot to the OS you installed, e.g. Raspbian.
And then your network configuration will be broken again, as that uses the default dhcpcd configuration as well...

So IMHO deviating from the default in NOOBS would only makes much sense, if you can convince the maintainers of Raspbian to agree to deviate from the default as well.
And check if the other distributions do work correctly (think they use a different dhcp client then dhcpcd, so may already work).

[mam59]

Sorry, but I think, we dont go together very good :-(
I can tell you that Raspian (later on) and other OSses (mainly using Openelec) are doing fine with DHCP over here, BECAUSE THEY DO DUAL STACK!
Your kernel does not include V6, so the default configuration for DHCP is wrong in your case.

I just got some new SD cards, I will now format one and install Raspbian and do the same captures as before...

[maxnet]

I can tell you that Raspian (later on) and other OSses (mainly using Openelec) are doing fine with DHCP over here, BECAUSE THEY DO DUAL STACK!

Believe OpenELEC uses connman as DHCP client instead of dhcpcd, so may indeed behave differently.

Would personally expect Raspbian to send the same DHCPv4 packets to request an IPv4 though.
In addition to concurrent DHCPv6 packets that request an IPv6 address only.
But feel free to post packet dumps showing otherwise. I do not have your DHCP server software, so cannot test.

[mam59]

Ok, you dont believe it, so here is a just taken snapshot from a just freshly installed raspbian:

Still any questions? IT DOES NOT USE DUIDs!!! It just uses plain V4 Hardware Address as it is the only allowed client ID for plain V4.

(the same is true for Openelec, can I spare me the installation now and you believe me?)

[procount]

I don't have the latest version of Raspbian to hand. In the version that is on my SD card, uname -a reports:
Linux raspberrypi 4.1.17-v7+ #838 SMP Tue Feb 9 13:15:09 GMT 2016 armv7l GNU/Linux
Nevertheless, etc/dhcpcd.conf uses clientid and I confirm it send outs the same mac address in Option 61 as the Client Mac address, just as mam59 describes, which is different to NOOBS behaviour that sends out the wifi mac address instead.

NOOBS has never caused me a problem with its default settings, so I guess it is purely an issue when using the hardened Microsoft dhcp servers.

I have created a modified version of noobs to test out my theory above, so I'll see if it makes a difference.

[procount]

So this is my output from my modified NOOBS:

Any good?

[mam59]

looks good , length 7, ethernet type and just the normal LAN Mac. this is how it should be
what did you change?
btw, "clientid" is not the problem by itself, the option is valid on V4 and V6. It is just what is transmitted INSIDE clientid. The "old style" just sends the MAC, the "new style" a kind of UUID which should be unique but constant even between reboots. The "new style" is just not understood by older servers.

Normally DHCP servers ignore this field and just use the MAC from the header of the packet. Even the Microsoft server works until you turn on the filter feature.

[procount]

Well, I don't have a clue on the relevant RFCs, what Debian/Raspbian/OPENElec does or should do or the intracacies of DHCP boot protocols on IPv4 vs IPv6 - I'll leave that to others to decide about. :)
I just used the dhcpcd.conf file from Raspbian in noobs
AFAICT, the only option that looks remotely like affecting this behaviour is the clientid/duid option.

[maxnet]

Nevertheless, etc/dhcpcd.conf uses clientid

Ah, seems Raspbian indeed deviates from the dhcpcd default.

https://github.com/raspberrypi/dhcpcd-6.7.1/commit/c71d973aa72e29e90e479b1409ed0c3e22b84ba5

Was not aware of that.
In that case I no longer have objections against NOOBS deviating as well.