Add InfiniBand (IPoIB) support (LP: #1848471)

[slyon]

Description

InfiniBand (IPoIB) devices have a long-form HWADDR, as opposed to the 'normal' 6-byte ethernet HWADDR. When trying to use this HWADDR in a netplan config, it errors out expecting a 6-byte HWADDR. Example:
a0:00:02:20:fe:80:00:00:00:00:00:00:ec:0d:9a:03:00:15:e2:c1

• We want to adapt netplan's MAC address validation, to accept any 20-byte IPoIB HWADDR.
• We also want to introduce a new YAML stanza infiniband-mode: datagram|connected to enable IB mode configuration.

Example:

network:
  version: 2
  renderer: networkd
  ethernets:
    ib0:
      match:
        macaddress: "a0:00:02:20:fe:80:00:00:00:00:00:00:ec:0d:9a:03:00:15:e2:c1"
      dhcp4: true
      infiniband-mode: "connected"

Checklist

• Runs make check successfully.
• Retains 100% code coverage (make check-coverage).
• New/changed keys in YAML format are documented.
• (Optional) Adds example YAML for new feature.
• (Optional) Closes an open bug in Launchpad. LP#1848471)

[schopin-pro]

I only have one holdout about this, which is the use of a generic string in the data model. However, since we already do it elsewhere you can disregard the comment entirely, I won't hold it against you ;-)

[sparkiegeek]

I'm not in a position to review this - I don't know C and don't have access to Infiniband

[slyon]

I'm not in a position to review this - I don't know C and don't have access to Infiniband

Alright @sparkiegeek, I just wanted to keep you in the loop as this feature request originated from a MAAS context a while ago.

We already got @schopin-pro's review (thanks for that!) and I adopted the remarks accordingly. Let's try to get the new infiniband-mode stanza approved schema wise, otherwise this PR should be good to be merged!

[schopin-pro]

LGTM with the new changes :)

[sparkiegeek]

As noted - am an interested end-customer but not qualified to review :)

[slyon]

As noted - am an interested end-customer but not qualified to review :)

ACK. We're waiting for schema review of the new infiniband-mode: datagram|connected setting, which is already on Steve's list. Afterwards we are good to merge this.

[vorlonofportland]

Proposed schema additions look ok, with one question inline (and a minor editorial fix requested).

[vorlonofportland]

Is this the preferred format for representing infiniband mac addresses? It doesn't seem very readable. It does seem to be the format that NM uses, but are there alternatives in common use that we should consider?

[slyon]

Yes, I couldn't find any other common format for infiniband mac addresses. In general, "XX-XX-XX-XX-XX-XX" or "XXXXXXXXXXXX" are other common mac address formats, but those aren't any more readable if extended to 20 bytes.

It is also the format that has been proposed (and requested) at other places:

• https://code.launchpad.net/~darren-birkett/cloud-init/+git/cloud-init/+merge/373759
• https://bugs.launchpad.net/netplan/+bug/1848474

[slyon]

Proposed schema additions look ok, with one question inline (and a minor editorial fix requested).

Thanks, I've added the "schema ok" label. Fixed your documentation remark and commented on the MAC address format. This should be good for merging now, if no further remarks are added.

[sil2100]

LGTM!

[slyon]

Merging. I think all comments have been addressed. If we'd want to support an additional format in the future, we can still add this as a follow-up.

[krono]

Hi,

sorry to comment on that late.
I am happy for infiniband-support, but there is a problem with the 20-Byte version of the "hwaddress", because only the last 8 byte are actually "stable".
This is a Red Hat resource, but it still stands: (https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/7/html/networking_guide/sec-configuring_ipoib#sec-Understanding_IPoIB_hardware_addresses):

IPoIB devices have a 20 byte hardware addresses. […]

The first 4 bytes of the IPoIB hardware address are flags and the queue pair number. The next 8 bytes are the subnet prefix. When the IPoIB device is first created, it will have the default subnet prefix of 0xfe:80:00:00:00:00:00:00. The device will use the default subnet prefix (0xfe80000000000000) until it makes contact with the subnet manager, at which point it will reset the subnet prefix to match what the subnet manager has configured it to be. The final 8 bytes are the GUID address of the InfiniBand port that the IPoIB device is attached to. Because both the first 4 bytes and the next 8 bytes can change from time to time, they are not used or matched against when specifying the hardware address for an IPoIB interface.

So, netplan cannot know whether the device has already joined a subnet manerg or unbeknownst changed the subnet manager (which is unrelated to the VLAN-like P_Key-functionality), or what the queue pair is.

So, specifying an EUI-64 for the GUID could also be considered.