docs: Update max MTU value for Nodeport XDP on AWS

[qmonnet]

The documentation for setting up Nodeport XDP acceleration on AWS mentions that the MTU for the ena interface must be lower down so that XDP can work. It is indeed necessary; but the value which is provided as the maximal possible MTU is outdated, and not working.

After installing the latest kernel through the RPM package kernel-ng (as prescribed in the documentation), the EKS nodes currently end up with Linux 5.10:

$ uname -r
5.10.106-102.504.amzn2.x86_64

If we keep on following the docs and lower the MTU to 3818, the Cilium pods fail to get ready, and tell in their logs that the XDP program cannot be set due to the MTU. This is also confirmed from the dmesg of the nodes:

[ 3617.059219] ena 0000:00:05.0 eth0: Failed to set xdp program,
    the current MTU (3818) is larger than the maximum allowed MTU (3498) while xdp is on

The value 3818 comes from the legacy definition of ENA_XDP_MAX_MTU, in drivers/net/ethernet/amazon/ena/ena_netdev.h, which used to be defined as such:

#define ENA_XDP_MAX_MTU (ENA_PAGE_SIZE - ETH_HLEN - ETH_FCS_LEN - \
                         VLAN_HLEN - XDP_PACKET_HEADROOM)

Where ETH_LEN is 14, ETH_FCS_LEN and VLAN_HLEN are both 4, and XDP_PACKET_HEADROOM is 256.

But after Linux commit 08fc1cfd2d25 ("ena: Add XDP frame size to amazon NIC driver"), from Linux 5.8, the definition changed to:

#define ENA_XDP_MAX_MTU (ENA_PAGE_SIZE - ETH_HLEN - ETH_FCS_LEN -	\
                         VLAN_HLEN - XDP_PACKET_HEADROOM -		\
                         SKB_DATA_ALIGN(sizeof(struct skb_shared_info)))

Where SKB_DATA_ALIGN(sizeof(struct skb_shared_info)) is currently 320 (multiple of 64 bytes, for alignment on the L1 cache lines).

As a result, the maximum value for the MTU for kernels 5.8+ is 3498 bytes. This is indeed the maximum value that I could use when setting up XDP on an EKS cluster. Let's update the documentation accordingly.

[qmonnet]

Checkpatch warns about an Unknown commit id '08fc1cfd2d25', expected given that this is a reference to a commit in Linux (not in Cilium).