datapath: Add a flag to set VXLAN and Geneve ports

[errordeveloper]

This change moves port defaulting logic from the kernel into the
agent and exposes a flag that lets user set a custom port.

Defaults ports remain unchanged.

Towards: #15956

[errordeveloper]

/test

[errordeveloper]

/test

[joestringer]

Upgrade question, bear with me if anyone already asked and I just missed the answer:

Commit message:

This change makes it possible for user to set a custom port
for VXLAN or Geneve. In order to anable that, the defaulting
logic was introduced in the agent, so kernel defaults are no
longer relied upon.

Default ports remain unchanged.

If we no longer rely on the kernel's default, how do we know that this won't create an upgrade regression impact? Is it because all kernels we support have the same default port? Otherwise, switching from defaulting to the kernel default to an internal value could plausibly change the port. So I think there's two questions, (1), could this change the effective port? (eg on older kernels), and (2) If yes, then what kind of network disruption would we expect during upgrade?

[errordeveloper]

Upgrade question, bear with me if anyone already asked and I just missed the answer:

I the topic was touched on briefly, maybe it was on slack. I have a TODO in #15956 (comment) to do document this properly.

Commit message:

This change makes it possible for user to set a custom port
for VXLAN or Geneve. In order to anable that, the defaulting
logic was introduced in the agent, so kernel defaults are no
longer relied upon.
Default ports remain unchanged.

If we no longer rely on the kernel's default, how do we know that this won't create an upgrade regression impact?

There will be impact only if user decide changes the port on upgrade.

Is it because all kernels we support have the same default port?

From my understanding there is no global settable default. The default is per-interface, overriding it requires a very specific ip link invocation with dstport specified. Until now init.sh didn't pass dsport at all, so the default was used. The default port is also assumed by system requirements docs.

And this PR doesn't actually change the default, it only sets what's the current default port is explicitly. It's been assumed until now that the kernel's default won't change, but now if it does actually change, Cilium will keep on using the same port unless user changes or maintainers agree to change it.

Otherwise, switching from defaulting to the kernel default to an internal value could plausibly change the port.

No, as per above, default is per-interface Cilium own the interface anyway.

So I think there's two questions, (1), could this change the effective port? (eg on older kernels), and (2) If yes, then what kind of network disruption would we expect during upgrade?

1. not expected to happen
2. yes, and that is to be documented, but only in connection to user intentionally changing the port in an existing installation

concern raised is deemed to be satisfied

[errordeveloper]

Travis seems to run into this: 'The job exceeded the maximum log length, and has been terminated.' 🤔

I gets into an infinite loop here:

ake[2]: Entering directory '/home/travis/gopath/src/github.com/cilium/cilium/bpf/mock'
# Pre-pull FROM docker images due to Buildkit sometimes failing to pull them.
grep "^FROM " Dockerfile | cut -d ' ' -f2 | xargs -n1 docker pull
20.04: Pulling from library/ubuntu
Status: Downloaded newer image for ubuntu:20.04
docker.io/library/ubuntu:20.04
tar c Dockerfile \
 | docker build --tag cilium/bpf-mock -
 => [internal] load remote build context                                   0.0s
 => copy /context /                                                        0.1s
 => [internal] load metadata for docker.io/library/ubuntu:20.04            0.0s
 => [internal] load remote build context                                   0.0s
 => copy /context /                                                        0.1s
 => [internal] load metadata for docker.io/library/ubuntu:20.04            0.0s
 => [ 1/10] FROM docker.io/library/ubuntu:20.04                            0.0s
 => => resolve docker.io/library/ubuntu:20.04                              0.0s
 => [ 2/10] RUN apt-get update                                             0.1s
 => [internal] load remote build context                                   0.0s
 => copy /context /                                                        0.1s
 => [internal] load metadata for docker.io/library/ubuntu:20.04            0.0s
 => [ 1/10] FROM docker.io/library/ubuntu:20.04                            0.0s
 => => resolve docker.io/library/ubuntu:20.04                              0.0s
 => [ 2/10] RUN apt-get update                                             0.3s
 => [internal] load remote build context                                   0.0s
 => copy /context /                                                        0.1s
 => [internal] load metadata for docker.io/library/ubuntu:20.04            0.0s
 => [ 1/10] FROM docker.io/library/ubuntu:20.04                            0.0s
 => => resolve docker.io/library/ubuntu:20.04                              0.0s
 => [ 2/10] RUN apt-get update                                             0.4s
 => [internal] load remote build context                                   0.0s
 => copy /context /                                                        0.1s
 => [internal] load metadata for docker.io/library/ubuntu:20.04            0.0s
 => [ 1/10] FROM docker.io/library/ubuntu:20.04                            0.0s
 => => resolve docker.io/library/ubuntu:20.04                              0.0s
 => [ 2/10] RUN apt-get update                                             0.6s
...

[errordeveloper]

/test

[llhhbc]

good