Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: NodePort XDP on Azure #12150

Merged
merged 1 commit into from Jun 17, 2020
Merged

Conversation

gandro
Copy link
Member

@gandro gandro commented Jun 17, 2020

This adds a basic section with details on how to run NodePort XDP on
Azure. There are two things needed to get native XDP on a Azure VM:

  • The VM needs to be have "Accelerated Networking", which attaches
    an native XDP capable mlx4/mlx5 NIC.
  • Because the above VF card acts as a slave to the Hyper-V NIC on
    Azure, the running kernel must in addition also have native XDP
    for the hv_netvsc driver. This is available in Linux >= 5.6 [1].

Unfortunately, this means that running NodePort XDP on e.g. AKS is not
straightforward, as none of the provided VM images right now offer
native XDP for hv_netsvc. There is an open issue to add backport this
to Ubuntu, which is likely to land in the next few weeks [2].

Beacuse we cannot use the stock images for now, I have therefore
validated the setup on a self-managed Kubernetes cluster set up
with kubeadm on Flatcar Container Linux Edge. This one of few
Azure supporting distributions with kernel 5.6+. It runs NodePort XDP
and passes the connectivity-check.yaml with the helm install
command provided in the section. I used Azure IPAM, as I was not able
to convince the underlying vnet forward the pod-to-pod traffic
otherwise.

Because my setup is rather ad-hoc and therefore not recommended for
production use, we abstain from a step-by-step guide until native XDP
for hv_netsvc is available in official Azure images.

[1] https://lore.kernel.org/netdev/1579816355-6933-2-git-send-email-haiyangz@microsoft.com/
[2] https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1877654

@gandro gandro added area/documentation Impacts the documentation, including textual changes, sphinx, or other doc generation code. needs-backport/1.8 labels Jun 17, 2020
@gandro gandro requested a review from borkmann June 17, 2020 19:25
@gandro gandro requested a review from a team as a code owner June 17, 2020 19:25
@maintainer-s-little-helper
Copy link

Please set the appropriate release note label.

2 similar comments
@maintainer-s-little-helper
Copy link

Please set the appropriate release note label.

@maintainer-s-little-helper
Copy link

Please set the appropriate release note label.

@maintainer-s-little-helper maintainer-s-little-helper bot added this to In progress in 1.8.0 Jun 17, 2020
@maintainer-s-little-helper maintainer-s-little-helper bot added this to Needs backport from master in 1.8.0 Jun 17, 2020
@gandro gandro added the release-note/misc This PR makes changes that have no direct user impact. label Jun 17, 2020
@b3a-dev b3a-dev mentioned this pull request Jun 17, 2020
20 tasks
This adds a basic section with details on how to run NodePort XDP on
Azure. There are two things needed to get native XDP on a Azure VM:

  - The VM needs to be have "Accelerated Networking", which attaches
    an native XDP capable `mlx4`/`mlx5` NIC.
  - Because the above VF card acts as a slave to the Hyper-V NIC on
    Azure, the running kernel must in addition also have native XDP
    for the `hv_netvsc` driver. This is available in Linux >= 5.6 [1].

Unfortunately, this means that running NodePort XDP on e.g. AKS is not
straightforward, as none of the provided VM images right now offer
native XDP for `hv_netsvc`. There is an open issue to add backport this
to Ubuntu, which is likely to land in the next few weeks [2].

Beacuse we cannot use the stock images for now, I have therefore
validated the setup on a self-managed Kubernetes cluster set up
with `kubeadm` on Flatcar Container Linux Edge. This one of few
Azure supporting distributions with kernel 5.6+. It runs NodePort XDP
and passes the `connectivity-check.yaml` with the `helm install`
command provided in the section. I used Azure IPAM, as I was not able
to convince the underlying vnet forward the pod-to-pod traffic
otherwise.

Because my setup is rather ad-hoc and therefore not recommended for
production use, we abstain from a step-by-step guide until native XDP
for `hv_netsvc` is available in official Azure images.

[1] https://lore.kernel.org/netdev/1579816355-6933-2-git-send-email-haiyangz@microsoft.com/
[2] https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1877654

Signed-off-by: Sebastian Wicki <sebastian@isovalent.com>
@gandro gandro force-pushed the pr/gandro/docs-nodeport-xdp-on-azure branch from ecc18a3 to 42263ba Compare June 17, 2020 19:34
@borkmann borkmann merged commit 97e3fc4 into master Jun 17, 2020
1.8.0 automation moved this from In progress to Merged Jun 17, 2020
@borkmann borkmann deleted the pr/gandro/docs-nodeport-xdp-on-azure branch June 17, 2020 19:37
@coveralls
Copy link

Coverage Status

Coverage decreased (-0.09%) to 37.025% when pulling 42263ba on pr/gandro/docs-nodeport-xdp-on-azure into 134fdb5 on master.

@maintainer-s-little-helper maintainer-s-little-helper bot moved this from Needs backport from master to Backport pending to v1.8 in 1.8.0 Jun 18, 2020
@maintainer-s-little-helper maintainer-s-little-helper bot moved this from Needs backport from master to Backport pending to v1.8 in 1.8.0 Jun 18, 2020
@maintainer-s-little-helper maintainer-s-little-helper bot moved this from Backport pending to v1.8 to Backport done to v1.8 in 1.8.0 Jun 18, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/documentation Impacts the documentation, including textual changes, sphinx, or other doc generation code. release-note/misc This PR makes changes that have no direct user impact.
Projects
No open projects
1.8.0
  
Merged
1.8.0
Backport done to v1.8
Development

Successfully merging this pull request may close these issues.

None yet

4 participants