Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cilium: various xdp related follow-ups #10910

Merged
merged 5 commits into from Apr 10, 2020
Merged

cilium: various xdp related follow-ups #10910

merged 5 commits into from Apr 10, 2020

Conversation

borkmann
Copy link
Member

@borkmann borkmann commented Apr 9, 2020

  • cilium status output
  • CI tests
  • Hide generic XDP for testing only

@borkmann
cilium: remap xdp 'generic' to hidden 'testing-only' option
71315dd 
Remap the XDP generic option from "generic" into "testing-only". The
reason is that we don't want to encourage users to run with generic XDP
at this point but only with the fast native one instead. Generic XDP has
several downside:

 - Still operates on skbs
 - Linearizes & unclones every skb
 - Bypasses GRO

These are severe limitations which the native XDP mode does not have and
could potentially make its operation slower than the tc ingress one. The
generic XDP is however very useful for testing purpose and we would like
to use it in our CI. Therefore, change it into an undocumented "testing-only"
option value that we can use internally. For the --prefilter-mode, the
ship has sailed on "generic" therefore we keep it intact and remap into
"testing-only" internally. Simplify the documentation as well and remove
the prior warning that generic XDP should not be used in a production
environment.

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
@borkmann
cilium: add nodeport acceleration to cilium status report
65a8536 
Dump the XDP acceleration mode in the cilium status output in order
to see if BPF was attached and in which XDP mode. The hidden generic
option value is dumped as "GENERIC" here since in XDP context it's
called generic XDP.

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
@borkmann borkmann added pending-review sig/datapath Impacts bpf/ or low-level forwarding details, including map management and monitor messages. labels Apr 9, 2020
@borkmann borkmann requested review from brb and a team April 9, 2020 10:51
@borkmann borkmann requested review from a team as code owners April 9, 2020 10:51
@maintainer-s-little-helper
Copy link

Please set the appropriate release note label.

3 similar comments
@maintainer-s-little-helper
Copy link

Please set the appropriate release note label.

@maintainer-s-little-helper
Copy link

Please set the appropriate release note label.

@maintainer-s-little-helper
Copy link

Please set the appropriate release note label.

@maintainer-s-little-helper maintainer-s-little-helper bot added this to In progress in 1.8.0 Apr 9, 2020
@borkmann borkmann added the release-note/misc This PR makes changes that have no direct user impact. label Apr 9, 2020
@borkmann
Copy link
Member Author

borkmann commented Apr 9, 2020

test-me-please

@borkmann
Copy link
Member Author

borkmann commented Apr 9, 2020

test-me-please

@borkmann
Copy link
Member Author

borkmann commented Apr 9, 2020

unrelated vagrant issue:

 97 9861M   97 9609M    0     0   100M      0  0:01:38  0:01:35  0:00:03  112M
 98 9861M   98 9721M    0     0   100M      0  0:01:37  0:01:36  0:00:01  112M
 99 9861M   99 9833M    0     0   100M      0  0:01:37  0:01:37 --:--:--  112M
100 9861M  100 9861M    0     0   100M      0  0:01:37  0:01:37 --:--:--  112M
11:27:23  The machine index which stores all required information about
11:27:23  running Vagrant environments has become corrupt. This is usually
11:27:23  caused by external tampering of the Vagrant data folder.
11:27:23  
11:27:23  Vagrant cannot manage any Vagrant environments if the index is
11:27:23  corrupt. Please attempt to manually correct it. If you are unable
11:27:23  to manually correct it, then remove the data file at the path below.
11:27:23  This will leave all existing Vagrant environments "orphaned" and
11:27:23  they'll have to be destroyed manually.
11:27:23  
11:27:23  Path: /root/.vagrant.d/data/machine-index/index
11:27:24  box locked or unavailable, adding box from vagrant cloud
11:27:25  The machine index which stores all required information about
11:27:25  running Vagrant environments has become corrupt. This is usually
11:27:25  caused by external tampering of the Vagrant data folder.
11:27:25  
11:27:25  Vagrant cannot manage any Vagrant environments if the index is
11:27:25  corrupt. Please attempt to manually correct it. If you are unable
11:27:25  to manually correct it, then remove the data file at the path below.
11:27:25  This will leave all existing Vagrant environments "orphaned" and
11:27:25  they'll have to be destroyed manually.
11:27:25  
11:27:25  Path: /root/.vagrant.d/data/machine-index/index
Post stage

@coveralls
Copy link

coveralls commented Apr 9, 2020 •
edited

Coverage Status

Coverage increased (+0.0004%) to 46.636% when pulling 44cd7e9 on pr/xdp-follow-ups into 30e8837 on master.

@borkmann
Copy link
Member Author

borkmann commented Apr 9, 2020

test-me-please

@borkmann
Copy link
Member Author

borkmann commented Apr 9, 2020

provisioning issue again:

[Pipeline] sh
12:37:23  + /usr/local/bin/add_vagrant_box /home/jenkins/workspace/Cilium-PR-Ginkgo-Tests-Validated/src/github.com/cilium/cilium/vagrant_box_defaults.rb
12:37:23  The machine index which stores all required information about
12:37:23  running Vagrant environments has become corrupt. This is usually
12:37:23  caused by external tampering of the Vagrant data folder.
12:37:23  [...]

@borkmann
Copy link
Member Author

borkmann commented Apr 9, 2020

test-me-please

@borkmann
Copy link
Member Author

borkmann commented Apr 9, 2020

Hit flake #10821

@borkmann
Copy link
Member Author

borkmann commented Apr 9, 2020

test-me-please

@borkmann
Copy link
Member Author

borkmann commented Apr 9, 2020

Hit flake #10821

@borkmann
Copy link
Member Author

borkmann commented Apr 9, 2020

test-focus K8sService.*

@borkmann
Copy link
Member Author

borkmann commented Apr 9, 2020

Cilium-Tests-With-Kernel provision issue

20:45:04  Checking for older boxes...
20:45:04  No old versions of boxes to remove...
20:45:04  VBoxManage: error: Machine '9c85c1b7-53ab-4012-837a-749c5acf0af8' is not currently running
20:45:04  0%...10%...20%...30%...40%...
20:45:04  Progress state: VBOX_E_OBJECT_IN_USE
20:45:04  VBoxManage: error: Machine delete failed
20:45:04  VBoxManage: error: Cannot close medium '/root/VirtualBox VMs/ubuntu-18.04.4-amd64-11_1586464889876_63179/ubuntu-18.04.4-amd64-11-disk001.vmdk' because it has 1 child media
20:45:04  VBoxManage: error: Details: code VBOX_E_OBJECT_IN_USE (0x80bb000c), component MediumWrap, interface IMedium
20:45:04  VBoxManage: error: Context: "RTEXITCODE handleUnregisterVM(HandlerArg*)" at line 162 of file VBoxManageMisc.cpp
20:45:04  VBoxManage: error: Machine 'ab8f39a8-4e2d-4dc1-bb3e-da306b749f17' is not currently running
20:45:06  0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
20:45:06  Clean docker images from registry

@borkmann
Copy link
Member Author

borkmann commented Apr 9, 2020

test-with-kernel

@borkmann
Copy link
Member Author

borkmann commented Apr 9, 2020

test-focus K8sService.*

@borkmann
Copy link
Member Author

borkmann commented Apr 9, 2020

vagrant issue again in CI...

@borkmann
Copy link
Member Author

borkmann commented Apr 9, 2020

test-focus K8sService.*

@borkmann
Copy link
Member Author

borkmann commented Apr 9, 2020

test-focus K8sService.*

@borkmann
Copy link
Member Author

borkmann commented Apr 9, 2020

test-me-please

@borkmann
Copy link
Member Author

vbox error in CI:

00:31:39  Stderr: VBoxManage: error: Failed to create the VirtualBox object!
00:31:39  VBoxManage: error: Document is empty.
00:31:39  VBoxManage: error: Location: '/root/.config/VirtualBox/VirtualBox.xml', line 1 (0), column 1.
00:31:39  VBoxManage: error: /home/vbox/vbox-6.0.18/src/VBox/Main/src-server/VirtualBoxImpl.cpp[624] (nsresult VirtualBox::init())
00:31:39  VBoxManage: error: Details: code NS_ERROR_FAILURE (0x80004005), component VirtualBoxWrap, interface IVirtualBox
00:31:39  script returned exit code 143

@borkmann
Copy link
Member Author

test-me-please

This was referenced Apr 10, 2020
Closed
Closed
@joestringer
Copy link
Member

Hit #10929 and #10930, will restart ginkgo.

@joestringer
Copy link
Member

restart-ginkgo

@borkmann
bpf: clean up XDP from prior devices when cilium configuration changes
7d1a16f 
Avoid having to leave around stale XDP programs when the config
changes. Therefore do the same as we do in tc which is to clean
up prior state.

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
@borkmann
testing: enable generic XDP tests for NodePort in CI
79aaac6 
Add tests for generic XDP under DSR/SNAT/Hybrid modes to our CI and also
test base functionality to make sure all the rest is working as expected
when XDP is attached.

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
@borkmann
testing: enable external TC tests for NodePort in CI
44cd7e9 
Add also tests for (external) TC under DSR/SNAT/Hybrid modes to our
CI so they can be performed from the third host. So far we don't seem
to test all of them (only for the metal LB case).

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
@borkmann borkmann requested a review from a team April 10, 2020 12:57
@borkmann
Copy link
Member Author

test-me-please

@borkmann
Copy link
Member Author

test-focus K8sService.*

@borkmann borkmann merged commit 0626fcd into master Apr 10, 2020
1.8.0 automation moved this from In progress to Merged Apr 10, 2020
@borkmann borkmann deleted the pr/xdp-follow-ups branch April 10, 2020 15:19
@christarazi christarazi mentioned this pull request Oct 16, 2020
Merged
@christarazi
Copy link
Member

Backported only 7d1a16f to v1.7 (PR: #13595) as #13532 depended on this commit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@brb brb brb approved these changes

Assignees
No one assigned
Labels
release-note/misc This PR makes changes that have no direct user impact. sig/datapath Impacts bpf/ or low-level forwarding details, including map management and monitor messages.
Projects
No open projects
1.8.0
  
Merged
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@borkmann @coveralls @joestringer @christarazi @brb