bpf: fix in-cluster connectivity for externalTrafficPolicy=Local #12311

borkmann · 2020-06-26T22:13:09Z

See commit messages.

coveralls · 2020-06-26T23:10:12Z

Coverage increased (+0.007%) to 36.917% when pulling 775b097 on pr/traffic-policy-local into 65e68d0 on master.

gandro

Overall looks good so far! One issue with regards to HealthCheckNodePort

pkg/service/service.go

borkmann · 2020-06-30T12:25:00Z

test-me-please

borkmann · 2020-06-30T12:26:36Z

test-focus K8sService*

borkmann · 2020-07-01T17:06:00Z

(don't merge yet, still debugging on sth)

borkmann · 2020-07-01T20:54:30Z

test-me-please

borkmann · 2020-07-01T20:54:40Z

test-focus K8sService*

borkmann · 2020-07-01T21:24:28Z

(ready to merge once CI is green)

borkmann · 2020-07-01T23:46:09Z

Runtime-4.9 looks unrelated:

	 START: proxy_test.go:306: DNSProxyTestSuite.TestRejectNonMatchingRefusedResponse
	 proxy_test.go:329:
	     c.Assert(err, IsNil, Commentf("DNS request from test client failed when it should succeed"))
	 ... value *net.OpError = &net.OpError{Op:"dial", Net:"tcp", Source:net.Addr(nil), Addr:(*net.TCPAddr)(0xc00041a1e0), Err:(*poll.TimeoutError)(0x2f8d460)} ("dial tcp [::]:36405: i/o timeout")
	 ... DNS request from test client failed when it should succeed
	 
	 START: proxy_test.go:219: DNSProxyTestSuite.TearDownTest
	 PASS: proxy_test.go:219: DNSProxyTestSuite.TearDownTest	0.000s
	 
	 FAIL: proxy_test.go:306: DNSProxyTestSuite.TestRejectNonMatchingRefusedResponse

borkmann · 2020-07-01T23:46:59Z

test-focus had vbox error:

23:26:58  Progress state: VBOX_E_INVALID_OBJECT_STATE
23:26:58  VBoxManage: error: Machine delete failed
23:26:58  VBoxManage: error: Medium '/root/VirtualBox VMs/ubuntu-18.04.4-amd64-60_1593638818730_5500/ubuntu-18.04.4-amd64-60-disk001.vmdk' is locked for reading by another task
23:26:58  VBoxManage: error: Details: code VBOX_E_INVALID_OBJECT_STATE (0x80bb0007), component MediumWrap, interface IMedium
23:26:58  VBoxManage: error: Context: "RTEXITCODE handleUnregisterVM(HandlerArg*)" at line 162 of file VBoxManageMisc.cpp

borkmann · 2020-07-01T23:48:05Z

4.19 bailed on HealthCheckNodePort ... looking into it:
https://jenkins.cilium.io/job/Cilium-PR-Ginkgo-Tests-Kernel/2366/

borkmann · 2020-07-01T23:54:46Z

test-me-please

Finally, wire-up Kubernetes side to add surrogate entries for cluster-internal communication in case of externalTrafficPolicy=Local. We only filter local backends for ScopeExternal. ScopeInternal will have everything. Example from nginx deployment with externalTrafficPolicy=Local, listing from node apoc: # ./cilium/cilium service list ID Frontend Service Type Backend 4 10.100.125.187:80 ClusterIP 1 => 10.217.0.252:80 2 => 10.217.1.86:80 5 192.168.178.29:30465 NodePort 1 => 10.217.0.252:80 6 0.0.0.0:30465 NodePort 1 => 10.217.0.252:80 7 192.168.178.29:30465/i NodePort 1 => 10.217.0.252:80 2 => 10.217.1.86:80 8 0.0.0.0:30465/i NodePort 1 => 10.217.0.252:80 2 => 10.217.1.86:80 # ./cilium/cilium bpf lb list SERVICE ADDRESS BACKEND ADDRESS 10.100.125.187:80 0.0.0.0:0 (4) [ClusterIP] 10.217.0.252:80 (4) 10.217.1.86:80 (4) 192.168.178.29:30465 0.0.0.0:0 (5) [NodePort, Local] 10.217.0.252:80 (5) 192.168.178.29:30465/i 0.0.0.0:0 (7) [NodePort, Local] 10.217.0.252:80 (7) 10.217.1.86:80 (7) 0.0.0.0:30465 0.0.0.0:0 (6) [NodePort, Local] 10.217.0.252:80 (6) 0.0.0.0:30465/i 10.217.1.86:80 (8) 10.217.0.252:80 (8) 0.0.0.0:0 (8) [NodePort, Local] # kubectl get pods --all-namespaces -o wide NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES default nginx6-7d4b5d6bdf-7tpk4 1/1 Running 0 127m 10.217.1.86 tank <none> <none> default nginx6-7d4b5d6bdf-wd9hf 1/1 Running 0 127m 10.217.0.252 apoc <none> <none> [...] As can be seen the ClusterIP has all backends, the external facing NodePort only has local backends whereas the internal facing NodePort has all backends. The 0.0.0.0:30465 could still be optimized away since genCartesianProduct() has no awareness from the logic in ParseService() which adds the surrogate entries. Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>

... given kube-proxy-free setup supports them now, lets run these. Also add tests from third node to make sure they fail to a node that has no local backend. Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>

Add an 'umbrella' section on client source IP preservation with links to the related sections. Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>

borkmann · 2020-07-01T23:57:27Z

test-me-please

EDIT: net-next failed provisioning:
https://jenkins.cilium.io/job/Cilium-PR-K8s-oldest-net-next/1066/

borkmann · 2020-07-02T00:00:19Z

test-focus K8sService*

joestringer · 2020-07-02T02:18:11Z

retest-net-next

joestringer

docs/cli/bpf LGTM. I didn't look so closely at services side.

travisghansen · 2020-07-02T06:18:24Z

#11724 (comment)

borkmann added release-note/misc This PR makes changes that have no direct user impact. area/kube-proxy-free labels Jun 26, 2020

borkmann requested review from gandro and brb June 26, 2020 22:13

maintainer-s-little-helper bot added this to Needs backport from master in 1.8.1 Jun 26, 2020

borkmann force-pushed the pr/traffic-policy-local branch from 8f90365 to 7a409c6 Compare June 26, 2020 22:34

borkmann force-pushed the pr/traffic-policy-local branch 2 times, most recently from 3be22fd to ee7e264 Compare June 29, 2020 21:34

borkmann added the priority/high This is considered vital to an upcoming release. label Jun 29, 2020

borkmann changed the title ~~bpf: allow in-cluster connectivity for externalTrafficPolicy=Local~~ bpf: fix in-cluster connectivity for externalTrafficPolicy=Local Jun 29, 2020

borkmann force-pushed the pr/traffic-policy-local branch 2 times, most recently from 824df7d to d345586 Compare June 30, 2020 09:59

gandro requested changes Jun 30, 2020

View reviewed changes

pkg/service/service.go Outdated Show resolved Hide resolved

borkmann force-pushed the pr/traffic-policy-local branch 4 times, most recently from 672c047 to e7ac2a0 Compare June 30, 2020 12:23

borkmann marked this pull request as ready for review June 30, 2020 12:24

borkmann requested a review from a team as a code owner June 30, 2020 12:24

borkmann requested a review from a team June 30, 2020 12:24

borkmann requested review from a team as code owners June 30, 2020 12:24

borkmann requested review from a team June 30, 2020 12:24

borkmann requested a review from a team as a code owner June 30, 2020 12:45

borkmann force-pushed the pr/traffic-policy-local branch from 77325b3 to aa73fd9 Compare June 30, 2020 12:49

borkmann force-pushed the pr/traffic-policy-local branch from 465a50e to 4675fe0 Compare July 1, 2020 20:54

borkmann requested a review from a team as a code owner July 1, 2020 23:50

borkmann added 3 commits July 2, 2020 01:56

tests: enable additional externalTrafficPolicy=Local tests

79d2b6a

... given kube-proxy-free setup supports them now, lets run these. Also add tests from third node to make sure they fail to a node that has no local backend. Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>

docs: misc improvements to kube-proxy free gsg

775b097

Add an 'umbrella' section on client source IP preservation with links to the related sections. Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>

borkmann force-pushed the pr/traffic-policy-local branch from 34e6282 to 775b097 Compare July 1, 2020 23:57

joestringer approved these changes Jul 2, 2020

View reviewed changes

joestringer merged commit 30f44a5 into master Jul 2, 2020

joestringer deleted the pr/traffic-policy-local branch July 2, 2020 04:21

joestringer mentioned this pull request Jul 2, 2020

v1.8.1 release #12373

Closed

26 tasks

borkmann mentioned this pull request Jul 2, 2020

v1.8 backport 2020-07-02 #12378

Merged

borkmann added backport-pending/1.8 and removed needs-backport/1.8 labels Jul 2, 2020

maintainer-s-little-helper bot moved this from Needs backport from master to Backport pending to v1.8 in 1.8.1 Jul 2, 2020

borkmann added backport-done/1.8 and removed backport-pending/1.8 labels Jul 2, 2020

maintainer-s-little-helper bot moved this from Backport pending to v1.8 to Backport done to v1.8 in 1.8.1 Jul 2, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

bpf: fix in-cluster connectivity for externalTrafficPolicy=Local #12311

bpf: fix in-cluster connectivity for externalTrafficPolicy=Local #12311

borkmann commented Jun 26, 2020 •

edited

coveralls commented Jun 26, 2020 •

edited

gandro left a comment

borkmann commented Jun 30, 2020

borkmann commented Jun 30, 2020

borkmann commented Jul 1, 2020

borkmann commented Jul 1, 2020

borkmann commented Jul 1, 2020

borkmann commented Jul 1, 2020

borkmann commented Jul 1, 2020

borkmann commented Jul 1, 2020

borkmann commented Jul 1, 2020

borkmann commented Jul 1, 2020

borkmann commented Jul 1, 2020 •

edited by joestringer

borkmann commented Jul 2, 2020

joestringer commented Jul 2, 2020

joestringer left a comment

travisghansen commented Jul 2, 2020

bpf: fix in-cluster connectivity for externalTrafficPolicy=Local #12311

bpf: fix in-cluster connectivity for externalTrafficPolicy=Local #12311

Conversation

borkmann commented Jun 26, 2020 • edited

coveralls commented Jun 26, 2020 • edited

gandro left a comment

Choose a reason for hiding this comment

borkmann commented Jun 30, 2020

borkmann commented Jun 30, 2020

borkmann commented Jul 1, 2020

borkmann commented Jul 1, 2020

borkmann commented Jul 1, 2020

borkmann commented Jul 1, 2020

borkmann commented Jul 1, 2020

borkmann commented Jul 1, 2020

borkmann commented Jul 1, 2020

borkmann commented Jul 1, 2020

borkmann commented Jul 1, 2020 • edited by joestringer

borkmann commented Jul 2, 2020

joestringer commented Jul 2, 2020

joestringer left a comment

Choose a reason for hiding this comment

travisghansen commented Jul 2, 2020

borkmann commented Jun 26, 2020 •

edited

coveralls commented Jun 26, 2020 •

edited

borkmann commented Jul 1, 2020 •

edited by joestringer