bpf: Preserve source identity for hairpin via stack

[tgraf]

When Cilium is used in chaining mode with portmap, the hostPort is
translated using iptables DNAT as inserted by the portmap plugin. When
this happens all within a node, we can preserve the source identity for
the reply traffic for correct visibility. The traffic will be allowed
anyway based on the connection tracking state.

Updates: #9784

[coveralls]

Coverage decreased (-0.008%) to 46.639% when pulling 5b80b69 on pr/tgraf/preserve-portmap-identity into 5958dd6 on master.

[tgraf]

test-me-please

The privileged tests seem to have been interrupted:

00:44:22.643    Run Tests [It]
00:44:22.643    /home/jenkins/workspace/Cilium-PR-Ginkgo-Tests-Validated/runtime-gopath/src/github.com/cilium/cilium/test/ginkgo-ext/scopes.go:430
00:44:22.643  
00:44:22.643    Failed to run privileged unit tests
[2020-04-10T01:04:53.068Z]   Expected command: sudo make -C /home/vagrant/go/src/github.com/cilium/cilium/ tests-privileged 
[2020-04-10T01:04:53.068Z]   To succeed, but it failed:
[2020-04-10T01:04:53.068Z]   Exitcode: 1 
[2020-04-10T01:04:53.068Z]   Stdout:
[2020-04-10T01:04:53.068Z]    	 make: Entering directory '/home/vagrant/go/src/github.com/cilium/cilium'
[2020-04-10T01:04:53.068Z]   	 # cilium-map-migrate is a dependency of some unit tests.
[2020-04-10T01:04:53.068Z]   	 make  -C bpf cilium-map-migrate
[2020-04-10T01:04:53.068Z]   	 make[1]: Entering directory '/home/vagrant/go/src/github.com/cilium/cilium/bpf'
[2020-04-10T01:04:53.068Z]   	 make[1]: 'cilium-map-migrate' is up to date.
[2020-04-10T01:04:53.068Z]   	 make[1]: Leaving directory '/home/vagrant/go/src/github.com/cilium/cilium/bpf'
[2020-04-10T01:05:08.034Z]   	 CGO_ENABLED=0 go test -mod=vendor -ldflags "-X github.com/cilium/cilium/pkg/kvstore.consulDummyAddress=https://consul:8443 -X github.com/cilium/cilium/pkg/kvstore.etcdDummyAddress=http://etcd:4002 -X github.com/cilium/cilium/pkg/testutils.CiliumRootDir=/home/vagrant/go/src/github.com/cilium/cilium -X github.com/cilium/cilium/pkg/datapath.DatapathSHA=1234567890abcdef7890" github.com/cilium/cilium/cilium/cmd -test.v -timeout 360s -check.vv -tags=privileged_tests || exit 1;  CGO_ENABLED=0 go test -mod=vendor -ldflags "-X github.com/cilium/cilium/pkg/kvstore.consulDummyAddress=https://consul:8443 -X github.com/cilium/cilium/pkg/kvstore.etcdDummyAddress=http://etcd:4002 -X github.com/cilium/cilium/pkg/testutils.CiliumRootDir=/home/vagrant/go/src/github.com/cilium/cilium -X github.com/cilium/cilium/pkg/datapath.DatapathSHA=1234567890abcdef7890" github.com/cilium/cilium/daemon/cmd -test.v -timeout 360s -check.vv -tags=privileged_tests || exit 1;  CGO_ENABLED=0 go test -mod=vendor -ldflags "-X github.com/cilium/cilium/pkg/kvstore.consulDummyAddress=https://consul:8443 -X github.com/cilium/cilium/pkg/kvstore.etcdDummyAddress=http://etcd:4002 -X github.com/cilium/cilium/pkg/testutils.CiliumRootDir=/home/vagrant/go/src/github.com/cilium/cilium -X github.com/cilium/cilium/pkg/datapath.DatapathSHA=1234567890abcdef7890" github.com/cilium/cilium/pkg/bpf -test.v -timeout 360s -check.vv -tags=privileged_tests || exit 1;  CGO_ENABLED=0 go test -mod=vendor -ldflags "-X github.com/cilium/cilium/pkg/kvstore.consulDummyAddress=https://consul:8443 -X github.com/cilium/cilium/pkg/kvstore.etcdDummyAddress=http://etcd:4002 -X github.com/cilium/cilium/pkg/testutils.CiliumRootDir=/home/vagrant/go/src/github.com/cilium/cilium -X github.com/cilium/cilium/pkg/datapath.DatapathSHA=1234567890abcdef7890" github.com/cilium/cilium/pkg/datapath/link -test.v -timeout 360s -check.vv -tags=privileged_tests || exit 1;  CGO_ENABLED=0 go test -mod=vendor -ldflags "-X github.com/cilium/cilium/pkg/kvstore.consulDummyAddress=https://consul:8443 -X github.com/cilium/cilium/pkg/kvstore.etcdDummyAddress=http://etcd:4002 -X github.com/cilium/cilium/pkg/testutils.CiliumRootDir=/home/vagrant/go/src/github.com/cilium/cilium -X github.com/cilium/cilium/pkg/datapath.DatapathSHA=1234567890abcdef7890" github.com/ci
[2020-04-10T01:05:08.034Z] Ginkgo ran 1 suite in 9m22.853817995s
[2020-04-10T01:05:08.034Z] Test Suite Failed

The k8s tests contain several errors which seem to indicate a setup failure

time="2020-04-10T00:57:02Z" level=debug msg="running command: kubectl -n kube-system get pods -l k8s-app=cilium -o json"
time="2020-04-10T00:57:12Z" level=error msg="Error executing command 'kubectl -n kube-system get pods -l k8s-app=cilium -o json'" error="signal: killed"
time="2020-04-10T00:57:12Z" level=error msg="Error executing command 'kubectl -n kube-system get pods -l k8s-app=cilium -o json'" error="signal: killed"
cmd: "kubectl -n kube-system get pods -l k8s-app=cilium -o json" exitCode: -1 duration: 10.001587473s stdout:

time="2020-04-10T00:57:12Z" level=info msg="Error while getting PodList: unexpected end of JSON input"
time="2020-04-10T00:57:12Z" level=debug msg="running command: kubectl -n kube-system get pods -l k8s-app=cilium -o json"
time="2020-04-10T00:57:22Z" level=error msg="Error executing command 'kubectl -n kube-system get pods -l k8s-app=cilium -o json'" error="signal: killed"
time="2020-04-10T00:57:22Z" level=error msg="Error executing command 'kubectl -n kube-system get pods -l k8s-app=cilium -o json'" error="signal: killed"
cmd: "kubectl -n kube-system get pods -l k8s-app=cilium -o json" exitCode: -1 duration: 10.001626092s stdout:

[jrfastab]

Patch looks OK to me but looks like some tests failed. I scanned the logs and wasn't obvious to me off-hand what went wrong.

[tgraf]

test-me-please

Hit #10942

[tgraf]

test-me-please

[brb]

Too late to the party, but this is breaking BPF masquerading and BPF NodePort, as the mark set in this PR clashes with MARK_MAGIC_SNAT_DONE which makes https://github.com/cilium/cilium/blob/master/bpf/bpf_netdev.c#L715 to bypass nodeport_nat_fwd().