Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix fromCIDR policy on kernels 4.10 or older and extend test coverage #11333

Merged

Conversation

willdeuschle
Copy link

Builds on the draft PR originally opened by @joestringer #10580

That PR body:

This series extends the runtime CI tests to validate that fromCIDR policy works correctly using a local docker network, similar to how existing FQDN tests do.

Review patch-by-patch, there is some large refactoring of the CIDR tests in policy but it's mostly kept to one non-functional patch.
On top of that, see the final commit I added. The tests were failing in CI (and for me locally) because for kernel versions without LPM maps, ipcache_lookup4 is unable to lookup CIDRs based on a prefix other than a /32. Changing from ipcache_lookup4 to lookup_ip4_remote_endpoint fixes this because the latter function uses unrolled hash map lookups to simulate a LPM map lookups for kernels that don't support LPM maps. See bpf/lib/eps.h for further details on that.

It may be desirable to replace usage of ipcache_lookup[46] everywhere with lookup_ip[46]_remote_endpoint but that is beyond the scope of this PR.

Make use of Context to share more test preparation for the CIDR tests,
and allow the toCIDR / fromCIDR tests to be split out to validate them
separately.

Signed-off-by: Joe Stringer <joe@cilium.io>
This docker network will be used by upcoming CIDR policy tests, so share
it in the common helpers package.

Signed-off-by: Joe Stringer <joe@cilium.io>
Previously, we only tested that CIDR policy does not unintentionally
open up connectivity to containers that reside within the CIDR range.

This test now actually validates that the "fromCIDR" policy applies to
traffic from outside Cilium's control, assuming it resides within the IP
range allowed by the policy.

Signed-off-by: Joe Stringer <joe@cilium.io>
Signed-off-by: Joe Stringer <joe@cilium.io>
@willdeuschle willdeuschle requested a review from a team May 5, 2020 04:55
@willdeuschle willdeuschle requested a review from a team as a code owner May 5, 2020 04:55
@maintainer-s-little-helper
Copy link

Please set the appropriate release note label.

@coveralls
Copy link

coveralls commented May 5, 2020

Coverage Status

Coverage remained the same at 42.624% when pulling ddda0bb on willdeuschle:wd/resubmit-from-cidr-runtime-tests into ff5683c on cilium:master.

@joestringer joestringer added the release-note/bug This PR fixes an issue in a previous release of Cilium. label May 5, 2020
@joestringer joestringer added the sig/datapath Impacts bpf/ or low-level forwarding details, including map management and monitor messages. label May 5, 2020
@nebril
Copy link
Member

nebril commented May 5, 2020

test-me-please

Copy link
Member

@joestringer joestringer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BPF changes LGTM. I suggest someone else reviews my CI changes in this PR :-)

@will would you mind renaming the PR to something like "Fix fromCIDR policy on kernels 4.10 or older and extend test coverage"? This will be useful for generating the release notes to clearly highlight which release gets this fix.

@will
Copy link

will commented May 6, 2020

BPF changes LGTM. I suggest someone else reviews my CI changes in this PR :-)

@will would you mind renaming the PR to something like "Fix fromCIDR policy on kernels 4.10 or older and extend test coverage"? This will be useful for generating the release notes to clearly highlight which release gets this fix.

Sorry, it doesn't look like I have permissions to change the name of the PR.

@willdeuschle willdeuschle changed the title CI: Extend runtime tests to validate fromCIDR policy Fix fromCIDR policy on kernels 4.10 or older and extend test coverage May 6, 2020
@willdeuschle
Copy link
Author

Updated, looks like the test failure was an etcd flake

@joestringer
Copy link
Member

@will ha! My apologies, my not-so-nimble fingers intended that for @willdeuschle ;-)

@joestringer
Copy link
Member

@willdeuschle I wouldn't worry too much about the GKE CI run right now, it's not marked required and there are known flakes on that job at the moment. I think we just need CI review.

Copy link
Member

@ianvernon ianvernon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work!

@maintainer-s-little-helper maintainer-s-little-helper bot added the ready-to-merge This PR has passed all tests and received consensus from code owners to merge. label May 6, 2020
bpf/bpf_netdev.c Show resolved Hide resolved
@pchaigno pchaigno removed the ready-to-merge This PR has passed all tests and received consensus from code owners to merge. label May 6, 2020
@pchaigno
Copy link
Member

pchaigno commented May 6, 2020

It looks like the bug was introduced in 1.2 so we would at least need to backport to 1.7. Do we also want to backport to previous releases? As far as I understand the impact of the bug is unexpected packet drops, right?
For the 1.7 backport, do we usually backport tests alongside the fix or should we just backport that last commit?

@maintainer-s-little-helper maintainer-s-little-helper bot added this to Needs backport from master in 1.7.4 May 6, 2020
@joestringer
Copy link
Member

I'm not sure whether the tests will backport cleanly as they might be relying on other refactoring that occurred during the v1.8 dev cycle. I don't mind taking on the backport of the functional commit only to v1.7. I'd probably avoid backporting further just given that we have no user reports of this issue.

@willdeuschle willdeuschle force-pushed the wd/resubmit-from-cidr-runtime-tests branch from 685c4b4 to 69258bb Compare May 7, 2020 04:29
Signed-off-by: Will Deuschle <wdeuschle@palantir.com>
@willdeuschle willdeuschle force-pushed the wd/resubmit-from-cidr-runtime-tests branch from 69258bb to ddda0bb Compare May 7, 2020 17:57
@joestringer
Copy link
Member

test-me-please

@joestringer joestringer self-assigned this May 7, 2020
@joestringer joestringer merged commit bad730c into cilium:master May 7, 2020
1.8.0 automation moved this from In progress to Merged May 7, 2020
@maintainer-s-little-helper maintainer-s-little-helper bot added the ready-to-merge This PR has passed all tests and received consensus from code owners to merge. label May 7, 2020
@maintainer-s-little-helper maintainer-s-little-helper bot moved this from Needs backport from master to Backport pending to v1.7 in 1.7.4 May 7, 2020
@maintainer-s-little-helper maintainer-s-little-helper bot moved this from Backport pending to v1.7 to Backport done to v1.7 in 1.7.4 May 14, 2020
@maintainer-s-little-helper maintainer-s-little-helper bot moved this from Backport pending to v1.7 to Backport done to v1.7 in 1.7.4 May 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready-to-merge This PR has passed all tests and received consensus from code owners to merge. release-note/bug This PR fixes an issue in a previous release of Cilium. sig/datapath Impacts bpf/ or low-level forwarding details, including map management and monitor messages.
Projects
No open projects
1.7.4
Backport done to v1.7
1.8.0
  
Merged
Development

Successfully merging this pull request may close these issues.

None yet

8 participants