Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: Cilium L4LB XDP - NAT46x64 test #25892

Closed
rastislavs opened this issue Jun 5, 2023 · 5 comments
Closed

CI: Cilium L4LB XDP - NAT46x64 test #25892

rastislavs opened this issue Jun 5, 2023 · 5 comments
Labels
area/CI Continuous Integration testing issue or flake ci/flake This is a known failure that occurs in the tree. Please investigate me!

Comments

@rastislavs
Copy link
Contributor

rastislavs commented Jun 5, 2023

CI failure

Error:

Run cd /home/runner/work/cilium/cilium/test/nat46x64 && sudo ./test.sh cilium 4f91a9ec07a4cb879bdd780a3e910b81ceda38de

...

Error: Cannot add/update recorder: [PUT /recorder/{id}][500] putRecorderIdFailure  loading eBPF collection into the kernel: program tail_lb_ipv6: load program: argument list too long: BPF program is too large. Processed 1000001 insn (2474 line(s) omitted)

Hit on two different PRs already:

Sysdump could not be retrieved due to an error:

Run sudo cilium sysdump --output-filename cilium-sysdump-out
🔍 Collecting sysdump with cilium-cli version: v0.14.5, args: [sysdump --output-filename cilium-sysdump-out]
failed to create sysdump collector: failed to detect Cilium namespace: Get "http://localhost:8080/api/v1/namespaces/kube-system": dial tcp [::1]:8080: connect: connection refused
@rastislavs rastislavs added area/CI Continuous Integration testing issue or flake ci/flake This is a known failure that occurs in the tree. Please investigate me! labels Jun 5, 2023
@sayboras
Copy link
Member

sayboras commented Jun 5, 2023

Happens in the below as well

PR: #25711
Run: https://github.com/cilium/cilium/actions/runs/5161149871/attempts/2

@julianwiedmann
Copy link
Member

julianwiedmann commented Jun 5, 2023

The funky thing is that the same test then succeeds when re-running.

@sayboras
Copy link
Member

sayboras commented Jun 5, 2023

seems like I am not lucky enough, 5 failures and counting

https://github.com/cilium/cilium/actions/runs/5161149871

@julianwiedmann
Copy link
Member

Looks like a complexity regression in the verifier, I see

good run: 5.15.0-1036-azure
bad run: 5.15.0-1038-azure

And as GH is slowly updating their fleet, we now probably end up on bad kernels more often and fail consistently.

@rastislavs
Copy link
Contributor Author

It seems that #25826 resolved the issue, the test is passing on my PRs after rebasing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/CI Continuous Integration testing issue or flake ci/flake This is a known failure that occurs in the tree. Please investigate me!
Projects
None yet
Development

No branches or pull requests

3 participants