Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CI: TestMapBatch/Hash is flaky (arm64?) #1390

Closed
lmb opened this issue Mar 22, 2024 · 2 comments · Fixed by #1485
Closed

CI: TestMapBatch/Hash is flaky (arm64?) #1390

lmb opened this issue Mar 22, 2024 · 2 comments · Fixed by #1485
Labels
bug Something isn't working

Comments

@lmb
Copy link
Collaborator

lmb commented Mar 22, 2024 •
edited
Loading

Describe the bug

TestMapBatch/Hash fails due to ENOSPC on arm64.

=== Failed
=== FAIL: . TestMapBatch/Hash (0.01s)
    quicktest.go:12: 
        error:
          got non-nil value
        got:
          e"no space left on device"
        stack:
          /home/runner/actions-runner/_work/ebpf/ebpf/map_test.go:167
            qt.Assert(t, qt.IsNil(err))
        

=== FAIL: . TestMapBatch (0.01s)

https://github.com/cilium/ebpf/actions/runs/8388876993/job/22973993334?pr=1366#step:5:311

How to reproduce

Unclear.

Version information

main
@lmb lmb added the bug Something isn't working label Mar 22, 2024
@lmb
Copy link
Collaborator Author

lmb commented Apr 9, 2024

Thinking out loud: maybe we're only seeing this on arm64 since that runner has more cores than the VM based tests (which only have two cpus)?

@dylandreimerink dylandreimerink mentioned this issue May 15, 2024
Merged
@lmb
Copy link
Collaborator Author

lmb commented Jun 10, 2024

Just hit this on 6.6 / amd64:

 === FAIL: . TestMapBatch/Hash (0.01s)
    quicktest.go:12: 
        error:
          got non-nil value
        got:
          e"no space left on device"
        stack:
          /home/runner/work/ebpf/ebpf/map_test.go:167
            qt.Assert(t, qt.IsNil(err))

https://github.com/cilium/ebpf/actions/runs/9448347532/job/26022255052?pr=1477#step:8:247

lmb added a commit to lmb/ebpf that referenced this issue Jun 10, 2024
@lmb
map: fix TestMapBatch failing with ENOSPC for hash maps
3f7c817 
The batch lookup API for hash maps returns ENOSPC when a bucket doesn't
fit into the provided batch. This is unfortunate since user space has no
way to figure out what the largest bucket size is. This makes the API
very awkward to use.
It also seems that hash tables don't deal well with sequential keys,
hashing them into the same bucket with fairly high probability.

Fix the flaky test by always making the batch size equal to the map
size, which means we can never trigger ENOSPC. Also make the error
message more descriptive.

Fixes: cilium#1390
Signed-off-by: Lorenz Bauer <lmb@isovalent.com>
@lmb lmb mentioned this issue Jun 10, 2024
Merged
lmb added a commit to lmb/ebpf that referenced this issue Jun 10, 2024
@lmb
map: fix TestMapBatch failing with ENOSPC for hash maps
f705846 
The batch lookup API for hash maps returns ENOSPC when a bucket doesn't
fit into the provided batch. This is unfortunate since user space has no
way to figure out what the largest bucket size is. This makes the API
very awkward to use.
It also seems that hash tables don't deal well with sequential keys,
hashing them into the same bucket with fairly high probability.

Fix the flaky test by always making the batch size equal to the map
size, which means we can never trigger ENOSPC. Also make the error
message more descriptive.

Fixes: cilium#1390
Signed-off-by: Lorenz Bauer <lmb@isovalent.com>
@lmb lmb closed this as completed in #1485 Jun 11, 2024
lmb added a commit that referenced this issue Jun 11, 2024
@lmb
map: fix TestMapBatch failing with ENOSPC for hash maps
9d02bd3 
The batch lookup API for hash maps returns ENOSPC when a bucket doesn't
fit into the provided batch. This is unfortunate since user space has no
way to figure out what the largest bucket size is. This makes the API
very awkward to use.
It also seems that hash tables don't deal well with sequential keys,
hashing them into the same bucket with fairly high probability.

Fix the flaky test by always making the batch size equal to the map
size, which means we can never trigger ENOSPC. Also make the error
message more descriptive.

Fixes: #1390
Signed-off-by: Lorenz Bauer <lmb@isovalent.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

map: fix TestMapBatch failing with ENOSPC for hash maps lmb/ebpf
1 participant
@lmb