Improve performance of BPF map buffer writes#1312
Merged
Merged
Conversation
brancz
approved these changes
Feb 7, 2023
Member
brancz
left a comment
There was a problem hiding this comment.
I think I would have personally wrapped the byte slice in a struct and kept a current index instead (I just find that a little easier to reason about), but this works perfectly well!
df2cff0 to
dda31b7
Compare
dda31b7 to
0916278
Compare
Using `binary.Write` to write to memory buffers we use to update BPF maps is not only using quite a bit of CPU cycles but also incurs in quite a bit of small allocations. In particular, every single write performs an allocation [0]. This commit reduces allocations to 0 and CPU cycles by ~14x by writing to the buffer without performing any allocations. We do this with a new type, defined over a byte slice, which directly writes to the buffer using the lower level `binary` APIs. This also introduces two other advantages: - The underlying array backing the slice will never grow in case of bugs. We want the buffers to be static and of exactly the size we expect in BPF. - We can zero the buffers more efficiently. We need to do this to ensure that there is not stale information in the buffers we reuse. This new type has to be used with care as we expect the `Slice` method to request a size equal or smaller to the available capacity. Test Plan ========= Added unit-tests, ran locally for a while with no issues and `binary.Write` barely showing up in the profiles. The new methods don't take much CPU either. ``` $ go test -v ./pkg/profiler/ -bench=. -count=30 > bench.txt ``` ``` $ benchstat bench.txt name time/op EfficientBufferSliceWrite-12 2.18ns ± 4% BinaryWrite-12 31.3ns ±26% name alloc/op EfficientBufferSliceWrite-12 8.00B ± 0% BinaryWrite-12 16.0B ± 0% name allocs/op EfficientBufferSliceWrite-12 0.00 BinaryWrite-12 1.00 ± 0% ``` [0]: https://github.com/golang/go/blob/02d8ebda8398342516a24a3e8435d527ed156cab/src/encoding/binary/binary.go#L345 Signed-off-by: Francisco Javier Honduvilla Coto <javierhonduco@gmail.com>
0916278 to
14ee01c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Using
binary.Writeto write to memory buffers we use to update BPF maps is not only using quite a bit of CPU cycles but also incurs quite a bit of small allocations. In particular, every single write performs an allocation.This commit reduces allocations to 0 and CPU cycles by ~14x by writing to the buffer without performing any allocations. We do this with a new type, defined over a byte slice, which directly writes to the buffer using the lower-level
binaryAPIs.This also introduces two other advantages:
This new type has to be used with care as we expect the
Slicemethod to request a size equal or smaller to the available capacity.Test Plan
Added unit tests, ran locally for a while with no issues and
binary.Writebarely showing up in the profiles. The new methods don't take much CPU either.Part of #1286
Signed-off-by: Francisco Javier Honduvilla Coto javierhonduco@gmail.com