Release the shared kernel BTF cache after Start() - #283
Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 1 commit intoAug 4, 2026
Merged
Conversation
Manager.Start() releases VerifierOptions.Programs.KernelTypes once loading is done (unless KeepKernelBTF is set), but it did not release VerifierOptions.Cache, which was added alongside the cilium/ebpf v0.22.0 upgrade. A long-lived manager therefore keeps the shared *btf.Cache -- and the parsed kernel vmlinux BTF it holds -- pinned for the entire process lifetime, regressing idle memory for downstream consumers. Clear VerifierOptions.Cache symmetrically with KernelTypes, extracted into a small releaseKernelBTF helper. The cache is only consumed while loading the collection (before Start); nothing reads it afterwards: CloneProgram reloads through the exported NewProgramWithOptions, which allocates its own fresh cache and relies on KernelTypes for CO-RE relocations. Gated on KeepKernelBTF so callers that clone programs are unaffected. Add TestReleaseKernelBTF covering both the release and retain paths. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
mbertrone
added a commit
to DataDog/datadog-agent
that referenced
this pull request
Jul 23, 2026
Pin DataDog/ebpf-manager to the commit that clears the shared BTF cache and KernelTypes spec in Manager.Start() once program loading finishes, so a long-lived system-probe no longer keeps the parsed kernel vmlinux BTF pinned for the whole process lifetime (idle-memory regression). Temporary pseudo-version pending the v0.8.1 tag; see DataDog/ebpf-manager#283. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
gh-worker-dd-mergequeue-cf854d Bot
pushed a commit
to DataDog/datadog-agent
that referenced
this pull request
Jul 27, 2026
) ### What does this PR do? Bumps `github.com/DataDog/ebpf-manager` to the commit that releases kernel BTF once eBPF program loading finishes. `Manager.Start()` now clears both `VerifierOptions.Programs.KernelTypes` and the shared `VerifierOptions.Cache` after the collection is loaded (unless the caller opts out via `KeepKernelBTF`). Pinned as a temporary pseudo-version (`v0.8.1-0.20260723114030-df952c39a4e3`) pending the `v0.8.1` tag — see DataDog/ebpf-manager#283. ### Motivation A long-lived `system-probe` kept the parsed kernel vmlinux BTF pinned in the shared verifier cache for the entire process lifetime, even though it is only needed while loading programs. This regressed idle heap/RSS. Releasing the cache after load lets that memory be reclaimed. ### Describe how you validated your changes - Upstream change carries a unit test (`TestReleaseKernelBTF`) covering both the default-release and `KeepKernelBTF`-retain paths. - Built `system-probe` against the pinned version and confirmed it compiles and loads eBPF programs (NPM + USM + CWS) without errors. ### Additional Notes Follow-up: once DataDog/ebpf-manager#283 merges and `v0.8.1` is tagged, swap the pseudo-version for the tag and re-run `dda inv tidy`. Co-authored-by: matteo.bertrone <matteo.bertrone@datadoghq.com>
mbertrone
marked this pull request as ready for review
August 3, 2026 14:38
brycekahle
approved these changes
Aug 3, 2026
gh-worker-dd-mergequeue-cf854d
Bot
deleted the
mbertrone/release-btf-cache-after-start
branch
August 4, 2026 10:39
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Manager.Start()already releasesVerifierOptions.Programs.KernelTypesonce program loading is done (unlessKeepKernelBTFis set), but it never releasedVerifierOptions.Cache— the shared*btf.Cacheintroduced with the cilium/ebpf v0.22.0 upgrade. A long-lived manager therefore keeps that cache, and the parsed kernelvmlinuxBTF it holds, pinned for the entire process lifetime, regressing idle memory for downstream consumers.This clears
VerifierOptions.Cachesymmetrically withKernelTypes, via a smallreleaseKernelBTF()helper.Why it's safe
VerifierOptions.Cacheis only consumed while loading the collection (loadCollection→ebpf.NewCollectionWithOptions), which runs duringInitWithOptions/postInit, strictly beforeStart()..CacheafterStart(). The only post-Start program load,CloneProgram, goes through the exportedebpf.NewProgramWithOptions, which allocates its own freshbtf.NewCache()and relies onKernelTypes(not.Cache) for CO-RE relocations — which is why the clear stays gated on!KeepKernelBTF.Why it's sufficient
The loaded
ebpf.Collection/ebpf.Programdon't retain the cache (aProgramkeeps only its ownbtf.Handle); the*btf.Cacheotherwise lives only in the transient collection loader. So the manager'sVerifierOptions.Cachefield is the only long-lived reference — clearing it lets the parsed kernel BTF become GC-eligible.Test
TestReleaseKernelBTF(table-driven, no kernel/root needed) covers both paths: released when!KeepKernelBTF, retained when set.gofmtandgo vetclean.🤖 Generated with Claude Code