sandbox: shrink image 7.2GB → 3.8GB (cache cleanup, multi-stage Go build, drop ZAP)#474
Conversation
Greptile SummaryThis PR substantially reduces the sandbox image size while keeping the remaining tools available. The main changes are:
Confidence Score: 5/5This looks safe to merge.
Important Files Changed
Reviews (5): Last reviewed commit: "sandbox: keep trufflehog self-update wor..." | Re-trigger Greptile |
44e87ca to
daf39a2
Compare
Three safe reclamations to containers/Dockerfile that trim the final image from ~9.8GB to ~7.5GB without removing any user-facing tool: - `go clean -modcache` after the ProjectDiscovery installs removes ~1.7GB of Go module sources. The resulting binaries in /home/pentester/go/bin are self-contained and don't need the module cache at runtime. - Purge non-English glibc locales (~160MB). Sandbox runs with LANG=C.UTF-8 implicitly; no locale beyond en/en_US/C is used. - Remove /usr/share/doc, /usr/share/doc-base, and /usr/share/man (~95MB). Runtime doesn't call man or read package docs. Verified by building + smoke-testing the slim image: all path-visible tools (semgrep, nuclei, httpx, katana, trivy, zaproxy, trufflehog, gitleaks, sqlmap, ffuf, subfinder, naabu, bandit, arjun, dirsearch, wafw00f, tree-sitter, ast-grep, eslint, retire, jshint, js-beautify, wapiti) report version / usage cleanly on the slim image.
man-db is vestigial once the man page tree is removed in the cleanup layer — the man binary has nothing to serve. Dropping it from the install list recovers a few more MB and avoids shipping a no-op package + its dependencies (groff-base, libpipeline1, etc). Raised by greptile on the slim-image PR.
6b9b16a to
c23d135
Compare
E2E test results — slimmed sandbox image (updated)Tested by building the image locally and exercising it with Docker + real Strix scans (shell-only; no UI). Final image Every tool present & functional (full system-prompt TOOLS list + Dockerfile utilities)Recon/scan: nmap 7.99 · ncat · ndiff · subfinder · naabu 2.6.1 · httpx · gospider v1.1.6 Real network runs (with SAST chain detects seeded vulnssemgrep (3 findings) · bandit (B608/B602/B105) · gitleaks (github-pat) · trufflehog (Github) · ast-grep (matched sinks) · tree-sitter (parsed → compiles grammar via the retained gcc, proving that keep-decision). trufflehog self-update — fixed (commit
|
The trufflehog binary is root-owned but the sandbox runs as non-root
pentester, so its self-update fails ('cannot move binary') and aborts
the scan. Pin the version and wrap the binary to force --no-update
(guarded so a caller-supplied --no-update doesn't duplicate the flag).
…ned dir) Instead of disabling self-update, install trufflehog under /home/pentester/.local/bin (on PATH, owned by the non-root runtime user) so its in-place binary swap succeeds. This fixes the 'cannot move binary' abort while preserving runtime detector freshness. Initial version pinned for reproducible builds.
|
LGTM! |
Summary
Three safe reclamations to
containers/Dockerfilethat shrink the final image by ~2.3GB / ~23% without removing any user-facing tool:go clean -modcacheaftergo installchainen,en_US,C)/usr/share/{doc,doc-base,man}Measured: 9.78GB → 7.49GB on linux/amd64 build.
Why
The sandbox image is pulled per-scan in CI setups that use Strix, and cold-runner
docker pull/docker loadtimes are ~3-5 min, dominating the warm-up envelope for fast scans. Trimming ~2GB directly reduces that latency. No functional change — every binary present before is present after.Safety
Go modcache.
go install -v <pkg>@latestdownloads sources into$GOPATH/pkg/modand produces a statically-linked binary in$GOPATH/bin. The binaries don't reference the module cache at runtime.go clean -modcacheremoves the source tree; subsequentgo installinvocations re-download on demand (same behaviour as a fresh container).Verified locally by running each Go binary after the clean:
Locales. The image runs with
LANG=C.UTF-8implicitly (no explicitLANG=set, nolocalespackage pulled in with a locale-gen step). glibc falls back toCwhen requested locales are absent. Purging/usr/share/locale/*excepten,en_US,Cchanges nothing for any process running in the sandbox.Docs/man. Tool runtime doesn't call
manor read/usr/share/doc/*. Package postinst scripts reference these during install, not at runtime.All three reclamations ride inside the existing
apt-get autoremovecleanup layer, so no new layer is introduced.Smoke test (manual, linux/amd64)
All tools report OK on the slim image. Size confirmed via
docker images.Non-goals
docker-entrypoint.sh, Caido proxy, or Python/Node runtime.github/workflows/*that builds the sandbox today, so this PR has been tested manually)Update: additional cache reclamation (+~1.1GB)
Follow-up commit
sandbox: also clear go build cache and npm cacheextends the same "safe, same-layer, no tool removal" approach:go clean -cache(added alongside the existing-modcache)npm cache clean --forceafter thenpm install -gchainWhy:
go clean -modcacheonly removes the module source cache ($GOPATH/pkg/mod); the Go build cache ($GOCACHE,~/.cache/go-build) was still shipping in the image at ~1.1GB. It only speeds up recompilation and is never read at runtime, so clearing it is safe. Likewise the npm cache (~/.npm) is build-time only.Measured (linux/amd64, all off the same
mainbase):main(base)Net vs base: −2.49GB (~34%).
Verified on the trimmed image: all 30 tools present and runnable; every Go binary executes after the cache clean;
tree-sitter parseworks;httpxlive-probes a target (200);$GOCACHEdown to ~12K. A fullstrixscan boots the sandbox and runs httpx/nuclei from the trimmed image with no functional change.Update 2: multi-stage build + drop ZAP & C/C++ build chain (−~1GB more)
Structural trims on top of the cache work. Same "no user-facing tool loss" philosophy, with the one intentional exception of ZAP (removed on request).
golang-go, ~225MB) + module/build caches now live only in agobuilderstage; just the 5 static binaries (httpx,katana,vulnx,gospider,interactsh-client) areCOPY --from'd into runtimebuild-essential,pkg-config,libpcap-dev,libssl-dev,python3-dev)@ast-grep/cliships two byte-identical binaries (ast-grepandsg); symlinksg → ast-grepKept:
gcc+libc6-dev(tree-sitter compiles grammar.sos on demand at runtime — dropping the C compiler breakstree-sitter parse), plustrivy,wapiti,caido,uv, andvim. Also removed the twozaproxyreferences and the staleGo/go installmention from the agent system prompt (the Go scanners are prebuilt and on PATH).Measured (linux/amd64): 7.24GB → 3.77GB (−3.47GB / −48% vs base).
Verified on the final image: all tools present & runnable except the intentionally-removed ZAP (
httpx/katana/vulnx/gospider/interactsh-client,trivy,wapiti,caido-cli,nuclei/subfinder/naabu/ffuf/sqlmap,semgrep/bandit/trufflehog/gitleaks,uv,vim);sg runandtree-sitter parsework (the latter compilespython.sovia the retainedgcc);goandzaproxyare gone as intended.Link to Devin session: https://app.devin.ai/sessions/7bfd32c02ecd423fa3654576b9352c04
Requested by: @0xallam