feat(autotune): embed fpm-tune as a built-in runtime PHP-FPM autotuner - #129
Merged
Merged
Conversation
cbox-init sized php-fpm only once, at boot, from a static profile, and never observed the running master or resized it as load changed. This adds a built-in runtime autotuner: a `global.fpm_tune` block embeds cboxdk/fpm-tune's serve loop, which measures live per-worker memory (PSS), and in apply mode rewrites a pool drop-in and reloads php-fpm with SIGUSR2 (never a restart), validating each change against a throwaway copy and rolling back if the master does not come back. The boot-time calculator is kept as the seed: it gives php-fpm a sane pm.max_children before it starts (which the loop cannot, needing a running master), and the loop owns the number from there. - cmd/cbox-init/fpmtune.go: startFPMTune builds serve.Config (apply by default, p95 hybrid sizing) and runs serve.Loop.Run in a goroutine, returning a stop closure that halts the loop and releases its state lock. - cmd/cbox-init/serve.go: starts it after the processes are up, and stops it FIRST in performGracefulShutdown, before php-fpm is drained, since it rewrites and reloads php-fpm config. - config: fpm_tune block with defaults (setGlobalFPMTuneDefaults) and validation on both the fail-fast (validateFPMTune) and check-config (validateGlobalFPMTuneSettings) paths. - serve.New already takes the state-file lock and self-repairs, so no lock is added here; do not also run a standalone fpm-tune against the same pools. - example config, docs, and regenerated SBOM; pins github.com/cboxdk/fpm-tune v0.1.0-beta.20.
Runs cbox-init as PID 1 in a php:8.4-fpm container, supervising real php-fpm with the embedded fpm-tune loop in apply mode, and asserts the full chain end to end: the loop discovers www, enables its status page, sizes it, applies a resize under load, and reloads php-fpm with SIGUSR2 (the master pid is unchanged, so it is a reload, not a restart), then shuts down cleanly. The driver is external (docker exec / curl from the host) because php-fpm is a longrun, so the oneshot-verify pattern (Dockerfile.pid1) does not fit. libfcgi-bin and busy.php are baked into the image so the load needs no network. The container runs memory-bounded (--memory=512m) for a predictable budget. - tests/integration/Dockerfile.fpmtune, fpm-tune-config.yaml, e2e-fpm-tune.sh - make test-e2e-fpm-tune (depends on build-all) - CI: a debian-gated step in the integration-test job, reusing the built binary
Backward compatible: serve.New/Run/Close are unchanged and serve.Config only gained optional fields. Wire init's build version into the loop's new Config.Version, which the daemon reports on its /history.json. beta.25's CPU-measurement feature reaches gopsutil, which transitively pulls github.com/shoenig/go-m1cpu (MPL-2.0) — a darwin/arm64-only Apple Silicon CPU helper (//go:build darwin && arm64 && cgo) that is never compiled into the Linux production binary. cyclonedx lists the whole module graph regardless, so add a documented license-check exception for it: file-level weak copyleft, not modified, not shipped in prod. (The dep was already present at beta.20; the earlier license-check only passed because it ran against a pre-regeneration SBOM.) Regenerate the SBOM.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
cbox-init sized php-fpm only once, at boot, from a static workload profile
(
runAutoTuning→PHP_FPM_*env vars). It never observed the running master orresized it as load changed.
This adds a built-in runtime autotuner: a
global.fpm_tuneblock embedscboxdk/fpm-tune'sserveloop as abackground service. It measures live per-worker memory (PSS, which does not
double-count shared OPcache), and in apply mode rewrites a pool drop-in and
reloads php-fpm with
SIGUSR2(a graceful reload, never a restart), validatingeach change against a throwaway copy and rolling back if the master does not come
back.
The two compose across the timeline: the boot calculator is kept as the seed
(it gives php-fpm a sane
pm.max_childrenbefore it starts, which the loop cannot— it needs a running master), and the runtime loop owns the number from there.
Design notes
cmd/cbox-init/fpmtune.go):startFPMTunebuildsserve.Config(apply by default, p95 hybrid sizing
state.Sizing{0.95, 0.10}, workloadresolve), runs
serve.Loop.Runin a goroutine, and returns a stop closure.serve.go): started after the processes are up; stopped FIRSTin
performGracefulShutdown, before php-fpm is drained, because it rewrites andreloads php-fpm config.
serve.Newalready takes the state-file lock andself-repairs internally, so init adds none. Corollary: do not also run a
standalone
fpm-tuneagainst the same pools — the second copy refuses to start.fpm_tuneblock with defaults and validation on both the fail-fast(
validateFPMTune) and check-config (validateGlobalFPMTuneSettings) paths.Strict-decode safe (guarded by
knownfields_test.go)./metricsruns onits own port (
metrics_addr), not init's:9090.github.com/cboxdk/fpm-tune v0.1.0-beta.20; all 63 deps permissive, no vulns.Gate
fmt ✓ vet ✓ lint (0 issues) ✓ test -race ✓ vulncheck (0) ✓ license-check (63 permissive) ✓ check-configs (15/15) ✓— SBOM regenerated (additions only).End-to-end (real container)
Ran cbox-init as PID 1 in a
php:8.4-fpmcontainer withfpm_tune: apply:Runtime PHP-FPM autotuner started mode=apply interval=5sEnabled the status page on pools that lacked one pools=[www]→ wrotezz-fpm-tune-status.conf/metricsshowsfpm_tune_pool_*{pool="www"}zz-fpm-tune.confpm.max_children = 7;Pool resized pool=www from=5 to=7using inherited socket fd=9)Not in this cut
internal/autotune/calculator.go,profiles.go) stays asthe seed — not replaced.
DetectBudgetadapter — fpm-tune reads the master's cgroup, which in acontainer is init's budget.