Skip to content

feat(autotune): embed fpm-tune as a built-in runtime PHP-FPM autotuner - #129

Merged
sylvesterdamgaard merged 3 commits into
mainfrom
feat/embed-fpm-tune-runtime-autotuner
Sep 4, 2026
Merged

feat(autotune): embed fpm-tune as a built-in runtime PHP-FPM autotuner#129
sylvesterdamgaard merged 3 commits into
mainfrom
feat/embed-fpm-tune-runtime-autotuner

Conversation

@sylvesterdamgaard

Copy link
Copy Markdown
Contributor

What

cbox-init sized php-fpm only once, at boot, from a static workload profile
(runAutoTuningPHP_FPM_* env vars). It never observed the running master or
resized it as load changed.

This adds a built-in runtime autotuner: a global.fpm_tune block embeds
cboxdk/fpm-tune's serve loop as a
background service. It measures live per-worker memory (PSS, which does not
double-count shared OPcache), and in apply mode rewrites a pool drop-in and
reloads php-fpm with SIGUSR2 (a graceful reload, never a restart), validating
each change against a throwaway copy and rolling back if the master does not come
back.

The two compose across the timeline: the boot calculator is kept as the seed
(it gives php-fpm a sane pm.max_children before it starts, which the loop cannot
— it needs a running master), and the runtime loop owns the number from there.

global:
  fpm_tune:
    enabled: true
    mode: apply           # write drop-ins + reload; "advisory" to only recommend
    interval: 30s
    metrics_addr: ":9110" # fpm_tune_* metrics; empty disables

Design notes

  • Driver (cmd/cbox-init/fpmtune.go): startFPMTune builds serve.Config
    (apply by default, p95 hybrid sizing state.Sizing{0.95, 0.10}, workload
    resolve), runs serve.Loop.Run in a goroutine, and returns a stop closure.
  • Lifecycle (serve.go): started after the processes are up; stopped FIRST
    in performGracefulShutdown, before php-fpm is drained, because it rewrites and
    reloads php-fpm config.
  • No hand-rolled lock/repair: serve.New already takes the state-file lock and
    self-repairs internally, so init adds none. Corollary: do not also run a
    standalone fpm-tune against the same pools — the second copy refuses to start.
  • Config: fpm_tune block with defaults and validation on both the fail-fast
    (validateFPMTune) and check-config (validateGlobalFPMTuneSettings) paths.
    Strict-decode safe (guarded by knownfields_test.go).
  • Metrics: fpm-tune uses its own Prometheus registry, so its /metrics runs on
    its own port (metrics_addr), not init's :9090.
  • Pins github.com/cboxdk/fpm-tune v0.1.0-beta.20; all 63 deps permissive, no vulns.

Gate

fmt ✓ vet ✓ lint (0 issues) ✓ test -race ✓ vulncheck (0) ✓ license-check (63 permissive) ✓ check-configs (15/15) ✓ — SBOM regenerated (additions only).

End-to-end (real container)

Ran cbox-init as PID 1 in a php:8.4-fpm container with fpm_tune: apply:

Step Evidence
init supervises php-fpm php-fpm pid 15; Runtime PHP-FPM autotuner started mode=apply interval=5s
discovers + enables status Enabled the status page on pools that lacked one pools=[www] → wrote zz-fpm-tune-status.conf
scrapes + sizes + serves metrics /metrics shows fpm_tune_pool_*{pool="www"}
resize under load (40s, 15 concurrent) recommended 5→10, configured 5→7; wrote zz-fpm-tune.conf pm.max_children = 7; Pool resized pool=www from=5 to=7
reload, not restart master pid still 15 across the resize (using inherited socket fd=9)
graceful shutdown loop stopped before php-fpm drained; exit 0

Not in this cut

  • The boot calculator (internal/autotune/calculator.go, profiles.go) stays as
    the seed — not replaced.
  • The Percona/Valkey engine tuner is untouched.
  • The two metrics endpoints are not merged (fpm-tune keeps its own port).
  • No DetectBudget adapter — fpm-tune reads the master's cgroup, which in a
    container is init's budget.

cbox-init sized php-fpm only once, at boot, from a static profile, and never
observed the running master or resized it as load changed. This adds a built-in
runtime autotuner: a `global.fpm_tune` block embeds cboxdk/fpm-tune's serve loop,
which measures live per-worker memory (PSS), and in apply mode rewrites a pool
drop-in and reloads php-fpm with SIGUSR2 (never a restart), validating each change
against a throwaway copy and rolling back if the master does not come back.

The boot-time calculator is kept as the seed: it gives php-fpm a sane
pm.max_children before it starts (which the loop cannot, needing a running
master), and the loop owns the number from there.

- cmd/cbox-init/fpmtune.go: startFPMTune builds serve.Config (apply by default,
  p95 hybrid sizing) and runs serve.Loop.Run in a goroutine, returning a stop
  closure that halts the loop and releases its state lock.
- cmd/cbox-init/serve.go: starts it after the processes are up, and stops it FIRST
  in performGracefulShutdown, before php-fpm is drained, since it rewrites and
  reloads php-fpm config.
- config: fpm_tune block with defaults (setGlobalFPMTuneDefaults) and validation on
  both the fail-fast (validateFPMTune) and check-config (validateGlobalFPMTuneSettings)
  paths.
- serve.New already takes the state-file lock and self-repairs, so no lock is added
  here; do not also run a standalone fpm-tune against the same pools.
- example config, docs, and regenerated SBOM; pins github.com/cboxdk/fpm-tune
  v0.1.0-beta.20.
Runs cbox-init as PID 1 in a php:8.4-fpm container, supervising real php-fpm with
the embedded fpm-tune loop in apply mode, and asserts the full chain end to end:
the loop discovers www, enables its status page, sizes it, applies a resize under
load, and reloads php-fpm with SIGUSR2 (the master pid is unchanged, so it is a
reload, not a restart), then shuts down cleanly.

The driver is external (docker exec / curl from the host) because php-fpm is a
longrun, so the oneshot-verify pattern (Dockerfile.pid1) does not fit. libfcgi-bin
and busy.php are baked into the image so the load needs no network. The container
runs memory-bounded (--memory=512m) for a predictable budget.

- tests/integration/Dockerfile.fpmtune, fpm-tune-config.yaml, e2e-fpm-tune.sh
- make test-e2e-fpm-tune (depends on build-all)
- CI: a debian-gated step in the integration-test job, reusing the built binary
Backward compatible: serve.New/Run/Close are unchanged and serve.Config only
gained optional fields. Wire init's build version into the loop's new
Config.Version, which the daemon reports on its /history.json.

beta.25's CPU-measurement feature reaches gopsutil, which transitively pulls
github.com/shoenig/go-m1cpu (MPL-2.0) — a darwin/arm64-only Apple Silicon CPU
helper (//go:build darwin && arm64 && cgo) that is never compiled into the Linux
production binary. cyclonedx lists the whole module graph regardless, so add a
documented license-check exception for it: file-level weak copyleft, not
modified, not shipped in prod. (The dep was already present at beta.20; the
earlier license-check only passed because it ran against a pre-regeneration SBOM.)
Regenerate the SBOM.
@sylvesterdamgaard
sylvesterdamgaard merged commit dbd3315 into main Sep 4, 2026
9 checks passed
@sylvesterdamgaard
sylvesterdamgaard deleted the feat/embed-fpm-tune-runtime-autotuner branch September 4, 2026 12:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant