What an idle Campfire spends: job polling, checkpoint retries and a one-worker cluster

[namespaceMarcello]

We measured what a running Campfire spends with nobody using it: the production image of main at c8f74a0, on our machine (laptop, Docker Desktop), with Puma and the Resque pool sized as shipped from the CPUs the container sees, after ~25 s of pages and posts.

CPUs	processes	memory (sum of PSS)	CPU with nobody connected
1	8	477–509 MB	6.5–7.0 ms per second
4	11	874–875 MB	17.9–18.2 ms per second
16	25	2,478 MB	61 ms per second

Two changes from this went straight to PRs: #384 (count workers from the CPUs a container may use, which matters under ONCE's CPU limit) and #385 (Process.warmup before Puma forks). Three more change a default or a recent design choice, so they are here first, each with what it saved on the same machine and what it costs. Each was one run against two runs of main.

1. Job workers poll Redis every 0.1 s

The Procfile starts the pool with INTERVAL=0.1, and an idle worker makes about four Redis round trips per poll (the pause flag and the queue pop among them), ten times a second. At 16 CPUs that is 8 workers and 317 Redis commands per second, and 49 of the 61 ms above: 34 in the workers' loops, 15 in Redis.

With INTERVAL=1, at 4 CPUs: 18.0 → 6.4 ms per second, Redis 79 → 8 commands per second, wakeups 180 → 52 per second.

Cost: a job can wait up to a second longer before a worker takes it (unread notices, web push, bot webhooks), half a second on average. #346 keeps a 0.1 s polling interval for its Solid Queue workers, so the same choice would carry over there.

2. Every worker retries the WAL checkpoint lock every 0.25 s

SqliteWalCheckpoint runs a thread in each Puma and Resque worker. The one holding the flock checkpoints every 0.25 s; each of the others opens the lock file, tries the flock and closes it every 0.25 s, which only serves to notice when the holder is gone. At 16 CPUs those are 19 threads, 76 wakeups and 10.7 ms per second.

With the others retrying every 5 s, at 4 CPUs: 18.0 → 17.3 ms per second, 10 fewer wakeups per second.

Cost: when the holder exits, another worker takes over up to 5 s later instead of 0.25 s, and the WAL grows meanwhile, since autocheckpoint is off.

3. One CPU runs Puma as a cluster of one

At 1 CPU worker_count is 1, and workers 1 puts Puma in cluster mode: a master and one worker both hold the app, and Puma warns about it at boot ("Detected running cluster mode with 1 worker"). The same container in single mode (WEB_CONCURRENCY=0) used 418 MB against 477–509 MB.

Cost: in single mode, if the web process dies the whole app stops with it (bin/boot ends every process when one exits) and waits for the container to be restarted, where today Puma's master starts a new worker. The comment in puma.rb already covers single mode for the checkpointer.

Would any of these be welcome as a PR? Each is a change of a line or two, and we would measure it again on the branch.

Written by Claude Code (AI assistant) on behalf of @namespaceMarcello, who directs this work.