Horizontal autoscaling, opt-in per deployment

[Shine-neko]

Ring can now pick a deployment's instance count from its observed CPU, instead of holding the number in the manifest.

replicas: 2          # starting count
autoscale:
  min: 2
  max: 10
  target_cpu: 70     # aim for 70% CPU per instance

Opt-in, and that is the point. A deployment without an autoscale block never has its count changed by Ring. That is what lets an external controller keep owning the count for its own deployments while Ring autoscales only what was handed to it explicitly.

Two columns, not one

replicas keeps meaning "what the manifest declared" and is never written by the scheduler; the decision lives in desired_replicas next to it. So re-running ring apply does not fight the autoscaler, and both numbers stay readable:

$ ring deployment inspect <id>
Replicas      : 6 (declared 2, autoscale 2-10 targeting 70% CPU)

$ ring deployment list
Replicas
3/6*                 # the * marks an autoscaled deployment

A rolling update carries the current capacity to the new deployment, so a redeploy under load does not drop back to the starting count.

How it decides

One instance at a time; a ±10 point dead band around the target; 60s before adding, 300s before removing. The asymmetry is deliberate — shedding capacity slowly is what stops a load oscillating around the target from driving the count up and down with it. Missing or stale (>120s) metrics mean no decision at all, never a guess.

The bug this nearly shipped with

stats_cache reports CPU as the sum across instances — correct for the Prometheus gauge, and the exact opposite of a setpoint. Three instances at 30% report 90%, "above" a 70% target, so the controller would add a fourth, see the sum rise, and climb to max while every instance idled.

Rather than divide at the call site, the cache now exposes cpu_usage_percent_per_instance beside the sum, both documented with the trap. A future controller finds the right value and the warning together.

Rejected combinations

kind: job (no steady-state CPU), host networking with max > 1 (instances would fight over the same ports), and the containerd runtime — it reports CPU as a hardcoded 0, so a CPU target would read "idle" forever and walk the deployment down to min.

Single-node limit, documented not hidden

Autoscaling multiplies instances on one machine: it divides that machine's CPU more finely, it does not add capacity. It absorbs spikes on a host with headroom; it cannot rescue a saturated one. Per-instance memory admission still applies, so a deployment that outgrows the host stops with insufficient_resources rather than taking the machine down.

No CPU admission control was added, deliberately: #110 established that memory is gated because it kills workloads while CPU overcommit only degrades, and that reasoning still holds.

Testing

824 unit tests, clippy clean. Coverage includes the opt-in guarantee at target_replicas(), the summed-CPU regression, anti-flap under an oscillating load, cooldown asymmetry, rollout inheritance, and policy re-clamping.

Reviewed by codex over three rounds. It caught the summed-CPU runaway, the containerd zero metric, a Firecracker burst (it started the whole deficit in one pass), the rollout capacity reset, stale-snapshot decisions, and — last round — that (namespace, name) is only unique among rows with no parent_id, so measurements are now keyed by deployment id and both sides of a rollout are left alone.