Cloud Run's default startup probe is a TCP check on the container port. gunicorn's master binds that port immediately, while the app import runs post-fork (--preload is deliberately unset) and takes minutes. Cloud Run therefore marks a booting instance "started" and routes live traffic onto it, where requests queue until the 300s request timeout.
This is the mechanism behind the "no available instance" 500s and the 504s observed on every scale-out burst during the App Engine → Cloud Run migration ramp, including the user-visible incidents on 2026-07-21.
Fix
Pin an HTTP startup probe on /readiness-check so Cloud Run withholds traffic until the app can actually serve.
Sizing is constrained by measured boot-to-ready times (48 boots over 7 days): p50 201s, p90 371s, p95 417s, max 503s. Cloud Run caps failureThreshold × periodSeconds at 240s and initialDelaySeconds at 240s, shutting the container down past their sum. The threshold/period half is already at its ceiling, so initialDelaySeconds is the only way to widen the window — it is additive, but also delays availability for instances that boot faster than it.
Shipping initialDelaySeconds 180 + 24 × 10s = 420s, which kills ~6% of boots versus ~12.5% at 120 and ~23% at the un-extended 240s.
Accepted residual risk
~6% of boots still exceed 420s and will be killed and retried, which can fail a deploy. Accepted deliberately: the alternative (TCP) serves real users 5xx from instances that were never ready, and a failed deploy is recoverable on retry.
Real fix
~82% of boot is constructing the US tax-benefit system (policyengine_api.country builds all five countries at import; US alone is ~90% of that). At a fast boot this probe would need no initialDelaySeconds at all and would cover 100% of boots. Tracked separately.
Cloud Run's default startup probe is a TCP check on the container port. gunicorn's master binds that port immediately, while the app import runs post-fork (
--preloadis deliberately unset) and takes minutes. Cloud Run therefore marks a booting instance "started" and routes live traffic onto it, where requests queue until the 300s request timeout.This is the mechanism behind the "no available instance" 500s and the 504s observed on every scale-out burst during the App Engine → Cloud Run migration ramp, including the user-visible incidents on 2026-07-21.
Fix
Pin an HTTP startup probe on
/readiness-checkso Cloud Run withholds traffic until the app can actually serve.Sizing is constrained by measured boot-to-ready times (48 boots over 7 days): p50 201s, p90 371s, p95 417s, max 503s. Cloud Run caps
failureThreshold × periodSecondsat 240s andinitialDelaySecondsat 240s, shutting the container down past their sum. The threshold/period half is already at its ceiling, soinitialDelaySecondsis the only way to widen the window — it is additive, but also delays availability for instances that boot faster than it.Shipping
initialDelaySeconds 180 + 24 × 10s = 420s, which kills ~6% of boots versus ~12.5% at 120 and ~23% at the un-extended 240s.Accepted residual risk
~6% of boots still exceed 420s and will be killed and retried, which can fail a deploy. Accepted deliberately: the alternative (TCP) serves real users 5xx from instances that were never ready, and a failed deploy is recoverable on retry.
Real fix
~82% of boot is constructing the US tax-benefit system (
policyengine_api.countrybuilds all five countries at import; US alone is ~90% of that). At a fast boot this probe would need noinitialDelaySecondsat all and would cover 100% of boots. Tracked separately.