Skip to content

Correct the deploy runbook against a real first deploy - #168

Merged
rghvgrv merged 1 commit into
mainfrom
docs/keep-warm-target-and-redirect-status
Aug 10, 2026
Merged

Correct the deploy runbook against a real first deploy#168
rghvgrv merged 1 commit into
mainfrom
docs/keep-warm-target-and-redirect-status

Conversation

@rghvgrv

@rghvgrv rghvgrv commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Three corrections, all found by working through the runbook against real Render and Neon rather than by reading it.

1. Keep-warm ping → /health/live, not the root path

Root 404s. cron-job.org counts any non-2xx as a failed execution: it alerts every five minutes and disables jobs that keep failing. A keep-warm ping that switches itself off is the worst failure mode available here — the app silently returns to 40–60s cold starts and nothing says so.

/health/live is registered with Predicate = _ => false, so it runs no checks at all and still never wakes Neon's compute — which was the entire reason the doc steered away from /health. With a 200 expected, failure notifications become a real uptime alert instead of noise worth muting.

The doc already conceded this in a parenthetical and chose root anyway as "one less thing to get wrong". It was the more things to get wrong.

2. The HTTP→HTTPS check expects the wrong status

Documented as 307. Actual, against the deployed service:

$ curl -i http://subvora-api.onrender.com/health
HTTP/1.1 301 Moved Permanently
Location: https://subvora-api.onrender.com/health
Server: cloudflare

No x-render-origin-server: Kestrel, no rndr-id — headers present on every real response from the container. The redirect is issued by Cloudflare at Render's edge and never reaches the app, so UseHttpsRedirection's 307 only appears off-Render.

Both docs now state what the check is actually for: one hop, no loop. The loop is the failure it catches, and that still means UseForwardedHeaders is missing or mis-ordered.

3. GO_LIVE_CHECKLIST.md never mentioned the Render deploy hook

DEPLOYMENT.md §4 covers it. The tick-box version does not — so anyone working the checklist finishes with autoDeploy: false and no RENDER_DEPLOY_HOOK_URL, and every push to main migrates the database and then fails on an empty URL. Schema ahead of code, one red check mark to show for it.

This is not hypothetical — it is what happened on this deploy, and it was only caught by inspecting gh secret list. Added as a step-5 box with the gh secret set command and why the URL is a secret rather than a plain setting.

Also added to the checklist

  • A redeploy box for when the service first booted before the migration finished. SubscriptionCatalogSyncService runs only at boot and never retries, so the symptom is an empty provider catalog behind a perfectly healthy /health — silent, and confusing to diagnose later.
  • A /subscriptions/resolve check, which proves migration-before-boot in a single call. Each field pins a different half:
tier         : AutoFill        <- pg_trgm is live and scoring
providerName : Netflix         <- the catalog sync inserted providers
categoryId   : <non-null>      <- system categories seeded, sync resolved one by name

Faster and more specific than "add a subscription, then check the burn-rate breakdown", which is kept as the end-to-end confirmation.

  • The second-registration path noted as a free SMTP check — it emails the existing owner while returning an identical 202, so it confirms Brevo and the anti-enumeration behaviour at once.

Code change

One comment in Program.cs, at the /health/live registration, so the reason the cron points there lives next to the endpoint rather than only in the runbook. Build verified: 0 warnings, 0 errors. No behaviour change.

Three things the runbook got wrong, all found by following it end to end
against Render and Neon rather than by reading it.

The keep-warm ping now targets /health/live instead of the root path. Root
404s, and cron-job.org counts any non-2xx as a failed execution: it alerts
every five minutes and disables jobs that keep failing. A keep-warm ping that
switches itself off is the worst possible failure here, because the app
silently returns to 40-60s cold starts with nothing to say so. /health/live
answers 200 and is registered with Predicate = _ => false, so it runs no checks
at all and still never wakes Neon's compute - which was the whole reason the
doc steered away from /health. It already conceded this in a parenthetical and
picked root anyway as "one less thing to get wrong"; it was the more things to
get wrong.

The HTTP-to-HTTPS check expected a 307. On Render the redirect is a 301 issued
by Cloudflare at the edge and never reaches the app - the response carries
neither x-render-origin-server nor rndr-id. UseHttpsRedirection's 307 only
appears off-Render. Both docs now say what the check is actually for: one hop,
no loop.

GO_LIVE_CHECKLIST never mentioned the Render deploy hook at all. DEPLOYMENT.md
covers it, but anyone working the tick-box version ends up with autoDeploy off
and no RENDER_DEPLOY_HOOK_URL - so every push to main migrates the database and
then fails on an empty URL, leaving the schema ahead of the code with only a
red check mark to show for it. Added, with the gh command and why it is a
secret.

Also added to the checklist: a redeploy box for the case where the service
booted before the migration finished (SubscriptionCatalogSyncService runs only
at boot and never retries, so the symptom is an empty catalog behind a healthy
/health), and a /subscriptions/resolve check that proves migration-before-boot
in one call - tier, providerName and a non-null categoryId each pin a different
half of it. The second-registration path is noted as a free SMTP check, since
it emails the existing owner while returning an identical 202.
@rghvgrv rghvgrv self-assigned this Aug 10, 2026
@rghvgrv
rghvgrv merged commit 17f5eeb into main Aug 10, 2026
3 of 4 checks passed
@rghvgrv
rghvgrv deleted the docs/keep-warm-target-and-redirect-status branch August 10, 2026 21:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant