feat(seed): fail loudly when a Postgres DSN's user is not the server admin - #129
Merged
Conversation
…admin A DSN whose userinfo disagrees with the server's administrator_login produces FATAL "password authentication failed for user ..." at container start. That message names the password, so it sends you auditing postgres-admin-password — which is fine — while the actual defect is the username. This cost a six-day dev outage (2026-07-15 → 07-21). The environment was rebuilt with the sanitized default `dbadmin`, while postgres-connection-string and paperclip-db-url still carried the pre-v1.4 `vaultadmin` inherited from the platform this repo was extracted from. ca-paperclip-dev and ca-honcho-dev could not reach Postgres the entire time. Both DSNs are `external` secrets — operator-supplied, never generated here — so nothing in the repo could have caught the drift. Now seed-keyvault.sh checks the userinfo of postgres-connection-string and paperclip-db-url against the expected administrator_login, taken from POSTGRES_ADMIN_USERNAME or, failing that, the Terraform variable's own default so the two cannot drift apart. The check covers values kept from a previous run, not just newly provided ones — a stale vault value nobody re-seeds is precisely how this happened. administrator_login is immutable (changing it forces server replacement), so the DSN is always the side that has to move. `--postgres-admin-username` overrides for a server that really does use a different login; an empty value disables it. New `--self-check` runs the guard's cases offline (no vault, no az, no network), wired into the CI scripts job next to the canary self-check. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A DSN whose userinfo disagrees with the server's
administrator_loginfails at container start with:The message names the password, so it points at the wrong secret. The real defect is the username.
This cost a six-day dev outage, 2026-07-15 → 07-21. The environment was rebuilt using the sanitized default
dbadmin, whilepostgres-connection-stringandpaperclip-db-urlstill carried the pre-v1.4vaultadmininherited from the platform this repo was extracted from.ca-paperclip-devandca-honcho-devwere down the whole time; Log Analytics shows the failure starting 2026-07-15 09:50 and running continuously until the secrets were corrected.Both DSNs are
externalsecrets — operator-supplied, never generated by this script — so nothing in the repo could catch the drift.What
seed-keyvault.shnow validates the userinfo of both DSN secrets against the expectedadministrator_login:POSTGRES_ADMIN_USERNAME, else the Terraform variable's owndefault, parsed fromvariables.tfso the two cannot drift.__unset__placeholder and any non-DSN value.--postgres-admin-username NAMEoverrides; empty disables.administrator_loginis immutable (changing it forces Postgres replacement), so the DSN is always the side that must move — the error message says so.Verification
New
--self-checkruns offline (no vault, noaz, no network) and is wired into the CIscriptsjob beside the canary self-check:Also confirmed against the real failure shape —
POSTGRES_CONNECTION_STRING=postgresql+psycopg://vaultadmin:…now aborts the run with an actionable message instead of seeding a DSN that breaks every consumer.shellcheck --severity=errorclean.🤖 Generated with Claude Code