Skip to content

ci(config): add Slack notifications to the config release pipeline - #6436

Merged
Coly010 merged 2 commits into
developfrom
columferry/config-release-slack-notify
Sep 2, 2026
Merged

ci(config): add Slack notifications to the config release pipeline#6436
Coly010 merged 2 commits into
developfrom
columferry/config-release-slack-notify

Conversation

@Coly010

@Coly010 Coly010 commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Reuses the CLI release train's Slack webhook (SLACK_RELEASE_WEBHOOK, the cli-deployer-notifier app) for the @supabase/config release pipeline, covering both halves of the release lifecycle: paging reviewers when a release is awaiting approval, and confirming the publish actually landed.

What changed

  • slack-notify.yml is generalized beyond the CLI: a required package display-name input, tag_prefix so changelog links can point at config-v* tags, an optional npm_package input that adds an npm version-page link to success messages, and two new statuses — awaiting-approval and declined. Unknown statuses now fail the step instead of silently rendering as success.
  • release.yml passes the new required package input at both call sites. CLI notification firing conditions are unchanged; payload deltas are cosmetic copy only.
  • release-config.yml:
    • notify-slack-approval pings the channel when a real (non-dry) release arms the config-release environment gate. It needs only plan, so it runs while the publish job sits in waiting — the message links to the run, where the Approve button and the plan job's evidence summary live. The webhook is one-way, so the approval click itself deliberately stays on GitHub.
    • A new "Verify the release is live on npm" step runs after npm publish and before the tag push: it probes the registry with backoff (~2 min budget, --prefer-online) until the version resolves, then requires the packument's dist.integrity to match the reviewed tarball and the dist-tag to point at the published version. The success notification therefore means registry-visible with the approved bytes, not merely "the job didn't error". The step runs no package-controlled code, preserving the publish job's id-token: write boundary.
    • notify-slack posts the success message (npm + changelog links) via the implicit success() gate — dry runs and no-release pushes skip publish and stay silent.
    • classify-failure + notify-slack-failure report broken releases. The classifier reads the run's approvals record (GET .../actions/runs/{run_id}/approvals) and classifies on the last review's state, so a reviewer rejection is announced as "release not approved" instead of a broken release, while a reject → re-run → approve → genuine-failure sequence still pages as a real failure. The notify job fails open: if the classifier itself breaks, the page still goes out as a plain failure.
  • packages/config/AGENTS.md documents the webhook secret as standing invariant 5 (notify jobs are terminal — a missing/rotated webhook reddens the run but cannot affect the release) and the verify step in the publish sequence.

Reviewer notes

  • Rejection semantics were verified against GitHub docs: rejecting a pending deployment marks the waiting job failed, so failure() fires and the declined path is reachable; the approvals endpoint returns state ∈ approved|rejected|pending and is readable with the default token plus job-level permissions: actions: read.
  • The Slack payload stays heredoc-built rather than jq --arg: every new input is a workflow-file literal at all five call sites, and a jq --arg migration would re-escape the literal \n sequences the message bodies rely on. Considered and deliberately not done here.
  • There is no automated exercise of these workflows in CI (actionlint config exists but nothing runs it — possible follow-up); this diff was validated with actionlint locally and by executing the notifier script offline for every status/input combination.

Reuses the CLI release train's Slack webhook (cli-deployer-notifier) for
the @supabase/config release pipeline:

- slack-notify.yml generalized beyond the CLI: required package
  display-name input, tag_prefix for config-v changelog links, optional
  npm_package version-page link, and two new statuses
  (awaiting-approval, declined). release.yml call sites updated for the
  new required input; CLI firing conditions unchanged.
- release-config.yml pages the channel when a real release arms the
  config-release approval gate, verifies the published version is
  registry-visible with the reviewed tarball's integrity and expected
  dist-tag before the tag push, and reports the outcome — with a
  fail-open classifier that reads the run's approvals record so a
  reviewer rejection is announced as declined rather than a broken
  release.
@Coly010
Coly010 requested a review from a team as a code owner September 2, 2026 14:26
@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Supabase CLI preview

npx --yes https://pkg.pr.new/supabase/cli/supabase@c7033be22eff249f8305c732fa9dda8514f68f46

Preview package for commit c7033be.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI Review

Both independent reviews completed. Their 7 reports deduplicate to 5 findings, all confirmed by the checked-out code. The most significant issue is that the new dist-tag loop exits on any nonempty version instead of waiting for the expected version. The remaining findings concern misleading failure notifications, publish/tag ordering, and suppressed npm diagnostics. No critical or major issues were verified.

Findings

Severity Location Category Sources Claim
🟡 MINOR .github/workflows/release-config.yml:338 ci-correctness claude The dist-tag retry loop stops on any nonempty value rather than the expected version, so a stale registry response bypasses all remaining retries and fails the already-published release.
🟡 MINOR .github/workflows/release-config.yml:458 notification-correctness claude+codex A rejection from an earlier run attempt can cause a later planning failure to be reported as a reviewer-declined release.
🟡 MINOR .github/workflows/release-config.yml:313 release-consistency claude The new npm verification step can fail after publication but before the version tag is pushed, leaving npm published while origin remains untagged and requiring another approval to recover.
🟡 MINOR .github/workflows/release-config.yml:437 notification-correctness claude+codex A workflow-dispatch dry run can send a real release-failure notification if the plan job fails before its plan step writes the dry_run output.
⚪ NIT .github/workflows/release-config.yml:322 observability claude The npm verification probes discard stderr on every attempt, so terminal failures omit the underlying registry, authentication, or network error.

Stats

Claude findings: 5 · Codex findings: 2 · Confirmed: 5 · Refuted: 0 · Uncertain: 0


Models: claude-opus-5 + gpt-5.6-sol · Trigger: auto · Workflow run

This review runs once per PR. A maintainer can request another with a /ai-review comment.

Comment thread .github/workflows/release-config.yml Outdated
Comment thread .github/workflows/release-config.yml Outdated
Comment thread .github/workflows/release-config.yml Outdated
Comment thread .github/workflows/release-config.yml Outdated
Comment thread .github/workflows/release-config.yml Outdated
@Coly010 Coly010 self-assigned this Sep 2, 2026
- Verify step split: pre-tag-push verification is now integrity-only
  (the tag-push precondition), and the dist-tag assertion moved to its
  own step after the tag push — a dist-tag propagation hiccup can no
  longer strand the release npm-published but origin-untagged, and the
  retry loop now waits for the EXPECTED version instead of breaking on
  a stale nonempty value.
- classify-failure only reports declined when the publish job itself is
  the failed job (a rejection can only manifest there), so a plan
  failure after an earlier attempt's rejection pages as a plain failure.
- Dry-run guards on the failure notify paths (both trains) read the
  workflow_dispatch input directly, so a plan job that dies before
  recording dry_run can't page for an operator-watched dry run.
- npm probes capture stderr and surface it on terminal failures.
@Coly010
Coly010 added this pull request to the merge queue Sep 2, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Sep 2, 2026
@Coly010
Coly010 added this pull request to the merge queue Sep 2, 2026
Merged via the queue into develop with commit 1b482f4 Sep 2, 2026
66 checks passed
@Coly010
Coly010 deleted the columferry/config-release-slack-notify branch September 2, 2026 15:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants