feat(ops): webhooks, scheduled publishing, caching, and a shared status table - #63
Merged
Conversation
…line The status list and its per-status metadata lived in three hand-synced places: the ops UI registry, a literal options array in the articles collection, and the stage table's entry statuses. AGENTS.md guarded the duplication with a "keep these aligned" convention. Move the table to cms/src/lib/articleStatusMeta.ts (dependency-free, per the brandVoice convention), derive the UI registries and collection options from it, and assert pipeline alignment in a unit test instead of by convention. The table also declares readOnly and pickupStage columns for upcoming gates. Generated payload-types.ts is unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TpSZts1YiNwWeUzisL9Hb2
Article status changes are currently observable only by polling the admin. This adds the delivery half of an events system: a webhook-deliver task on its own webhooks queue (4 retries, 3s timeout) that POSTs a SHA256-HMAC-signed body, and a webhook-settings global resolved admin -> env -> disabled like the Models global. Nothing emits yet; the emitting hook lands separately. The timestamp is part of the signed material so captured deliveries cannot be replayed with fresh headers. Settings resolve at delivery time, not enqueue time, so the kill switch also silences queued jobs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TpSZts1YiNwWeUzisL9Hb2
An afterChange hook beside auditArticleChange queues a webhook-deliver job whenever an article's status changes (creates included, as a null -> status transition). The body carries id, slug, from/to, actor, and the pipeline run id when the update supplied audit context; it is derived from doc/previousDoc because plain admin edits never seed the audit context. Emission failures log and never fail the save, the same trade the pipeline's StageOutcome.warnings makes. Verified end to end: article create -> queued job -> payload jobs:run --queue webhooks -> signed POST whose HMAC verifies against the shared secret. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TpSZts1YiNwWeUzisL9Hb2
…publish The published-article page queried Postgres on every request. It is now ISR (revalidate 300 with an empty generateStaticParams so nothing needs a database at build time): each article renders once, then serves from cache, verified with x-nextjs-cache HIT against a production build. Invalidation is per path, which for this route means per article, so draft churn never touches reader-facing cache. Publishing purges both the slug and id paths immediately; the worker-side purge path (pipeline and scheduled publishes have no Next request context) arrives with the revalidate webhook consumer. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TpSZts1YiNwWeUzisL9Hb2
…ish events Worker-side publishes (pipeline runs and, next, scheduled publishing) happen outside a Next request context and cannot call revalidatePath. The webhook system is the bridge: POST /hooks/revalidate verifies the delivery HMAC and timestamp freshness, then purges the article's paths for transitions into or out of published. Everything else is acknowledged and ignored, keeping draft churn away from reader cache. Lives under /hooks because /api is Payload's REST namespace. Verified end to end against a production build: warm cache HIT, a republish through payload jobs:run --queue webhooks, then the new title served and re-cached. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TpSZts1YiNwWeUzisL9Hb2
…Client createAhrefsClient() read the module-global config.mockMode, so a queued run's Ahrefs mode could disagree with the mode on its pipeline-runs row. The factory now takes the mode explicitly: the job passes run.mode, the CLI passes its computed mode, and admin topic discovery passes the ambient config mode it shows the operator. A live request without a key still degrades to mock, now with a logged warning instead of silently. Verified with a mock-mode CLI fetch (canned SERP data, no outbound calls) and a factory unit test covering all three branches. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TpSZts1YiNwWeUzisL9Hb2
Approved articles gain a sidebar publishAt date; a publish-due task (own scheduled queue, cron every 5 minutes via Payload task scheduling) moves due articles to published through the normal update path, so gates, the audit row, the status webhook, and the cache purge all fire exactly as for a manual publish. Selection is purely by state (approved + due + not archived), so reruns converge and a stray date on a reviewed-back article does nothing. Not on the content queue: publish latency must not wait on a pipeline run's concurrency lock. Verified live: handleSchedules queued the job at the next 5-minute boundary and jobs:run --queue scheduled published the article. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TpSZts1YiNwWeUzisL9Hb2
The reports page showed dollars by stage and model but not how often each stage ran, what it read and wrote, or whether runs themselves are healthy. A pure aggregator (lib/opsKpis.ts, unit-tested without a database) now feeds two additions: a per-stage table of calls, tokens, and cost under the spend panel, and a pipeline-runs panel with succeeded/failed/active counts plus up to five recent failures and their already-redacted error summaries. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TpSZts1YiNwWeUzisL9Hb2
…olds an article Phase 1's status table declared readOnly on drafted and qa_passed; this is the gate that enforces it, Hygraph's read-only workflow step. While a run owns the article, human edits to the scored content fields are rejected with a 400 naming the blocked fields, instead of being silently overwritten by the next stage or invalidating paid-for work. Pipeline and system updates pass (the machine owning the article is the point), and so do status transitions, so send-back and reset actions keep working with their notes and QA payloads. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TpSZts1YiNwWeUzisL9Hb2
Datum had no api-limits-style page: queue names, the scheduler invocations production needs, the webhook signing contract, the revalidate endpoint, scheduled-publish semantics, and fixed limits were only discoverable in code. docs/operations.md collects them, written after the features so it documents what actually shipped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TpSZts1YiNwWeUzisL9Hb2
…atuses gateReadOnlyStatus exempted any status change, so a single update could smuggle content edits through a jump between machine statuses (drafted -> qa_passed skips QA entirely, and scoring picks articles up by status alone). The exemption now requires the target status to be non-readOnly, matching the documented intent: a person pulling the article out of the machine state. Found by the review pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TpSZts1YiNwWeUzisL9Hb2
snowopsdev
marked this pull request as ready for review
August 31, 2026 19:49
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Owner
Author
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b83de3a1f2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Three fixes from review: - publish-due now repeats its eligibility conditions in the update's own where clause instead of writing unconditionally by id, so an article a reviewer moves off approved (or archives) mid-batch is not published over their decision. - Both job-task migrations' down() delete rows carrying the removed task slug before narrowing the enum; rolling back with queued or executed jobs present used to abort on the enum cast. Verified by running migrate:down and migrate against a database holding publish-due rows. - Status events carry previousSlug, and the revalidate consumer purges it, so an unpublish that renames the slug in the same save cannot leave the old slug's cached page serving withdrawn content. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TpSZts1YiNwWeUzisL9Hb2
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements the prioritized "worth borrowing" list from the Hygraph competitive research (
.omc/research/hygraph-competitive-research.md): the operational ergonomics Datum lacked, without touching the review-gate guarantees it already has. One commit per phase, each independently verified.What changes
cms/src/lib/articleStatusMeta.ts). The status list and its per-status metadata lived in three hand-synced places; the ops UI, the collection options, and (via a unit test) the pipeline stage table now all derive from one table. The AGENTS.md "keep these aligned" convention is replaced bypipeline/test/statusAlignment.test.ts.webhook-delivertask (ownwebhooksqueue, 3s timeout, 4 retries, SHA256-HMAC overtimestamp.body) plus anafterChangeemitter; endpoint and secret come from a Webhooks admin global withWEBHOOK_URL/WEBHOOK_SECRETfallback and a kill switch. Nothing was observable without polling before./articles/[slug]becomes ISR (300s); publishing purges per article. Worker-side publishes purge through a newPOST /hooks/revalidateconsumer that verifies the delivery signature — draft churn never touches reader cache.publishAton approved articles; a cron-scheduledpublish-duetask publishes due articles through the normal update path, so gates, audit, webhook, and purge all fire./admin/ops/reports, fed by a pure aggregator (cms/src/lib/opsKpis.ts).createAhrefsClient(mode)mirrorscreateLlmClient(mode); a queued run's mode now comes from itspipeline-runsrow, and live-without-key degrades loudly.gateReadOnlyStatusrejects human edits to scored content whiledrafted/qa_passedhold an article (Hygraph's read-only step), with pipeline/system writes and status transitions exempt.docs/operations.md: queues and the two new production scheduler invocations, the webhook signing contract, the revalidate endpoint, and fixed limits.Deliberately deferred (reasons in the research doc / plan): roles and conditional permissions, an MCP server, a releases collection, a multi-provider remote-source registry.
Deployment notes for the operator
payload jobs:run --queue webhooksandpayload jobs:run --queue scheduled --handle-schedules --limit 1(details indocs/operations.md).webhook_settings_and_delivery_task,scheduled_publishing);payload-types.tsregenerated and committed.Verification
npm run typecheck,npm run lint,npm test: 560 pipeline + 300 CMS integration tests pass, including new specs for the delivery task, event emission, the revalidate route, publish-due, the read-only gate, KPIs, and status alignment.x-nextjs-cache: HIT; a republish driven throughjobs:run --queue webhookspurged and re-cached the page with the new title; the cron scheduler enqueuedpublish-dueat the 5-minute boundary and published a due article; a mock CLI fetch exercised the new Ahrefs seam with zero outbound calls.🤖 Generated with Claude Code
https://claude.ai/code/session_01TpSZts1YiNwWeUzisL9Hb2