Skip to content

feat(engine): module substrate wave 2 — events, outbox drain, webhook delivery, admin API (HT-69) - #79

Merged
zaridan merged 3 commits into
mainfrom
feat/ht-69-events-webhooks
Jul 19, 2026
Merged

feat(engine): module substrate wave 2 — events, outbox drain, webhook delivery, admin API (HT-69)#79
zaridan merged 3 commits into
mainfrom
feat/ht-69-events-webhooks

Conversation

@zaridan

@zaridan zaridan commented Jul 19, 2026

Copy link
Copy Markdown
Contributor

Summary

Wave 2 of the module substrate (HT-69; spec §4/§5, merged in #76). Rebased onto main after #77 merged (GitHub closed the stacked #78 on base-branch deletion — this supersedes it; identical content, review trail there).

  • Transactional event emission at six points; rollback ⇒ no event (test-forced); thin payloads; soft delete emits nothing ever, including the stranded-outbound delivery-race guard.
  • Outbox drain on the queue/cron pattern; fan-out dedupe key eventId:endpointId.
  • Webhook delivery: Stripe-shape HMAC, 10s timeout, no redirects, SSRF guard with resolved-IP pinning (live-verified), retry/dead-letter, auto-disable at 20.
  • Admin API /api/v1/webhooks (+/test), secret shown once; health webhooks section; outbox-drain cron; runbook Part G.

Review trail

Sonnet-authored → Opus adversarial review FIX-FIRST (1 MAJOR: soft-delete/delivery race; 1 MINOR) → both fixed (see #78 for the full review record). Producer/consumer signature interop verified byte-exact against the draft-assistant module's verifier by running both sides.

Gates at rebase: typecheck 0; full suite ran green pre-rebase (1230/1230) — CI re-verifies on this base.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Added an admin-only Webhooks API (list/create/patch/delete) plus a /test endpoint to enqueue signed webhook delivery jobs.
    • Implemented webhook delivery with HTTPS-only signing, retry with attempt limits, dead-lettering, and SSRF protection.
    • Added scheduled outbox draining to fan out events to webhook delivery jobs.
    • Expanded HT-69 health reporting for auto-disabled webhook endpoints and recent delivery dead-letter growth.
    • Updated inbound ingest/storage to emit transactional outbox events with correct “reopened” behavior.
  • Documentation

    • Updated the deployment/monitoring runbook: cron details, alert codes, structured Vercel logging, and webhook outbox/queue drain documentation.

zaridan and others added 2 commits July 19, 2026 09:01
… delivery, admin API (HT-69)

Implements specs/modules/substrate-v1.md §4/§5 on top of wave 1's schema and
stores (HT-68): transactional event emission at every non-deleted state
change (conversation.created/message_received in the ingestion append path,
status_changed/tags_changed/assignee_changed on the corresponding
ConversationStore writes, reply_sent on the delivery-status→'sent'
transition), a cron-driven outbox drain that fans each event out to its
matching active webhook endpoints via the existing queue, a signed HTTPS
delivery handler with resolve-then-connect SSRF pinning and its own
retry/dead-letter ceiling, and the webhooks admin API (CRUD + test-ping)
mirroring agents.ts's conventions.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…edupe-key doc (HT-69 review)

MAJOR: releaseThreadLease's 'sent' branch emitted conversation.reply_sent
unconditionally, but delivery is not conversation-status-scoped — a thread
claimed/leased before its conversation was soft-deleted could still be
force-delivered afterward and fire the event, breaching spec §4's absolute
"no event of any type fires for a soft-deleted conversation" exclusion.
Gate the outbox append (not the delivery-status write, which must still
proceed — mail already went out) on the conversation's live status, read in
the SAME UPDATE...RETURNING statement via a correlated subquery, mirroring
listAwaitingDrafts' existing status <> 'deleted' carve-out for drafts.

MINOR: migration 023's doc comment said the outbox->queue hand-off is keyed
by dedupe_key = event_id (singular); corrected to the implemented per-(event,
endpoint) fan-out key event_id:endpointId, and dropped outbox-drain.ts's now-
redundant "written before finalized" apology.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jul 19, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 748b1cee-a51f-40fa-a5ec-7f6427374723

📥 Commits

Reviewing files that changed from the base of the PR and between 52ab7db and 2594c44.

📒 Files selected for processing (2)
  • src/mail/ingest.test.ts
  • src/mail/ingest.ts
🚧 Files skipped from review as they are similar to previous changes (2)
  • src/mail/ingest.ts
  • src/mail/ingest.test.ts

📝 Walkthrough

Walkthrough

Adds transactional webhook event emission, an admin webhook API, secure signed delivery with SSRF protection, event-outbox draining into queue jobs, webhook health reporting, cron wiring, and deployment documentation.

Changes

Webhook event and delivery platform

Layer / File(s) Summary
Transactional event emission
src/mail/ingest.ts, src/store/conversations.ts, src/db/migrate.ts, src/mail/ingest.test.ts, src/store/conversations.test.ts
Conversation ingestion and state changes append transactional outbox events, including reopen, reply, status, tag, assignee, and rollback semantics.
Webhook vocabulary and admin API
src/webhooks/event-types.ts, src/api/router.ts, src/api/webhooks.ts, src/api/index.ts, src/api/webhooks.test.ts, src/api/*test.ts
Adds validated event types and authenticated admin routes for listing, creating, updating, deleting, and testing webhook endpoints.
Secure webhook delivery
src/webhooks/delivery.ts, src/webhooks/ssrf.ts, src/webhooks/delivery.test.ts, src/webhooks/ssrf.test.ts
Adds signed HTTPS delivery, DNS address pinning, SSRF filtering, timeout handling, retry limits, dead-lettering, and endpoint state checks.
Outbox drain and application wiring
src/webhooks/outbox-drain.ts, src/composition/app.ts, src/composition/root.ts, src/composition/app.test.ts, vercel.json
Claims outbox events, fans them out to active matching endpoints, enqueues delivery jobs, exposes a protected cron route, and schedules it every minute.
Webhook health reporting
src/composition/health.ts, src/composition/health.test.ts, specs/deploy/gmail-inbound-runbook.md
Reports auto-disabled endpoints and recent webhook dead-letter growth, with corresponding tests, alert codes, logging details, and deployment documentation.

Estimated code review effort: 5 (Critical) | ~120 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 67.74% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main additions: transactional events, outbox draining, webhook delivery, and the admin API.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/ht-69-events-webhooks

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/mail/ingest.ts`:
- Around line 714-721: Update the appendThreadInTx success branch to emit the
conversation.message_received outbox event only when appended.created is true.
Preserve the existing return behavior for all successful appends, including
replayed results with created false.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 8228189c-b2b9-4fc7-81e5-e45263eecb57

📥 Commits

Reviewing files that changed from the base of the PR and between 1d09631 and 52ab7db.

📒 Files selected for processing (26)
  • specs/deploy/gmail-inbound-runbook.md
  • src/api/agents.test.ts
  • src/api/index.test.ts
  • src/api/index.ts
  • src/api/router.ts
  • src/api/webhooks.test.ts
  • src/api/webhooks.ts
  • src/composition/app.test.ts
  • src/composition/app.ts
  • src/composition/health.test.ts
  • src/composition/health.ts
  • src/composition/root.ts
  • src/db/migrate.ts
  • src/mail/ingest.test.ts
  • src/mail/ingest.ts
  • src/store/conversations.test.ts
  • src/store/conversations.ts
  • src/webhooks/delivery.test.ts
  • src/webhooks/delivery.ts
  • src/webhooks/event-types.test.ts
  • src/webhooks/event-types.ts
  • src/webhooks/outbox-drain.test.ts
  • src/webhooks/outbox-drain.ts
  • src/webhooks/ssrf.test.ts
  • src/webhooks/ssrf.ts
  • vercel.json

Comment thread src/mail/ingest.ts
…CodeRabbit, HT-69)

appendThreadInTx's AppendResult.ok:true does not guarantee a row was
inserted — created:false is a replay (the get-or-insert found a
pre-existing row). The ingestion append path fired
conversation.message_received unconditionally on appended.ok, which would
re-emit the event with a fresh eventId on a replay, defeating consumer
eventId dedupe (spec §4).

Verified empirically before fixing: at this call site `created` is always
true today (inbound threads never carry an idempotencyKey — migration 003's
CHECK forbids it — and InboundDeliveryStore's own ledger dedup already
intercepts a genuine redelivery of the same raw message one layer up,
never re-invoking this code at all), so this closes a latent type-level
gap rather than an actively exploitable double-fire. Checked
conversation.created's two call sites for the same exposure:
createConversationInTx has no replay concept at all (always a fresh
INSERT, no created flag), so no equivalent gate applies there.

Added a regression test: redeliver the identical raw message
(mailboxId+providerMessageId) and assert exactly one
conversation.message_received (and one conversation.created) in the
outbox.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@zaridan
zaridan merged commit bcf8dba into main Jul 19, 2026
5 checks passed
@zaridan
zaridan deleted the feat/ht-69-events-webhooks branch July 19, 2026 16:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant