email: bounce + unsubscribe webhook ingestion + email_events table#56
Merged
Merged
Conversation
Adds the ingestion surface for provider-side delivery feedback so the worker can stop emailing addresses that bounce and respect users who unsubscribe. Three pieces: 1. Migration 025 creates email_events with a (email, event_type, created_at DESC) composite index for the suppression query and a partial UNIQUE index on (provider, event_type, email, message_id) to dedupe provider redeliveries. 2. POST /api/v1/email/webhook/brevo verifies HMAC-SHA256(BREVO_WEBHOOK_SECRET, body) in constant time, accepting both X-Sib-Signature and the legacy X-Mailin-Custom header. Maps Brevo's event taxonomy (hard_bounce/soft_bounce/unsubscribed/spam/blocked) onto the normalized event_type set. Returns 401 on bad signature, 200 on unhandled events (Brevo also fires opens/clicks we drop silently), 200 on insert success. 3. POST /api/v1/email/webhook/ses verifies SNS TopicArn against SES_SNS_SUBSCRIPTION_ARN. Full SNS signature verification (cert download + RSA verify) is reserved for a follow-up; ARN match stops drive-by traffic. Handles batched recipients (one row per bounced address) and short-circuits Delivery/DeliveryDelay notifications. SubscriptionConfirmation is logged for out-of-band handling — we never auto-confirm. models.HasSuppressionFor splits the query into two index range scans: unsubscribes (no decay) + bounces/spam_complaints (365d decay window). Soft bounces are deliberately omitted from the suppression set. Tests: 10 handler tests (signature paths, event-type mapping, both providers) and 9 model tests (decay window, unsubscribe permanence, message_id dedupe, validation). Full make test-unit green against latest master. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
email_eventstable (migration 025).models.HasSuppressionForfunction returns whether an address has a recent bounce / spam complaint (365d decay) or any unsubscribe (no decay) — the worker forwarder (PR refactor: extract all hardcoded URL/domain literals into internal/urls package #12 in worker repo) calls this before every send.(provider, event_type, email, raw->>'message_id');InsertEmailEventusesON CONFLICT DO NOTHINGso provider redeliveries are silent no-ops.config.Config:BREVO_WEBHOOK_SECRET(HMAC-SHA256 over raw body) andSES_SNS_SUBSCRIPTION_ARN(TopicArn match — full SNS RSA verification reserved for follow-up; the ARN check stops drive-by traffic, the topic ARN is the credential)./api/v1auth group so RequireAuth doesn't demand a Bearer from Brevo's / AWS's servers.Auth shape per provider
hex(HMAC-SHA256(BREVO_WEBHOOK_SECRET, rawBody))— acceptX-Sib-Signatureor legacyX-Mailin-Custom. Constant-time compare. Empty secret → 401 (closed by default).TopicArnmust equalSES_SNS_SUBSCRIPTION_ARN. Constant-time compare. SubscriptionConfirmation logged at INFO — never auto-confirmed.PII
Raw payloads land in
email_events.rawJSONB for audit but are never echoed to user-facing logs. Recipient addresses are stamped on OTel spans only.Test plan
make test-unit— all packages green against latest master (including new migrations 022/023)go vet ./...cleanBREVO_WEBHOOK_SECRET+SES_SNS_SUBSCRIPTION_ARNin k8s secrets before deploy; configure Brevo dashboard webhook URL + matching shared secretPushback / caveats
SigningCertURL, validate cert chain, verify signature) is a follow-up. An attacker who knows our topic ARN can forge inserts. For day-1 ingestion against a private SNS topic this is acceptable; not acceptable long-term.invalid_payload).message-idfield uses a hyphen;injectMessageIDrewrites it tomessage_idbefore insert so the dedupe index fires.🤖 Generated with Claude Code