Skip to content

Architecture

sarmakska edited this page May 31, 2026 · 2 revisions

Architecture

Single Express app, split into focused modules. No database. The only durable state is the dead-letter JSON Lines file. Trivially deployable as one container.

Module map

Module Responsibility
src/index.js Entrypoint. Reads env, wires the real Resend client, builds the queue and dead-letter inbox, starts the server, handles graceful shutdown
src/app.js Express app factory with injectable dependencies, route handlers, template dispatch and the default formatter
src/verify.js Per-provider HMAC verification (generic, GitHub, Cal.com, Linear, Stripe with timestamp tolerance)
src/render.js Dependency-free Markdown to HTML and plain-text rendering, and template result normalisation
src/queue.js In-memory retry queue with exponential backoff and full jitter
src/deadletter.js Dead-letter inbox: JSONL persistence plus a bounded in-memory ring
src/notify.js Delivery channels: email (Resend), Slack Block Kit, Telegram
src/templates/<source>.js Per-source formatters returning Markdown

Flow

graph TD
  EXT[Webhook source<br/>Stripe / GitHub / Cal.com / Linear] -->|POST JSON| API[Express /hooks/:source]
  API -->|per-provider HMAC| SIG{Signature ok?}
  SIG -->|no| REJ[401]
  SIG -->|yes| TPL{Template exists?}
  TPL -->|src/templates/source.js| FMT[Markdown formatter]
  TPL -->|no template| DEF[Default JSON formatter]
  FMT --> RND[Render Markdown to HTML + text]
  DEF --> RND
  RND --> Q[Enqueue, return 202]
  Q --> W[Worker: deliver with backoff]
  W -->|ok| EM[Resend]
  W -->|ok| FAN[Slack + Telegram best effort]
  W -->|attempts exhausted| DL[Dead-letter inbox]
Loading

Request sequence

sequenceDiagram
  participant Ext as Webhook source
  participant API as /hooks/source
  participant Q as Retry queue
  participant R as Resend
  participant DL as Dead-letter inbox

  Ext->>API: POST /hooks/stripe (raw body + Stripe-Signature)
  API->>API: verify per-provider HMAC
  API->>API: format to Markdown, render HTML + text
  API->>Q: enqueue
  API-->>Ext: 202 queued
  Q->>R: send email (retry with exponential backoff)
  alt delivered
    R-->>Q: ok
    Q-)Ext: fan out to Slack and Telegram (best effort)
  else attempts exhausted
    Q->>DL: record failure (browsable at GET /dead-letter)
  end
Loading

Design decisions

Decoupled delivery. The handler enqueues and returns 202 rather than sending inline, so response latency is flat and retries do not hold the source's connection open. A 202 means accepted and queued; durability comes from the dead-letter inbox, not from a synchronous send.

In-memory queue, file-based dead letter. A broker would add durability for in-flight jobs but also an external dependency. The chosen middle ground keeps the queue in memory for transient-outage resilience and persists only the failures that need recovering. A clean shutdown flushes any undelivered jobs to the dead-letter file.

Exponential backoff with full jitter. Backoff doubles each attempt up to RETRY_MAX_DELAY_MS, and full jitter spreads retries within the window so a recovering provider is not hit by a synchronised retry storm.

Per-provider verification. Signing schemes differ. The verifier encodes each provider's scheme behind one function and compares in constant time. Stripe additionally rejects timestamps outside the tolerance window.

Markdown templates. Templates return Markdown rather than hand-written HTML, so a new source is quick to add and consistently styled. The renderer is a small in-repo subset of Markdown, which avoids a dependency and removes any chance of third-party HTML injection. All payload values are escaped.

Best-effort fan-out. Email is the source of truth. Slack and Telegram failures are logged but never fail the job.

Injectable everything. The email sender, the fan-out fetch, the queue timer and the template directory are all injected, so the end-to-end tests run the real app on an ephemeral port and only fake the outermost network edges.

Resource usage

Single-process Node, roughly 50MB RSS at idle. The queue and dead-letter ring are bounded, so memory stays flat under sustained load. The practical bottleneck is the Resend send rate, not this service.

Clone this wiki locally