Skip to content

Ops: email operator error alerts to support@keywind.cc #7

Description

@brocla

Backlog: Observability & ops

Problem

When something goes wrong (systemic job failure, delivery failure, panic, worker-loop death), nobody is notified — an unattended background service fails silently.

Proposed solution

Email operator alerts to support@keywind.cc. Full design already written in ERROR-ALERTING-PLAN.md. Key points:

  • A DI Reporter (no-op default, Mailgun-backed impl reusing the existing send path), following the Engine pattern.
  • Alert only on systemic failures, not expected ones — this requires splitting the two kinds of "job failed" in App.execute (a transformer that cleanly reported success:false on bad input is the sender's problem and is already emailed to them; a crash/timeout/missing-result is systemic). That split is good design on its own.
  • Mandatory throttle/dedupe — the delivery loop ticks every second, so an unthrottled failure = one email/second.
  • Honest gap: "Mailgun send is failing" can't be reported via Mailgun — that's what the heartbeat issue covers.

Phasing

Per the plan: Reporter interface + fake + tests → taxonomy split in execute → Mailgun reporter + config + throttle → wire remaining sites (delivery failure, panic recovery, worker-loop fatal).

Notes

  • Config: alert_recipient: support@keywind.cc in email.yaml (empty = disabled).
  • See ERROR-ALERTING-PLAN.md for the full taxonomy table and open decisions.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions