Skip to content

ops: external uptime probe via GitHub Actions cron (#665)#674

Merged
lilyshen0722 merged 1 commit into
mainfrom
ops/uptime-check
Jul 11, 2026
Merged

ops: external uptime probe via GitHub Actions cron (#665)#674
lilyshen0722 merged 1 commit into
mainfrom
ops/uptime-check

Conversation

@lilyshen0722

Copy link
Copy Markdown
Contributor

Closes the last open half of #665 ("See & Hear" Wave 0).

Sentry (live since yesterday) only reports errors from a running app — a crash-looped backend, dead tunnel, or reclaimed spot node is silent there. This adds an outside-in probe:

  • Cron every 15 min (:07/:22/:37/:52) + manual dispatch, probing commonly.me and api.commonly.me/api/health.
  • 3 attempts spaced 50s (~100s+ window) so the ~90s Deploy Dev Recreate blip (backend Deployment uses Recreate + 1 replica — every deploy is a ~90s API outage #654) doesn't page.
  • Sustained failure → opens an issue labeled uptime (→ email); dedupes against an already-open one; auto-closes with a recovery comment so each outage's duration is recorded in one issue timeline.
  • permissions: issues: write only; no secrets.

Deliberate choice over an external SaaS monitor (no new account; lives in infra already watched daily). Known trade: GitHub-cron jitter of a few minutes, and it can't page if GitHub itself is down.

Closes #665

🤖 Generated with Claude Code

https://claude.ai/code/session_01MnRCAFgjrrGZxo9VRCmCm9

Probes commonly.me + api health every 15 min from GitHub's side; 3
attempts / ~100s window sized to outlast the #654 deploy Recreate blip.
Opens a labeled issue on sustained failure (email alert), auto-closes
on recovery so downtime duration lives in the issue timeline. Chosen
over an external SaaS monitor deliberately — no new account, lives in
infra we already watch (Sam's call, 2026-07-11).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MnRCAFgjrrGZxo9VRCmCm9
@lilyshen0722
lilyshen0722 merged commit 7947f2d into main Jul 11, 2026
11 checks passed
@lilyshen0722
lilyshen0722 deleted the ops/uptime-check branch July 11, 2026 10:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Ops: external uptime monitoring + DB backup/restore runbook

2 participants