-
Notifications
You must be signed in to change notification settings - Fork 2
0.36.0
Your 3 a.m. cron failed. Who found out? Before this release: nobody, until you happened to go look. Now the runtime can push failures to you.
Upgrade impact: none (additive), fully opt-in. No supervisor configured → nothing changes.
⚠️ Use 0.36.1, not 0.36.0. 0.36.0 shipped with a delivery bug: a handler using# Output: agent: ${SUPERVISOR_AGENT}(the example pattern below) substituted the message body but not the delivery target, so alerts were addressed to a phantom agent and reached nobody — failures were detected and the handler ran, but the notification went nowhere. Fixed in 0.36.1. If you're on 0.36.0, upgrade.
When you run a skill interactively, you see the result — if it failed, you know. But a cron- or event-fired skill has no caller watching it. It runs at 3 a.m., it errors (or gets deadline-cut leaving a mutation half-done), and the only record is a trace file nobody's reading. The failures that most need attention — the unattended ones — were the least visible.
Point one env var at a handler skill:
SKILLSCRIPT_SUPERVISOR_SKILL=supervisor-notifyNow, on every scheduler tick, after firing due triggers, the runtime scans the trace for non-clean runs since the last sweep — errored, deadline-cut, or "left an uncertain effect" — and routes each to your handler skill. That's it: failures push to you instead of waiting to be found.
The sweeper is the scheduler itself, not a new daemon. If it's alive to fire your crons, it's alive to notice they failed — no separate thing to keep running.
The runtime detects reliably; how you get notified is your script. Copy the example (examples/skillscripts/supervisor-notify.skill.md), adapt the delivery to your channel, and approve it:
# Skill: supervisor-notify
# Autonomous: true
# Vars: FAILED_SKILL=?, OUTCOME=?, TRACE_ID=?, ERROR_SUMMARY=, SUPERVISOR_AGENT=
# Output: agent: ${SUPERVISOR_AGENT}
⚠ ${FAILED_SKILL} failed (${OUTCOME}). ${ERROR_SUMMARY}
Trace: ${TRACE_ID}
Swap the # Output: agent: line for a $ email.send or shell(curl ...) and you've got email or a webhook — the runtime doesn't care. The handler receives the failure as vars: FAILED_SKILL, OUTCOME (errored / deadline-exceeded / uncertain-effects), TRACE_ID, TRIGGER, ERROR_SUMMARY, UNCERTAIN_EFFECTS, DETECTED_AT_MS, RAN_AS, and SUPERVISOR_AGENT (optional — only if you route to an agent).
The skills list gained a Health (24h) column: a skill that errored shows ⚠ N failed; one that left a mutation in doubt shows the loudest badge, ⚠ uncertain effect. Click in and the skill detail breaks out deadline-cut and uncertain-effect counts, and Recent runs shows each failure's error. The "a mutation may have landed" signal — previously buried in a trace — is now visible without digging.
-
A configured supervisor requires tracing on — or the runtime won't start. The sweeper reads the durable trace, so it's blind if scheduler tracing is off or sampled. If you set
SKILLSCRIPT_SUPERVISOR_SKILLbut tracing isn't on, the runtime hard-refuses at boot with a clear message. A safety net you've silently defeated is worse than none. (No supervisor set → feature just off, boots normally.) -
uncertain_effectsmeans reconcile, not retry. When a deadline cuts asendor awritemid-flight, the runtime can't know if it landed — so the handler tells you to go check, and never auto-retries (retrying a maybe-sent email is how you double-send). - The handler can't page you about itself. If your handler skill fails, the runtime logs it locally and stops — it does not route the handler's own failure back through the handler (no "the notifier failed" loop). Keep the handler simple and its delivery reliable.
- One alert per failed run. A persistently-failing every-minute cron alerts once per run, not once per sweep. Digest/severity logic lives in your handler skill, where you can shape it.
-
SUPERVISOR_AGENTis optional. An email- or webhook-only handler needs no agent. Set it only if your handler routes to one.