Skip to content

bug: enabled schedule shows "Next: 1d ago" — stale next_run_at, cron fires silently skipped with no execution record #1472

Description

@vybe

Summary

On Agent Detail → Schedules, an Active daily schedule (0 4 * * *, Europe/Kiev) displays "→ Next: 1d ago" with 0 runs recorded (7d success ). A "next run" timestamp in the past on an enabled schedule is a contradiction: at least two cron windows passed with no execution row, no skip record, and no advance of the stored next_run_at. A sibling schedule on the same agent fired successfully inside that same window, so the scheduler container was up and its sync loop was running.

Context

agent_schedules.next_run_at is a stored projection, written at creation (src/backend/db/schedules.py:_calculate_next_run_at), at _add_job time (startup + sync), and on the post-fire paths. Several runtime paths let a cron fire pass without advancing the projection or recording anything, which leaves the UI showing a receding "Next: Xd ago":

  1. Autonomy-disabled skipsrc/scheduler/service.py:717-719 returns without creating an execution row and without advancing run times. Fires keep silently skipping every window; next_run_at stays frozen, runs stay 0. (Contrast: the pre-check fire=false path at service.py:773-800 correctly records a skipped execution AND advances run times — the autonomy skip predates that pattern.)
  2. Lock-contention returnservice.py:690-693 returns with no advance and no record.
  3. Failure outcome branchservice.py:867-888: the comment says "update schedule run times" but the branch never calls update_schedule_run_times, so a repeatedly-failing schedule also shows a stale past "Next:" (runs > 0 in that variant).
  4. Sync-loop add-failure permanence — if _add_job throws, the exception is swallowed (service.py:399-400) but _sync_agent_schedules still records the snapshot (service.py:473), so the schedule is never retried — no APScheduler job exists until a container restart, and next_run_at stays at its creation-time value.
  5. No runtime repair — the missed-schedule catch-up (Scheduler fires Monday cron jobs on Tuesday (off-by-one day) #145, _get_missed_schedules) only runs at initialize(); nothing detects or repairs a stale next_run_at while the scheduler is running.

Frontend: src/frontend/src/components/SchedulesPanel.vue:321 renders Next: {{ formatRelativeTime(schedule.next_run_at) }} with no overdue handling, so a past timestamp renders as the nonsensical "Next: 1d ago" instead of an explicit overdue/warning state.

This is the same projection-vs-authority class as #1082 (status-as-projection): next_run_at is a projection of APScheduler's in-memory trigger state, and the write discipline around it is leaky.

Acceptance Criteria

  • Root cause identified for the observed case (enabled daily schedule, 0 runs, next_run_at ~1 day in the past, sibling schedule on the same agent firing normally)
  • Every path where a due cron fire is skipped either advances next_run_at or records an observable skip (execution row / event) — in particular the autonomy-disabled skip at service.py:717
  • Failure outcome branch advances run times (or the projection is advanced at fire time regardless of outcome)
  • _sync_agent_schedules no longer permanently orphans a schedule whose _add_job failed (retry on next sync, or surface the failure)
  • UI renders an explicit overdue state (e.g. "Overdue by 1d") instead of "Next: Xd ago" when next_run_at < now on an enabled schedule
  • Consider a canary invariant: enabled, non-deleted schedule with next_run_at older than now − misfire_grace_time (fits the CANARY-001 catalog)

Technical Notes

Metadata

Metadata

Assignees

Labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions