Recurring missing schedule runs despite successful workflow_run fallback — ichrob/gold-dashboard1

[ichrob]

🏷️ Discussion Type

Bug

💬 Feature/Topic Area

Schedule & Cron Jobs

Discussion Details

Please help diagnose recurring gaps in scheduled run creation for my public repository. This is intermittent: genuine schedule runs do occur and succeed, but many expected slots have no run record. The cause is not established.

Repository and workflow

• Repository: https://github.com/ichrob/gold-dashboard1
• Default branch: main
• Workflow: .github/workflows/bob-self-healing.yml
• Workflow ID: 382076329
• Cron (UTC): 4,14,24,34,44,54 * * * *
• Existing additional triggers: workflow_run fallback, push and workflow_dispatch.
• Workflow blob checked in the documented investigation: 11630a25078e0b0deb00e517b38a2b24639d3c54.

Recorded evidence on 8 November 2026 (UTC)

• Genuine schedule run #1292, ID 42435580364, created 01:44:56 UTC, completed successfully, including the monitor script:
  https://github.com/ichrob/gold-dashboard1/actions/runs/42435580364
• At the later recorded checks, subsequent expected slots were absent from the run list.
• Successful fallback run #1293, ID 42455895067, created 06:43:15 UTC, event=workflow_run:
  https://github.com/ichrob/gold-dashboard1/actions/runs/42455895067
• The recorded scheduled/fallback executions used actor and triggering_actor ichrob.
• This has been tracked since 27 October, with isolated successful schedule runs and long gaps. Some observation windows affected multiple repository workflows; in others, other workflows received schedule events.
• These timestamps are documented snapshots, not a claim that no further runs have appeared since.

Original investigation and history:
ichrob/gold-dashboard1#940
Latest evidence used for escalation:
ichrob/gold-dashboard1#940 (comment)

No specific YAML, checkout, Python or monitor-script fault has been established. Successful workflow_run executions do not establish healthy cron delivery. The existing cron already avoids minute 00. I have not changed the workflow speculatively or added a second scheduler.

Related reports and documentation reviewed

• https://docs.github.com/en/actions/how-tos/troubleshoot-workflows documents load-related delays/drops and the separate actor-account case. Neither establishes the cause here.
• https://github.com/orgs/community/discussions/185081 includes a January 2026 response about a rolled-back change and resynchronization via a default-branch commit, plus an explanation that dropped events may leave no run artifacts. That historical incident does not prove the same cause in November.
• https://github.com/orgs/community/discussions/211849 describes another recent missing-schedule case despite configuration checks and attempted reactivation. Similar symptoms are not proof of a shared cause.

GitHub Support directed me to community/self-service support without a technical diagnosis.

Questions

1. Can GitHub investigate schedule registration/event delivery for this workflow, especially the recorded 2026-11-08 01:44–06:56 UTC window?
2. What observable evidence distinguishes normal load-related event dropping from a stale schedule registration or an account/repository restriction?
3. Is there a supported diagnostic or recovery step for this intermittent case that preserves the existing cadence, without blind repeated cron edits or a duplicate scheduler?

Please distinguish confirmed platform findings from hypotheses. Thank you.

[github-actions[bot]]

💬 Your Product Feedback Has Been Submitted 🎉

Thank you for taking the time to share your insights with us! Your feedback is invaluable as we build a better GitHub experience for all our users.

Here's what you can expect moving forward ⏩

• Your input will be carefully reviewed and cataloged by members of our product teams. 
  • Due to the high volume of submissions, we may not always be able to provide individual responses.
  • Rest assured, your feedback will help chart our course for product improvements.
• Other users may engage with your post, sharing their own perspectives or experiences. 
• GitHub staff may reach out for further clarification or insight. 
  • We may 'Answer' your discussion if there is a current solution, workaround, or roadmap/changelog post related to the feedback.

Where to look to see what's shipping 👀

• Read the Changelog for real-time updates on the latest GitHub features, enhancements, and calls for feedback.
• Explore our Product Roadmap, which details upcoming major releases and initiatives.

What you can do in the meantime 💻

• Upvote and comment on other user feedback Discussions that resonate with you.
• Add more information at any point! Useful details include: use cases, relevant labels, desired outcomes, and any accompanying screenshots.

As a member of the GitHub community, your participation is essential. While we can't promise that every suggestion will be implemented, we want to emphasize that your feedback is instrumental in guiding our decisions and priorities.

Thank you once again for your contribution to making GitHub even better! We're grateful for your ongoing support and collaboration in shaping the future of our platform. ⭐

[NaniToka]

Hi @ichrob,

Here is the technical diagnosis for the recurring gaps in your scheduled workflow runs:

1. Root Cause: GitHub's Shared Cron Dispatcher & Queue Shedding

GitHub Actions scheduled workflows run on a shared, best-effort platform queue. Per official GitHub documentation:

• Load-induced dropping: When global Actions runner queue load spikes, scheduled runs are subject to throttling or delays.
• Slot collapse / deduplication: If a scheduled run from a previous interval is delayed or queued while the next slot arrives, GitHub's event scheduler drops the missed slot rather than accumulating a backlog. Running on a tight 10-minute cadence (4,14,24,34,44,54) makes this significantly more vulnerable to dropped slots during peak platform hours.

2. Distinguishing Load Drops vs. Account/Registration Inactivity

• 60-Day Inactivity Rule: GitHub automatically pauses scheduled workflows on public repositories that have had no push or workflow activity in the last 60 days. Because your repository has active pushes and fallback runs, inactivity suspension is ruled out.
• Observable Evidence: Because your workflow succeeds intermittently and fallback workflow_run triggers fire without issues, the repository registration is healthy; the drops are classic symptoms of the shared scheduler queue shedding during high platform load.

3. Recommended Recovery & Deterministic Solution

If you require a strict, guaranteed 10-minute heartbeat:

1. Offset off-cadence: Change your cron minutes to non-round numbers (e.g., 7,19,31,43,53 * * * *). GitHub's infrastructure experiences spikes around 5- and 10-minute boundaries.
2. Deterministic Trigger via workflow_dispatch (Best Practice): Rather than relying on GitHub's internal best-effort cron daemon, use an external scheduler (such as AWS EventBridge, Google Cloud Scheduler, or Cron-job.org) that calls your workflow's workflow_dispatch API endpoint with a fine-grained Personal Access Token:

   POST https://api.github.com/repos/ichrob/gold-dashboard1/actions/workflows/bob-self-healing.yml/dispatches

   API-dispatched workflow runs receive immediate event creation and are never dropped by the cron scheduler.

[NaniToka]

Hi @ichrob,

To address your questions regarding the intermittent scheduled runs and GitHub Actions cron event delivery:

Q1 & Q2: Understanding Dropped Cron Events vs. Account Restrictions

1. Load-Related Event Dropping vs. Stale Schedule:

   • Under periods of high platform load, GitHub Actions' scheduler queue drops scheduled events rather than queuing them indefinitely. According to GitHub's architecture docs, if a scheduled run cannot be queued within its window, the trigger is dropped and leaves no run record or event artifact in the Actions log.
   • If an account/repository restriction were the cause (e.g., billing cap, inactivity threshold, or repository disabled), all schedule runs would consistently fail or produce explicit disabled notifications in the workflow overview. The fact that genuine scheduled runs (such as run #1292) succeed intermittently rules out an account lockout or disabling.

2. Inactivity Rule (60-Day Policy):

   • Scheduled workflows are automatically paused on public repositories that have no commit activity for 60 consecutive days. A commit to the default branch automatically re-enables them. Since your repository is actively receiving commits, this is not the cause here.

---

Q3: Diagnostic & Stabilization Steps

Without modifying your cron frequency or deploying redundant schedulers, here are the most effective platform-compliant steps:

1. Re-register the Workflow Schedule Hook:
   GitHub Actions' internal cron schedule indexer can occasionally fall out of sync with repository refs. Making an empty commit to main forces the scheduler daemon to re-register the workflow cron table:

   git commit --allow-empty -m "chore(actions): re-register cron schedule"
   git push origin main

2. Review High-Traffic Minute Windows:
   Although your cron minutes (4, 14, 24, 34, 44, 54) correctly avoid the top of the hour (:00) and half hour (:30), minutes ending in 4 can still face contention during shift changes in standard UTC cron batches. Shifting to prime or odd offsets (e.g., 7, 17, 27, 37, 47, 57) significantly lowers the probability of platform load drops.
3. Repository Actions Permissions Toggle:
   Under Settings → Actions → General, toggle Workflow permissions (e.g., switch between Read/Write and save, then switch back) to trigger a backend permission refresh.

[kittimzhe]

Live-pulled your schedule-run record to quantify the gap — and it's larger than intermittent: the last 15 schedule runs of bob-self-healing.yml span Nov 6 21:57 → Nov 10 02:54 UTC (~77 hours), which averages one run per ~5.1 hours against a configured cadence of one per 10 minutes (770 schedule runs total in the workflow's history). That's a ~97% slot-drop rate, not random jitter.

The mechanism is documented behavior, unfortunately: scheduled workflows are best-effort and explicitly may be dropped or delayed during periods of high load, and high-frequency crons are first in line — a 4,14,24,34,44,54 * * * * schedule asks for 144 runs/day, which is exactly the profile that gets coalesced hardest. The silent part you already know: a dropped tick leaves no run object, so runs?event=schedule (your tool, and mine above) is the only record.

What actually holds up at this cadence:

1. Your workflow_run fallback is the right architecture — it's driven by real events, not the cron queue, and your data shows those delivering reliably. Lean on it harder: anything that must run after CI belongs on that path.
2. For the watchdog cadence itself, the highest-reliability slot GitHub offers is hourly at an off-minute (17 * * * * — away from the top-of-hour load peak). Sub-hourly crons will keep silently collapsing during load windows; that's a platform property, not a bug in your setup.
3. If you genuinely need 10-minute monitoring, the pattern that works is an external pinger (any cron service hitting repository_dispatch, or a self-hosted/minimal scheduler) — event-triggered runs don't go through the schedule queue at all. Same family, for reference: Actions schedule events created over 5 hours after the last permitted UTC cron time (October 2, 2026) #209399, Scheduled workflows stopped firing after workflow file replacement — workflow_dispatch still works #209438, Actions schedule events never create runs; push and manual runs succeed #211879.