Add workflow that fails when Dependabot updates error#323
Conversation
Dependabot version update failures only surface as a red triangle in the Dependabot tab, which nobody checks. This weekly scheduled workflow fails if any "Dependabot Updates" run concluded with failure, startup_failure, or timed_out in the last 8 days, so a broken ecosystem surfaces as a red scheduled run that emails a human. See maxmind/device-android#61 for the reference implementation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
📝 WalkthroughWalkthroughAdds a weekly and manually triggered GitHub Actions workflow that checks recent Dependabot Updates runs and fails with details when failure-like conclusions are detected. ChangesDependabot failure monitoring
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 Checkov (3.3.8).github/workflows/dependabot-failure-watcher.ymlTraceback (most recent call last): Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Pull request overview
Adds a scheduled GitHub Actions workflow to detect and surface silently failing “Dependabot Updates” runs by failing a weekly watcher job when recent Dependabot update runs have concluded unsuccessfully.
Changes:
- Introduces a new scheduled + manually-dispatchable workflow to scan recent “Dependabot Updates” workflow runs.
- Fails the watcher job and emits an error annotation listing failing runs when failures are detected.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| failures=$(gh run list \ | ||
| --repo "$REPO" \ | ||
| --workflow "Dependabot Updates" \ | ||
| --limit 100 \ |
| --workflow "Dependabot Updates" \ | ||
| --limit 100 \ | ||
| --json conclusion,createdAt,displayTitle,url \ | ||
| --jq "[.[] | select((.conclusion == \"failure\" or .conclusion == \"startup_failure\" or .conclusion == \"timed_out\") and .createdAt >= \"$since\")]") |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/dependabot-failure-watcher.yml:
- Around line 3-11: Update the comments in the Dependabot failure watcher
workflow to describe an 8-day lookback instead of the past week, and replace the
guaranteed email claim with an accurate statement that notifications depend on
GitHub notification settings and the scheduled-workflow actor. Leave the
workflow behavior unchanged.
- Around line 31-33: Update the gh run list invocation in the failures
assignment to include the --all option, ensuring disabled “Dependabot Updates”
workflow runs are considered while preserving the existing repository, workflow,
and time-window filters.
- Around line 34-36: Update the Dependabot run query in the workflow to paginate
through all matching runs instead of using --limit 100, and filter the failure
conditions by each run’s conclusion time rather than createdAt. Preserve the
existing failure conclusions and eight-day cutoff while ensuring runs that
conclude within the window are included even if they were created earlier.
- Around line 35-36: Update the `gh run list` JSON fields and jq filter to use
`updatedAt` instead of `createdAt` when comparing against `$since`, so runs
completed within the eight-day window are included even if they started earlier.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 5669ba20-fbfe-4847-af48-55eeade35840
📒 Files selected for processing (1)
.github/workflows/dependabot-failure-watcher.yml
| # Dependabot version updates run as GitHub Actions workflow runs named | ||
| # "Dependabot Updates". This scheduled job looks back over the past week for any | ||
| # of those runs that failed and fails itself if it finds one, so a silently-broken | ||
| # ecosystem surfaces as a red scheduled run instead of only a red triangle in the | ||
| # Dependabot tab that nobody checks. | ||
| # | ||
| # Runs entirely within this repo (no external service). A failed scheduled run | ||
| # emails the person who last edited the cron below. Note: GitHub auto-disables | ||
| # scheduled workflows after 60 days of repo inactivity. |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '--- workflow file outline ---\n'
wc -l .github/workflows/dependabot-failure-watcher.yml
cat -n .github/workflows/dependabot-failure-watcher.yml | sed -n '1,220p'
printf '\n--- search for related docs or comments ---\n'
rg -n "Dependabot Updates|scheduled workflows after 60 days|person who last edited the cron|notification|workflow runs" .github/workflows .github -g '!**/*.png' || trueRepository: maxmind/GeoIP2-php
Length of output: 247
🌐 Web query:
GitHub Actions notifications for workflow runs scheduled workflow actor cron editor email delivery docs
💡 Result:
For GitHub Actions scheduled workflows, notifications are automatically sent to the user identified as the actor [1][2]. Specifically, the system sends notifications to the user who last modified the cron syntax in the workflow file [1][2]. If the workflow is disabled and then re-enabled, the notifications are sent to the user who performed the re-enabling action rather than the original author of the cron syntax [1]. Key points regarding scheduled workflow notifications: - Recipient: Notifications go to the user associated with the workflow's cron schedule [1][2]. This is typically the last person who updated the cron syntax or re-enabled the workflow [1]. - Delivery Method: If you have email or web notifications enabled for GitHub Actions in your personal notification settings, these alerts will be delivered accordingly [1][3]. You can configure your preferences to receive updates for all workflow runs or only for failed runs [1][3]. - Customization: GitHub does not natively provide a way to change the notification recipient for a specific scheduled workflow to someone other than the current "actor." [1][2] - Alternative Solutions: To notify a broader team or a specific channel (e.g., Slack or Microsoft Teams) about a scheduled workflow's status, users often add an explicit notification step to their workflow file [4]. This is commonly done using actions or scripts that send a payload to an external webhook (such as a Slack webhook or a REST API endpoint) upon completion or failure [4]. For managing your personal notification settings, you can navigate to your GitHub notification settings page and look under the Actions section to select your preferred delivery method (Web or Email) and filter criteria (e.g., failed workflows only) [3][5].
Citations:
- 1: https://docs.github.com/en/actions/concepts/workflows-and-actions/notifications-for-workflow-runs
- 2: https://docs.github.com/en/actions/reference/workflows-and-actions/events-that-trigger-workflows
- 3: https://docs.github.com/en/subscriptions-and-notifications/how-tos/managing-github-actions-notifications
- 4: https://stackoverflow.com/questions/70450908/github-actions-notifications-for-scheduled-cron-jobs
- 5: https://docs.github.com/en/subscriptions-and-notifications/get-started/configuring-notifications
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo '--- workflow file ---'
cat -n .github/workflows/dependabot-failure-watcher.yml | sed -n '1,220p'
echo
echo '--- nearby references ---'
rg -n "past week|eight days|60 days|cron|email|Dependabot Updates|scheduled workflows" .github/workflows .github -g '!**/*.png' || trueRepository: maxmind/GeoIP2-php
Length of output: 4068
Align the comment with the actual behavior.
- The lookback is 8 days, not “the past week”.
- Don’t promise an email to the person who last edited the cron; scheduled-workflow notifications depend on GitHub notification settings and the scheduled-workflow actor.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/dependabot-failure-watcher.yml around lines 3 - 11, Update
the comments in the Dependabot failure watcher workflow to describe an 8-day
lookback instead of the past week, and replace the guaranteed email claim with
an accurate statement that notifications depend on GitHub notification settings
and the scheduled-workflow actor. Leave the workflow behavior unchanged.
| failures=$(gh run list \ | ||
| --repo "$REPO" \ | ||
| --workflow "Dependabot Updates" \ |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Inspect the workflow file around the referenced lines
nl -ba .github/workflows/dependabot-failure-watcher.yml | sed -n '1,120p'
# Show the exact gh run list help text for workflow filtering if gh is available
if command -v gh >/dev/null 2>&1; then
gh run list --help | sed -n '1,220p'
else
echo "gh CLI not available in sandbox"
fiRepository: maxmind/GeoIP2-php
Length of output: 196
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Locate and inspect the workflow file without relying on nl
git ls-files .github/workflows/dependabot-failure-watcher.yml
echo '---'
cat -n .github/workflows/dependabot-failure-watcher.yml | sed -n '1,140p'Repository: maxmind/GeoIP2-php
Length of output: 2272
🌐 Web query:
gh run list --workflow --all disabled workflows manual
💡 Result:
To list runs for disabled workflows using the GitHub CLI (gh), you must use the --all (or -a) flag in conjunction with the --workflow (-w) flag [1][2][3]. By default, when you filter by a workflow name using the -w flag, the GitHub CLI does not include disabled workflows in the results [1][4]. Adding the --all flag explicitly instructs the command to include them [1][5]. Example command: gh run list --workflow "Workflow Name" --all Key details: - The --all flag mirrors the behavior found in gh workflow list [5]. - When filtering by workflow, you can provide the workflow name, ID, or filename [6]. - If you encounter issues where disabled workflows are not appearing, ensure that you are providing the correct identifier and appending --all to your command [1][7].
Citations:
- 1: https://cli.github.com/manual/gh_run_list
- 2: https://man.archlinux.org/man/extra/github-cli/gh-run-list.1.en
- 3:
gh run listshould support--pr <num>cli/cli#10386 - 4:
run list: do not fail on organization/enterprise ruleset imposed workflows cli/cli#10660 - 5: feat: add
-aflag togh run listcli/cli#9162 - 6: https://github.com/github/docs/blob/main/content/actions/how-tos/monitor-workflows/view-workflow-run-history.md
- 7: Undocumented behaviour of the
-wflag forgh run listcli/cli#9038
Include disabled workflow runs.
gh run list --workflow ... skips disabled workflows unless --all is added. If Dependabot Updates is disabled, this watcher can miss a failure in the eight-day window and report success.
Proposed fix
failures=$(gh run list \
--repo "$REPO" \
+ --all \
--workflow "Dependabot Updates" \📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| failures=$(gh run list \ | |
| --repo "$REPO" \ | |
| --workflow "Dependabot Updates" \ | |
| failures=$(gh run list \ | |
| --repo "$REPO" \ | |
| --all \ | |
| --workflow "Dependabot Updates" \ |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/dependabot-failure-watcher.yml around lines 31 - 33,
Update the gh run list invocation in the failures assignment to include the
--all option, ensuring disabled “Dependabot Updates” workflow runs are
considered while preserving the existing repository, workflow, and time-window
filters.
| --limit 100 \ | ||
| --json conclusion,createdAt,displayTitle,url \ | ||
| --jq "[.[] | select((.conclusion == \"failure\" or .conclusion == \"startup_failure\" or .conclusion == \"timed_out\") and .createdAt >= \"$since\")]") |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift
Don't cap this completeness check at 100 runs.
--limit 100can hide a failing Dependabot run once more than 100 runs exist in the eight-day window.- Filtering on
createdAtalso misses runs created before the cutoff that fail inside the window.
Use pagination and filter by the run’s conclusion time, not createdAt.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/dependabot-failure-watcher.yml around lines 34 - 36,
Update the Dependabot run query in the workflow to paginate through all matching
runs instead of using --limit 100, and filter the failure conditions by each
run’s conclusion time rather than createdAt. Preserve the existing failure
conclusions and eight-day cutoff while ensuring runs that conclude within the
window are included even if they were created earlier.
| --json conclusion,createdAt,displayTitle,url \ | ||
| --jq "[.[] | select((.conclusion == \"failure\" or .conclusion == \"startup_failure\" or .conclusion == \"timed_out\") and .createdAt >= \"$since\")]") |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n## file slice\n'
nl -ba .github/workflows/dependabot-failure-watcher.yml | sed -n '1,140p'
printf '\n## search timestamp fields in repo\n'
rg -n --hidden --glob '!**/.git/**' '\b(createdAt|updatedAt|startedAt|completedAt)\b' .github/workflows . 2>/dev/null | sed -n '1,160p'Repository: maxmind/GeoIP2-php
Length of output: 211
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n## file slice\n'
cat -n .github/workflows/dependabot-failure-watcher.yml | sed -n '1,140p'
printf '\n## gh run list docs mentions\n'
python3 - <<'PY'
import urllib.request
from html.parser import HTMLParser
url = "https://cli.github.com/manual/gh_run_list"
print("Fetching:", url)
with urllib.request.urlopen(url, timeout=20) as r:
data = r.read().decode("utf-8", "replace")
for needle in ["createdAt", "updatedAt", "completedAt", "conclusion"]:
idx = data.find(needle)
print("\nNEEDLE", needle, "idx", idx)
if idx != -1:
start = max(0, idx - 200)
end = min(len(data), idx + 400)
print(data[start:end].replace("\n", " "))
PYRepository: maxmind/GeoIP2-php
Length of output: 4602
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n## file slice\n'
cat -n .github/workflows/dependabot-failure-watcher.yml | sed -n '1,140p'
printf '\n## gh run list docs mentions\n'
python3 - <<'PY'
import urllib.request
url = "https://cli.github.com/manual/gh_run_list"
print("Fetching:", url)
with urllib.request.urlopen(url, timeout=20) as r:
data = r.read().decode("utf-8", "replace")
for needle in ["createdAt", "updatedAt", "completedAt", "conclusion"]:
idx = data.find(needle)
print("\nNEEDLE", needle, "idx", idx)
if idx != -1:
start = max(0, idx - 200)
end = min(len(data), idx + 400)
print(data[start:end].replace("\n", " "))
PYRepository: maxmind/GeoIP2-php
Length of output: 4602
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n## file slice\n'
sed -n '1,160p' .github/workflows/dependabot-failure-watcher.yml | cat -n
printf '\n## grep timestamp-related lines\n'
grep -nE 'createdAt|updatedAt|completedAt|conclusion|gh run list' .github/workflows/dependabot-failure-watcher.yml || trueRepository: maxmind/GeoIP2-php
Length of output: 2630
🏁 Script executed:
#!/bin/bash
set -euo pipefail
gh run list --help | sed -n '1,240p'
printf '\n## timestamp field mentions\n'
gh run list --help | grep -nE 'createdAt|updatedAt|completedAt|json' || trueRepository: maxmind/GeoIP2-php
Length of output: 2504
🌐 Web query:
GitHub Actions workflow run createdAt updatedAt completion time gh run list updatedAt documentation
💡 Result:
In GitHub Actions, workflow run metadata includes timestamps for creation and updates, though "completion time" is not provided as a single field. Workflow Run Timestamps - created_at: The timestamp indicating when the workflow run was created [1][2][3]. - updated_at: The timestamp indicating the last time the workflow run was modified [1][2][3]. - run_started_at: In the REST API, this field provides the time when the workflow execution actually began [1][2]. - completion time: There is no dedicated completion_time field. To calculate the total duration, you typically use the run_started_at timestamp and the time at which the status reached completed, or derive it from the completed_at field available in individual job details [4]. Using gh run list The GitHub CLI (gh run list) supports displaying these timestamps via the --json flag. You can request the following fields [5][6]: - createdAt - updatedAt - startedAt (corresponds to run_started_at) - status (used to determine if the run is completed) Example usage: gh run list --json createdAt,updatedAt,startedAt,status,name For custom formatting or calculations (such as duration), you can use the --template flag with Go templates or pipe the JSON output to jq to process the timestamps [5][7]. The API and CLI return these values as ISO 8601 formatted strings [1][4].
Citations:
- 1: https://docs.github.com/en/rest/actions/workflow-runs?apiVersion=2026-03-10
- 2: http://help.github.com/en/enterprise-cloud@latest/rest/actions/workflow-runs
- 3: https://docs.github.com/enterprise-server@3.19/rest/actions/workflow-runs
- 4: https://github.com/cli/cli/blob/trunk/pkg/cmd/run/shared/shared.go
- 5: https://cli.github.com/manual/gh_run_list
- 6: Improve
run listdoc with available--jsonfields cli/cli#8934 - 7: https://manpages.debian.org/testing/gh/gh-run-list.1.en.html
Filter on updatedAt instead of createdAt. This check is meant to catch runs that concluded in the last 8 days, but createdAt can miss runs that start before the cutoff and finish after it. gh run list exposes updatedAt, which fits this window better.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/dependabot-failure-watcher.yml around lines 35 - 36,
Update the `gh run list` JSON fields and jq filter to use `updatedAt` instead of
`createdAt` when comparing against `$since`, so runs completed within the
eight-day window are included even if they started earlier.
Add a weekly scheduled workflow that fails when any "Dependabot Updates" run concluded with
failure,startup_failure, ortimed_outin the last 8 days, so a silently-broken Dependabot ecosystem surfaces as a red scheduled run that emails a human instead of only a red triangle in the Dependabot tab that nobody checks.This is a copy of the reference change in maxmind/device-android#61, adjusted to run on Wednesdays and to also treat
timed_outas a failure.🤖 Generated with Claude Code
Summary by CodeRabbit