Monitor branch protection and open Dependabot alerts - #15
Conversation
Two calls per repo, both on the existing per-repo loop.
Branch protection is the lever the board was missing. Every other metric here
reports what happened; this one reports what is *allowed* to happen, which is
the thing you can change once and have hold. On the current fleet it already
finds that one repo protects its default branch, and that one still permits
force pushes - a green tile on a branch anyone can rewrite.
Both endpoints 404 in two different situations, and conflating them would
make the board lie in opposite directions:
* protection 404s for an unprotected branch AND for a token without admin.
Exporting 0 for both would invent findings out of a permission gap, so
jq_branch_protected is absent rather than zero when GitHub will not say.
Absent and 0 are distinguishable in PromQL; a zero is not.
* the alerts endpoint 404s or 403s when the feature is off, which is
indistinguishable from "no alerts open". "Nothing to fix" and "nobody is
looking" are opposite facts, so jq_dependabot_alerts_enabled carries that
separately and the counts are absent entirely when it is off. Four repos
in the fleet turn out to have alerts disabled.
Severities are zero-filled, so a repo that clears its criticals reports 0
rather than dropping out of the query and stranding its last value on the
graph.
Cost is ~2 extra calls per repo, taking a pass over 25 repos from ~450 to
~700. That fits only because JQ_GITHUB_INTERVAL moved to 600s; at the old
300s this would have pushed an already-over-budget refresh further over.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two changes that belong together: the collector could not say "unprotected", and the board had nowhere to say it. branch_protection now reads the 404 body. GitHub answers an unprotected branch with "Branch not protected" and a permission gap with something else, so the two 404s are distinguishable after all. Collapsing them lost the finding entirely: protection was exported only where it existed, and 22 of 25 repos simply had no series - indistinguishable from a fleet nobody had looked at. Unknown is still absent rather than zero, so a permission gap never becomes an accusation. Four tiles in Fleet at a glance, each linked to a drill-down table: Default branches unprotected 22 Protected but force-pushable 1 Dependabot off 4 Open Dependabot alerts 1 "Protected but force-pushable" earns its own tile because it is the case a protection tile hides: the one repo on the fleet that protects its default branch also permits rewriting it, so counting protected repos alone would report that as solved. "Dependabot off" sits beside the alert count for the same reason - four repos contribute a truthful zero to a total that would otherwise read as an all-clear. The tiles use sum(x == bool 0), not count(). count() over a bool comparison counts every series regardless of the comparison, which read 23 unprotected of 23 and 25 Dependabot-off of 25 - the total dressed up as a finding, and plausible enough to miss. Deliberately not added to fleet-public.json. A world-readable list of which repos are unprotected and which have open advisories is a target list. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Pushed two more commits: the collector can now say unprotected, and the board shows it. The metric was half-blind
Now unprotected is a fact ( Four tiles in Fleet at a glance
Each links to a drill-down table in the existing collapsed row. Two of these exist to stop a tile lying by omission. Protected but force-pushable is the case a protection count hides — the single repo that protects A bug worth flaggingThe tiles first used Not on the public board
45 tests pass; |
Two of the four new tiles linked to a table that excluded, by construction, exactly the repos the tile counted: Dependabot off -> the open-alerts table, which lists only repos where Dependabot is enabled. Clicking "4 repos are off" showed repos that are on. Protected but force-pushable -> the unprotected table, which matches jq_branch_protected == 0. A force-pushable branch is protected, so the one repo behind that tile could never appear. Both were reachable in one click and answered a question with its own opposite, which is worse than having no link: a drill-down is read as the evidence for the number above it. Adds the two missing tables and repoints the tiles. Each of the four now matches its own count against live Prometheus - 22, 1, 4 and 0 rows for tiles reading 22, 1, 4 and 0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The Dependabot commits left two files unformatted and never re-ran the public dashboard generator, so CI's two guards - ruff format --check and the "fleet-public.json is stale" diff - both fired. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds two metric groups, both on the existing per-repo loop in
github.py.Branch protection
jq_branch_protected,jq_branch_required_reviews,jq_branch_allows_force_push.Every other metric on this board reports what happened. This one reports what is allowed to happen — the thing you change once and it holds. On the current 25-repo fleet it already finds:
Jebel-Quant/rhiza) protects its default branchDependabot
jq_dependabot_alerts_enabled,jq_dependabot_open_alerts{severity}.Four repos in the fleet have alerts switched off entirely. One (
tschm/proximal) has an open low-severity alert that nothing on the board showed before.The part worth reviewing
Both endpoints return 404 in two different situations, and conflating them makes the board lie in opposite directions:
branches/{b}/protectiondependabot/alerts_enabledgauge; counts absent when offExporting
0for unknown protection would put every repo on the "unprotected" list on the strength of a permission gap — a finding the board invented.absentand0are distinguishable in PromQL; two zeros are not.Likewise "nothing to fix" and "nobody is looking" must not render as the same green tile.
Severities are zero-filled so a repo that clears its criticals reports
0rather than dropping out of the query and stranding its last value on the graph.Cost
~2 extra calls per repo: a pass over 25 repos goes from ~450 to ~700 calls, 37s to 40s. That fits only because #14 moves
JQ_GITHUB_INTERVALto 600s — this should merge after #14, or the refresh goes further over an already-exceeded budget.Testing
4 new tests covering both unknown-vs-zero distinctions; full suite of 48 passes. Verified against the live fleet — the numbers quoted above are from the running collector, not fixtures.
🤖 Generated with Claude Code