feat(rules): BaselineHealth (BH001-BH003) — detect red-baseline conditions estate-wide#316
Merged
Merged
Conversation
…eline detection
Three new detection rules in lib/rules/baseline_health.ex, each
addressing a failure mode that combined into the breakage of
reasonably-good-token-vault main (see hyperpolymath/reasonably-good-
token-vault#89, the originating fix):
- BH001: `main` branch protection has no `required_status_checks`
(or empty contexts/checks). Severity :critical. Action :report via
sustainabot. Strictly stronger than the existing ERR-SEC-001
"missing branch protection" check — that fires when there's no
protection AT ALL; BH001 fires when protection exists but doesn't
require CI to pass, which is the exact gap that let red-CI PRs
reach main in r-g-t-v.
- BH002: Local Cargo.toml / package.json scan for lines where a
dependency declaration carries a TODO/FIXME mentioning migration,
rename, API, breaking, or major. Severity :high. Action
:open_followup_pr via rhodibot. Catches the exact pattern that left
r-g-t-v#82 stuck: `ureq = { version = "3" } # TODO: migrate to
3.x API` was the smell, and the migration TODO sat in a merged PR
with no follow-up issue.
- BH003: A workflow run targeting main has been failing for longer
than `threshold_hours` (default 24). Severity :critical. Action
:escalate via sustainabot. Distinguishes flakes (<24h) from real
baseline outages.
Conventions matched from existing rule modules:
- `System.cmd("curl", [...])` for GitHub API (per the HTTPoison TLS
comment in rules.ex)
- `GITHUB_TOKEN` or `HYPATIA_DISPATCH_PAT` for auth, with clean `[]`
return on missing token
- `extract_owner_repo/1` parses `git remote get-url origin`, mirroring
dependabot_alerts.ex
- Findings shape: `%{rule:, file:, severity:, reason:, action:,
detail:}`
- `scan/2` facade returns `%{findings, total, by_severity, dispatch}`
matching `GitState.scan/1`
- `defdelegate scan_baseline_health(...)` added to the `Hypatia.Rules`
facade
Testing: 12 tests in test/baseline_health_test.exs covering the
no-token paths, all four positive BH002 patterns, the benign-TODO
negative case, subdirectory walking, target/ and node_modules/
exclusion, the scan/2 facade shape, and the owner_repo override.
Excluding tags: [:verisim_data]
12 tests, 0 failures (mix test test/baseline_health_test.exs)
Propagation: once landed, these rules detect the originating-incident
class of failures estate-wide. Per the survey:
- .git-private-farm has no branch-protection authority — strictly
mirroring/dispatch. Not a placement candidate.
- gitbot-fleet documents required-status-checks wiring but no bot
enforces it; Hypatia's safety triangle routes BH001 advisories to
sustainabot for owner action.
- echidnabot is downstream of `proof_obligation.ex` only — no stake
in branch-protection / dep / CI-color invariants.
- ERR-SEC-001 in cicd_rules.ex covers the weaker case (no protection
at all); BH001 is the strictly-stronger refinement.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
🔍 Hypatia Security ScanFindings: 117 issues detected
View findings[
{
"reason": "Docker reference in Nickel config -- Podman/Containerfile highly preferred (Docker permitted) (1 occurrences, CWE-1104)",
"type": "ncl_docker_not_podman",
"file": "/home/runner/work/hypatia/hypatia/.machine_readable/svc/k9/hypatia-metadata.k9.ncl",
"action": "flag",
"rule_module": "code_safety",
"severity": "low"
},
{
"reason": "unwrap() without prior check -- DoS via panic (2 occurrences, CWE-754)",
"type": "unwrap_without_check",
"file": "/home/runner/work/hypatia/hypatia/cli/src/commands/batch.rs",
"action": "flag",
"rule_module": "code_safety",
"severity": "low"
},
{
"reason": "unwrap_or(0) with dangerous default (1 occurrences, CWE-754)",
"type": "unwrap_dangerous_default",
"file": "/home/runner/work/hypatia/hypatia/cli/src/commands/batch.rs",
"action": "flag",
"rule_module": "code_safety",
"severity": "low"
},
{
"reason": "unwrap() without prior check -- DoS via panic (1 occurrences, CWE-754)",
"type": "unwrap_without_check",
"file": "/home/runner/work/hypatia/hypatia/cli/src/commands/scan.rs",
"action": "flag",
"rule_module": "code_safety",
"severity": "low"
},
{
"reason": "unwrap_or(0) with dangerous default (1 occurrences, CWE-754)",
"type": "unwrap_dangerous_default",
"file": "/home/runner/work/hypatia/hypatia/cli/src/commands/scan.rs",
"action": "flag",
"rule_module": "code_safety",
"severity": "low"
},
{
"reason": "unwrap() without prior check -- DoS via panic (2 occurrences, CWE-754)",
"type": "unwrap_without_check",
"file": "/home/runner/work/hypatia/hypatia/cli/src/commands/fleet.rs",
"action": "flag",
"rule_module": "code_safety",
"severity": "low"
},
{
"reason": "unwrap() without prior check -- DoS via panic (2 occurrences, CWE-754)",
"type": "unwrap_without_check",
"file": "/home/runner/work/hypatia/hypatia/cli/src/output.rs",
"action": "flag",
"rule_module": "code_safety",
"severity": "low"
},
{
"reason": "unwrap() without prior check -- DoS via panic (2 occurrences, CWE-754)",
"type": "unwrap_without_check",
"file": "/home/runner/work/hypatia/hypatia/cli/build.rs",
"action": "flag",
"rule_module": "code_safety",
"severity": "low"
},
{
"reason": "unwrap() without prior check -- DoS via panic (3 occurrences, CWE-754)",
"type": "unwrap_without_check",
"file": "/home/runner/work/hypatia/hypatia/fixer/src/main.rs",
"action": "flag",
"rule_module": "code_safety",
"severity": "low"
},
{
"reason": "expect() in hot path (5 occurrences, CWE-754)",
"type": "expect_in_hot_path",
"file": "/home/runner/work/hypatia/hypatia/fixer/src/scanner.rs",
"action": "flag",
"rule_module": "code_safety",
"severity": "low"
}
]Powered by Hypatia Neurosymbolic CI/CD Intelligence |
🔍 Hypatia Security ScanFindings: 113 issues detected
View findings[
{
"reason": "Docker reference in Nickel config -- Podman/Containerfile highly preferred (Docker permitted) (1 occurrences, CWE-1104)",
"type": "ncl_docker_not_podman",
"file": "/home/runner/work/hypatia/hypatia/.machine_readable/svc/k9/hypatia-metadata.k9.ncl",
"action": "flag",
"rule_module": "code_safety",
"severity": "low"
},
{
"reason": "unwrap() without prior check -- DoS via panic (2 occurrences, CWE-754)",
"type": "unwrap_without_check",
"file": "/home/runner/work/hypatia/hypatia/cli/src/commands/batch.rs",
"action": "flag",
"rule_module": "code_safety",
"severity": "low"
},
{
"reason": "unwrap_or(0) with dangerous default (1 occurrences, CWE-754)",
"type": "unwrap_dangerous_default",
"file": "/home/runner/work/hypatia/hypatia/cli/src/commands/batch.rs",
"action": "flag",
"rule_module": "code_safety",
"severity": "low"
},
{
"reason": "unwrap() without prior check -- DoS via panic (1 occurrences, CWE-754)",
"type": "unwrap_without_check",
"file": "/home/runner/work/hypatia/hypatia/cli/src/commands/scan.rs",
"action": "flag",
"rule_module": "code_safety",
"severity": "low"
},
{
"reason": "unwrap_or(0) with dangerous default (1 occurrences, CWE-754)",
"type": "unwrap_dangerous_default",
"file": "/home/runner/work/hypatia/hypatia/cli/src/commands/scan.rs",
"action": "flag",
"rule_module": "code_safety",
"severity": "low"
},
{
"reason": "unwrap() without prior check -- DoS via panic (2 occurrences, CWE-754)",
"type": "unwrap_without_check",
"file": "/home/runner/work/hypatia/hypatia/cli/src/commands/fleet.rs",
"action": "flag",
"rule_module": "code_safety",
"severity": "low"
},
{
"reason": "unwrap() without prior check -- DoS via panic (2 occurrences, CWE-754)",
"type": "unwrap_without_check",
"file": "/home/runner/work/hypatia/hypatia/cli/src/output.rs",
"action": "flag",
"rule_module": "code_safety",
"severity": "low"
},
{
"reason": "unwrap() without prior check -- DoS via panic (2 occurrences, CWE-754)",
"type": "unwrap_without_check",
"file": "/home/runner/work/hypatia/hypatia/cli/build.rs",
"action": "flag",
"rule_module": "code_safety",
"severity": "low"
},
{
"reason": "unwrap() without prior check -- DoS via panic (3 occurrences, CWE-754)",
"type": "unwrap_without_check",
"file": "/home/runner/work/hypatia/hypatia/fixer/src/main.rs",
"action": "flag",
"rule_module": "code_safety",
"severity": "low"
},
{
"reason": "expect() in hot path (5 occurrences, CWE-754)",
"type": "expect_in_hot_path",
"file": "/home/runner/work/hypatia/hypatia/fixer/src/scanner.rs",
"action": "flag",
"rule_module": "code_safety",
"severity": "low"
}
]Powered by Hypatia Neurosymbolic CI/CD Intelligence |
hyperpolymath
added a commit
that referenced
this pull request
May 26, 2026
… detectors (#320) ## Summary Extends `BaselineHealth` (BH001-BH003, landed in #316) with four new rules captured from a follow-on estate-wide session. | Rule | Detects | Severity | Dispatch | |---|---|---|---| | **BH004** | Dead action SHA pin in workflow YAML (`uses: <repo>@<sha>` where SHA doesn't exist upstream) | `:critical` | rhodibot (auto-fix PR, conf 0.93) | | **BH005** | Required status check is push-only (workflow has no `pull_request` trigger) → every PR permanently `BLOCKED` | `:high` | sustainabot (conf 0.85) | | **BH006** | Required check name no longer corresponds to any workflow that emits (stale entry in `required_status_checks`) | `:high` | sustainabot (conf 0.85) | | **BH007** | Commits signed but `verified: false / reason: bad_email` — author email isn't a UID on the registered GPG/SSH key | `:high` | sustainabot (conf 0.80) | ## Why these now Each was a one-line config defect this session uncovered as it propagated across many repos: - **BH004** — \`actions/upload-artifact@65c79d7f...\` (a hallucinated SHA) propagated by template scaffolding into ~62 workflow files across 10 repos. Every workflow run using it died at action-resolution. Fixed in PRs across \`rsr-template-repo\`, \`me-dialect\`, \`ambientops\`. - **BH005** — an org-wide \`required_status_checks\` sweep against 330 repos locked push-context-only checks (\`Dependabot\`, \`mirror-gitlab\`, \`trigger-boj\`, etc.). Result: every PR in 290+ repos went to \`BLOCKED / MERGEABLE\` because those workflows never report on PRs. - **BH006** — \`rsr-template-repo\` had a required list full of names like \`analysis\`, \`dispatch\`, \`mirror-codeberg\` that no longer corresponded to any active workflow. Distinct from BH005: BH005 = "ran on push, not PR"; BH006 = "didn't run anywhere recently". - **BH007** — commits in this very session showed \`bad_email\` because the noreply UID was on the local GPG key but missing from the GitHub-registered public key. Branch protection requiring signed commits silently blocked all PRs. ## Tests \`mix test test/baseline_health_test.exs\` → **18 tests / 0 failures** (was 12). - Each new rule has a no-token regression test (returns \`[]\` cleanly). - BH004 has a positive fixture verifying that nested monorepo scaffolds are NOT scanned (only repo-root \`.github/workflows/\`). - BH002's existing positive tests still pass unchanged. ## Module summary \`lib/rules/baseline_health.ex\` grows from 438 → 877 lines (+458 / -19). Module docstring updated to cover BH001-BH007 with provenance notes for each. Companion PRs to follow: \`workflow_hardening.ex\` (zizmor/actionlint cluster), \`supply_chain.ex\` (Scorecard/SLSA/OWASP cluster), \`branch_protection.ex\` (CIS/NIST cluster). 🤖 Generated with [Claude Code](https://claude.com/claude-code)
hyperpolymath
added a commit
that referenced
this pull request
May 26, 2026
## Summary Adds delegates on \`Hypatia.Rules\` for the rule families landed in PRs #320, #321, #322, #323. The new modules were independently usable but invisible from the top-level facade; this PR closes that gap. ## New facade surface | Function | Delegates to | Rules | |---|---|---| | \`scan_baseline_health/2\` | \`BaselineHealth.scan\` (existing; docstring updated for BH007) | BH001-BH007 | | \`scan_workflow_hardening/2\` | \`WorkflowHardening.scan\` | WH001-WH012 | | \`scan_supply_chain/2\` | \`SupplyChain.scan\` | SC001-SC011 | | \`scan_branch_protection/2\` | \`BranchProtection.scan\` (API-backed; takes \`owner, repo\`) | BP001-BP007 | | \`scan_all_estate_policies/2\` | — runs all four families and merges results | 34 rules total | ## Why combined-scan helper Callers (\`pattern_analyzer\`, \`triangle_router\`, \`fleet_dispatcher\`) already treat \`Hypatia.Rules\` as the entrypoint. Without these delegates the new modules are reachable only via fully-qualified names, which works but inverts the facade pattern adopted in #316. ## RE001-RE010 (PR #325) not included PR #325 (ResearchExtensions) is still OPEN at the time of this PR. Adding its alias here would break compilation on \`main\` for the window between this PR landing and #325 landing. A follow-up commit will wire \`ResearchExtensions\` once #325 merges — tracked inline at the alias block as a \`# wired in follow-up\` comment. ## Test plan - Existing 18-test \`BaselineHealth\` suite still passes (untouched). - Module compile-checks pass for the four families + facade. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three new detection rules in
lib/rules/baseline_health.ex, each addressing a failure mode that combined into the breakage ofhyperpolymath/reasonably-good-token-vaultmain. The originating fix is r-g-t-v#89; these rules detect that class of failure across the estate so the next occurrence is caught before it lands.mainbranch protection has norequired_status_checks(or emptycontexts/checks).:critical. Dispatch:sustainabot:report. Strictly stronger than ERR-SEC-001 incicd_rules.ex(which fires only when there's no protection at all); BH001 fires when protection exists but doesn't gate on CI — the exact gap that let r-g-t-v#82 merge red.Cargo.toml/package.jsonscan for lines where a dependency declaration carries aTODO/FIXMEmentioning migration / rename / API / breaking / major.:high. Dispatch:rhodibot:open_followup_pr. Catches the exact pattern that left r-g-t-v#82 stuck:ureq = { version = "3" } # TODO: migrate to 3.x API.mainhas been failing for longer thanthreshold_hours(default 24).:critical. Dispatch:sustainabot:escalate. Distinguishes flakes (<24h) from real baseline outages.Conventions matched
From
dependabot_alerts.exandgit_state.ex:System.cmd("curl", [...])for GitHub API (per the HTTPoison-TLS comment inrules.ex:287)GITHUB_TOKENorHYPATIA_DISPATCH_PAT, clean[]return on missing token (matches DA001/DA002 test pattern)extract_owner_repo/1parsesgit remote get-url origin(SSH + HTTPS forms)%{rule:, file:, severity:, reason:, action:, detail:}scan/2facade returns%{findings, total, by_severity, dispatch}matchingGitState.scan/1Hypatia.Rules.scan_baseline_health/2defdelegated from the facadeWhy these belong in Hypatia (not elsewhere)
Per the survey:
.git-private-farmhas no branch-protection authority — strictly mirroring + dispatch. Not a placement candidate.gitbot-fleetdocuments required-status-checks wiring (docs/BRANCH-PROTECTION-SETUP.md) but no bot actively enforces it; the safety triangle in Hypatia routes BH001/BH003 advisories tosustainabotfor owner action, and BH002 torhodibotfor follow-up PR creation.echidnabotis downstream ofproof_obligation.exonly — proves mathematical/logical properties of code, no stake in branch-protection / dep / CI-color invariants.Test plan
12 tests in
test/baseline_health_test.exs:[]cleanlytarget/andnode_modules/exclusionscan/2facade shape (:findings, :total, :by_severity, :dispatch)owner_repo:opt overrideFollow-ups (not in this PR)
mix hypatia.scanafter merge to surface BH001/BH002/BH003 findings across the 302 scanned repos, route via existing safety triangle.gitbot-fleetcould grow arhodibotrecipe for BH002 (auto-open follow-up PR with the migration TODO discharged) once a few BH002 findings have been triaged manually for confidence calibration.hypatia.fix.bh001that usesgh api -X PUTagainst a known-good policy doc.🤖 Generated with Claude Code