Skip to content

feat(rules): BaselineHealth (BH001-BH003) — detect red-baseline conditions estate-wide#316

Merged
hyperpolymath merged 2 commits into
mainfrom
feat/baseline-health-rules
May 26, 2026
Merged

feat(rules): BaselineHealth (BH001-BH003) — detect red-baseline conditions estate-wide#316
hyperpolymath merged 2 commits into
mainfrom
feat/baseline-health-rules

Conversation

@hyperpolymath
Copy link
Copy Markdown
Owner

Summary

Three new detection rules in lib/rules/baseline_health.ex, each addressing a failure mode that combined into the breakage of hyperpolymath/reasonably-good-token-vault main. The originating fix is r-g-t-v#89; these rules detect that class of failure across the estate so the next occurrence is caught before it lands.

  • BH001main branch protection has no required_status_checks (or empty contexts/checks). :critical. Dispatch: sustainabot :report. Strictly stronger than ERR-SEC-001 in cicd_rules.ex (which fires only when there's no protection at all); BH001 fires when protection exists but doesn't gate on CI — the exact gap that let r-g-t-v#82 merge red.
  • BH002 — Local Cargo.toml / package.json scan for lines where a dependency declaration carries a TODO/FIXME mentioning migration / rename / API / breaking / major. :high. Dispatch: rhodibot :open_followup_pr. Catches the exact pattern that left r-g-t-v#82 stuck: ureq = { version = "3" } # TODO: migrate to 3.x API.
  • BH003 — A workflow run targeting main has been failing for longer than threshold_hours (default 24). :critical. Dispatch: sustainabot :escalate. Distinguishes flakes (<24h) from real baseline outages.

Conventions matched

From dependabot_alerts.ex and git_state.ex:

  • System.cmd("curl", [...]) for GitHub API (per the HTTPoison-TLS comment in rules.ex:287)
  • Auth via GITHUB_TOKEN or HYPATIA_DISPATCH_PAT, clean [] return on missing token (matches DA001/DA002 test pattern)
  • extract_owner_repo/1 parses git remote get-url origin (SSH + HTTPS forms)
  • Finding shape: %{rule:, file:, severity:, reason:, action:, detail:}
  • scan/2 facade returns %{findings, total, by_severity, dispatch} matching GitState.scan/1
  • Hypatia.Rules.scan_baseline_health/2 defdelegated from the facade

Why these belong in Hypatia (not elsewhere)

Per the survey:

  • .git-private-farm has no branch-protection authority — strictly mirroring + dispatch. Not a placement candidate.
  • gitbot-fleet documents required-status-checks wiring (docs/BRANCH-PROTECTION-SETUP.md) but no bot actively enforces it; the safety triangle in Hypatia routes BH001/BH003 advisories to sustainabot for owner action, and BH002 to rhodibot for follow-up PR creation.
  • echidnabot is downstream of proof_obligation.ex only — proves mathematical/logical properties of code, no stake in branch-protection / dep / CI-color invariants.

Test plan

12 tests in test/baseline_health_test.exs:

  • no-token paths for BH001 and BH003 return [] cleanly
  • all four positive BH002 patterns (TODO migrate, FIXME rename, etc.)
  • benign-TODO negative case
  • subdirectory walking
  • target/ and node_modules/ exclusion
  • scan/2 facade shape (:findings, :total, :by_severity, :dispatch)
  • owner_repo: opt override
Excluding tags: [:verisim_data]
............
Finished in 0.1 seconds (0.1s async, 0.00s sync)
12 tests, 0 failures

Follow-ups (not in this PR)

  • Estate-wide sweep: run mix hypatia.scan after merge to surface BH001/BH002/BH003 findings across the 302 scanned repos, route via existing safety triangle.
  • gitbot-fleet could grow a rhodibot recipe for BH002 (auto-open follow-up PR with the migration TODO discharged) once a few BH002 findings have been triaged manually for confidence calibration.
  • Branch protection itself (the BH001 fix) is owner-level and remains manual; the eventual play is a Mix task hypatia.fix.bh001 that uses gh api -X PUT against a known-good policy doc.

🤖 Generated with Claude Code

…eline detection

Three new detection rules in lib/rules/baseline_health.ex, each
addressing a failure mode that combined into the breakage of
reasonably-good-token-vault main (see hyperpolymath/reasonably-good-
token-vault#89, the originating fix):

- BH001: `main` branch protection has no `required_status_checks`
  (or empty contexts/checks). Severity :critical. Action :report via
  sustainabot. Strictly stronger than the existing ERR-SEC-001
  "missing branch protection" check — that fires when there's no
  protection AT ALL; BH001 fires when protection exists but doesn't
  require CI to pass, which is the exact gap that let red-CI PRs
  reach main in r-g-t-v.

- BH002: Local Cargo.toml / package.json scan for lines where a
  dependency declaration carries a TODO/FIXME mentioning migration,
  rename, API, breaking, or major. Severity :high. Action
  :open_followup_pr via rhodibot. Catches the exact pattern that left
  r-g-t-v#82 stuck: `ureq = { version = "3" }  # TODO: migrate to
  3.x API` was the smell, and the migration TODO sat in a merged PR
  with no follow-up issue.

- BH003: A workflow run targeting main has been failing for longer
  than `threshold_hours` (default 24). Severity :critical. Action
  :escalate via sustainabot. Distinguishes flakes (<24h) from real
  baseline outages.

Conventions matched from existing rule modules:
- `System.cmd("curl", [...])` for GitHub API (per the HTTPoison TLS
  comment in rules.ex)
- `GITHUB_TOKEN` or `HYPATIA_DISPATCH_PAT` for auth, with clean `[]`
  return on missing token
- `extract_owner_repo/1` parses `git remote get-url origin`, mirroring
  dependabot_alerts.ex
- Findings shape: `%{rule:, file:, severity:, reason:, action:,
  detail:}`
- `scan/2` facade returns `%{findings, total, by_severity, dispatch}`
  matching `GitState.scan/1`
- `defdelegate scan_baseline_health(...)` added to the `Hypatia.Rules`
  facade

Testing: 12 tests in test/baseline_health_test.exs covering the
no-token paths, all four positive BH002 patterns, the benign-TODO
negative case, subdirectory walking, target/ and node_modules/
exclusion, the scan/2 facade shape, and the owner_repo override.

  Excluding tags: [:verisim_data]
  12 tests, 0 failures (mix test test/baseline_health_test.exs)

Propagation: once landed, these rules detect the originating-incident
class of failures estate-wide. Per the survey:
- .git-private-farm has no branch-protection authority — strictly
  mirroring/dispatch. Not a placement candidate.
- gitbot-fleet documents required-status-checks wiring but no bot
  enforces it; Hypatia's safety triangle routes BH001 advisories to
  sustainabot for owner action.
- echidnabot is downstream of `proof_obligation.ex` only — no stake
  in branch-protection / dep / CI-color invariants.
- ERR-SEC-001 in cicd_rules.ex covers the weaker case (no protection
  at all); BH001 is the strictly-stronger refinement.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown

ghost commented May 25, 2026

🔍 Hypatia Security Scan

Findings: 117 issues detected

Severity Count
🔴 Critical 0
🟠 High 16
🟡 Medium 101
View findings
[
  {
    "reason": "Docker reference in Nickel config -- Podman/Containerfile highly preferred (Docker permitted) (1 occurrences, CWE-1104)",
    "type": "ncl_docker_not_podman",
    "file": "/home/runner/work/hypatia/hypatia/.machine_readable/svc/k9/hypatia-metadata.k9.ncl",
    "action": "flag",
    "rule_module": "code_safety",
    "severity": "low"
  },
  {
    "reason": "unwrap() without prior check -- DoS via panic (2 occurrences, CWE-754)",
    "type": "unwrap_without_check",
    "file": "/home/runner/work/hypatia/hypatia/cli/src/commands/batch.rs",
    "action": "flag",
    "rule_module": "code_safety",
    "severity": "low"
  },
  {
    "reason": "unwrap_or(0) with dangerous default (1 occurrences, CWE-754)",
    "type": "unwrap_dangerous_default",
    "file": "/home/runner/work/hypatia/hypatia/cli/src/commands/batch.rs",
    "action": "flag",
    "rule_module": "code_safety",
    "severity": "low"
  },
  {
    "reason": "unwrap() without prior check -- DoS via panic (1 occurrences, CWE-754)",
    "type": "unwrap_without_check",
    "file": "/home/runner/work/hypatia/hypatia/cli/src/commands/scan.rs",
    "action": "flag",
    "rule_module": "code_safety",
    "severity": "low"
  },
  {
    "reason": "unwrap_or(0) with dangerous default (1 occurrences, CWE-754)",
    "type": "unwrap_dangerous_default",
    "file": "/home/runner/work/hypatia/hypatia/cli/src/commands/scan.rs",
    "action": "flag",
    "rule_module": "code_safety",
    "severity": "low"
  },
  {
    "reason": "unwrap() without prior check -- DoS via panic (2 occurrences, CWE-754)",
    "type": "unwrap_without_check",
    "file": "/home/runner/work/hypatia/hypatia/cli/src/commands/fleet.rs",
    "action": "flag",
    "rule_module": "code_safety",
    "severity": "low"
  },
  {
    "reason": "unwrap() without prior check -- DoS via panic (2 occurrences, CWE-754)",
    "type": "unwrap_without_check",
    "file": "/home/runner/work/hypatia/hypatia/cli/src/output.rs",
    "action": "flag",
    "rule_module": "code_safety",
    "severity": "low"
  },
  {
    "reason": "unwrap() without prior check -- DoS via panic (2 occurrences, CWE-754)",
    "type": "unwrap_without_check",
    "file": "/home/runner/work/hypatia/hypatia/cli/build.rs",
    "action": "flag",
    "rule_module": "code_safety",
    "severity": "low"
  },
  {
    "reason": "unwrap() without prior check -- DoS via panic (3 occurrences, CWE-754)",
    "type": "unwrap_without_check",
    "file": "/home/runner/work/hypatia/hypatia/fixer/src/main.rs",
    "action": "flag",
    "rule_module": "code_safety",
    "severity": "low"
  },
  {
    "reason": "expect() in hot path (5 occurrences, CWE-754)",
    "type": "expect_in_hot_path",
    "file": "/home/runner/work/hypatia/hypatia/fixer/src/scanner.rs",
    "action": "flag",
    "rule_module": "code_safety",
    "severity": "low"
  }
]

Powered by Hypatia Neurosymbolic CI/CD Intelligence

@hyperpolymath hyperpolymath enabled auto-merge (squash) May 25, 2026 22:14
@github-actions
Copy link
Copy Markdown

ghost commented May 26, 2026

🔍 Hypatia Security Scan

Findings: 113 issues detected

Severity Count
🔴 Critical 0
🟠 High 6
🟡 Medium 107
View findings
[
  {
    "reason": "Docker reference in Nickel config -- Podman/Containerfile highly preferred (Docker permitted) (1 occurrences, CWE-1104)",
    "type": "ncl_docker_not_podman",
    "file": "/home/runner/work/hypatia/hypatia/.machine_readable/svc/k9/hypatia-metadata.k9.ncl",
    "action": "flag",
    "rule_module": "code_safety",
    "severity": "low"
  },
  {
    "reason": "unwrap() without prior check -- DoS via panic (2 occurrences, CWE-754)",
    "type": "unwrap_without_check",
    "file": "/home/runner/work/hypatia/hypatia/cli/src/commands/batch.rs",
    "action": "flag",
    "rule_module": "code_safety",
    "severity": "low"
  },
  {
    "reason": "unwrap_or(0) with dangerous default (1 occurrences, CWE-754)",
    "type": "unwrap_dangerous_default",
    "file": "/home/runner/work/hypatia/hypatia/cli/src/commands/batch.rs",
    "action": "flag",
    "rule_module": "code_safety",
    "severity": "low"
  },
  {
    "reason": "unwrap() without prior check -- DoS via panic (1 occurrences, CWE-754)",
    "type": "unwrap_without_check",
    "file": "/home/runner/work/hypatia/hypatia/cli/src/commands/scan.rs",
    "action": "flag",
    "rule_module": "code_safety",
    "severity": "low"
  },
  {
    "reason": "unwrap_or(0) with dangerous default (1 occurrences, CWE-754)",
    "type": "unwrap_dangerous_default",
    "file": "/home/runner/work/hypatia/hypatia/cli/src/commands/scan.rs",
    "action": "flag",
    "rule_module": "code_safety",
    "severity": "low"
  },
  {
    "reason": "unwrap() without prior check -- DoS via panic (2 occurrences, CWE-754)",
    "type": "unwrap_without_check",
    "file": "/home/runner/work/hypatia/hypatia/cli/src/commands/fleet.rs",
    "action": "flag",
    "rule_module": "code_safety",
    "severity": "low"
  },
  {
    "reason": "unwrap() without prior check -- DoS via panic (2 occurrences, CWE-754)",
    "type": "unwrap_without_check",
    "file": "/home/runner/work/hypatia/hypatia/cli/src/output.rs",
    "action": "flag",
    "rule_module": "code_safety",
    "severity": "low"
  },
  {
    "reason": "unwrap() without prior check -- DoS via panic (2 occurrences, CWE-754)",
    "type": "unwrap_without_check",
    "file": "/home/runner/work/hypatia/hypatia/cli/build.rs",
    "action": "flag",
    "rule_module": "code_safety",
    "severity": "low"
  },
  {
    "reason": "unwrap() without prior check -- DoS via panic (3 occurrences, CWE-754)",
    "type": "unwrap_without_check",
    "file": "/home/runner/work/hypatia/hypatia/fixer/src/main.rs",
    "action": "flag",
    "rule_module": "code_safety",
    "severity": "low"
  },
  {
    "reason": "expect() in hot path (5 occurrences, CWE-754)",
    "type": "expect_in_hot_path",
    "file": "/home/runner/work/hypatia/hypatia/fixer/src/scanner.rs",
    "action": "flag",
    "rule_module": "code_safety",
    "severity": "low"
  }
]

Powered by Hypatia Neurosymbolic CI/CD Intelligence

@hyperpolymath hyperpolymath merged commit 5ecccdb into main May 26, 2026
35 checks passed
@hyperpolymath hyperpolymath deleted the feat/baseline-health-rules branch May 26, 2026 01:12
hyperpolymath added a commit that referenced this pull request May 26, 2026
… detectors (#320)

## Summary

Extends `BaselineHealth` (BH001-BH003, landed in #316) with four new
rules captured from a follow-on estate-wide session.

| Rule | Detects | Severity | Dispatch |
|---|---|---|---|
| **BH004** | Dead action SHA pin in workflow YAML (`uses: <repo>@<sha>`
where SHA doesn't exist upstream) | `:critical` | rhodibot (auto-fix PR,
conf 0.93) |
| **BH005** | Required status check is push-only (workflow has no
`pull_request` trigger) → every PR permanently `BLOCKED` | `:high` |
sustainabot (conf 0.85) |
| **BH006** | Required check name no longer corresponds to any workflow
that emits (stale entry in `required_status_checks`) | `:high` |
sustainabot (conf 0.85) |
| **BH007** | Commits signed but `verified: false / reason: bad_email` —
author email isn't a UID on the registered GPG/SSH key | `:high` |
sustainabot (conf 0.80) |

## Why these now

Each was a one-line config defect this session uncovered as it
propagated across many repos:

- **BH004** — \`actions/upload-artifact@65c79d7f...\` (a hallucinated
SHA) propagated by template scaffolding into ~62 workflow files across
10 repos. Every workflow run using it died at action-resolution. Fixed
in PRs across \`rsr-template-repo\`, \`me-dialect\`, \`ambientops\`.
- **BH005** — an org-wide \`required_status_checks\` sweep against 330
repos locked push-context-only checks (\`Dependabot\`,
\`mirror-gitlab\`, \`trigger-boj\`, etc.). Result: every PR in 290+
repos went to \`BLOCKED / MERGEABLE\` because those workflows never
report on PRs.
- **BH006** — \`rsr-template-repo\` had a required list full of names
like \`analysis\`, \`dispatch\`, \`mirror-codeberg\` that no longer
corresponded to any active workflow. Distinct from BH005: BH005 = "ran
on push, not PR"; BH006 = "didn't run anywhere recently".
- **BH007** — commits in this very session showed \`bad_email\` because
the noreply UID was on the local GPG key but missing from the
GitHub-registered public key. Branch protection requiring signed commits
silently blocked all PRs.

## Tests

\`mix test test/baseline_health_test.exs\` → **18 tests / 0 failures**
(was 12).
- Each new rule has a no-token regression test (returns \`[]\` cleanly).
- BH004 has a positive fixture verifying that nested monorepo scaffolds
are NOT scanned (only repo-root \`.github/workflows/\`).
- BH002's existing positive tests still pass unchanged.

## Module summary

\`lib/rules/baseline_health.ex\` grows from 438 → 877 lines (+458 /
-19).
Module docstring updated to cover BH001-BH007 with provenance notes for
each.

Companion PRs to follow: \`workflow_hardening.ex\` (zizmor/actionlint
cluster), \`supply_chain.ex\` (Scorecard/SLSA/OWASP cluster),
\`branch_protection.ex\` (CIS/NIST cluster).

🤖 Generated with [Claude Code](https://claude.com/claude-code)
hyperpolymath added a commit that referenced this pull request May 26, 2026
## Summary

Adds delegates on \`Hypatia.Rules\` for the rule families landed in PRs
#320, #321, #322, #323. The new modules were independently usable but
invisible from the top-level facade; this PR closes that gap.

## New facade surface

| Function | Delegates to | Rules |
|---|---|---|
| \`scan_baseline_health/2\` | \`BaselineHealth.scan\` (existing;
docstring updated for BH007) | BH001-BH007 |
| \`scan_workflow_hardening/2\` | \`WorkflowHardening.scan\` |
WH001-WH012 |
| \`scan_supply_chain/2\` | \`SupplyChain.scan\` | SC001-SC011 |
| \`scan_branch_protection/2\` | \`BranchProtection.scan\` (API-backed;
takes \`owner, repo\`) | BP001-BP007 |
| \`scan_all_estate_policies/2\` | — runs all four families and merges
results | 34 rules total |

## Why combined-scan helper

Callers (\`pattern_analyzer\`, \`triangle_router\`,
\`fleet_dispatcher\`) already treat \`Hypatia.Rules\` as the entrypoint.
Without these delegates the new modules are reachable only via
fully-qualified names, which works but inverts the facade pattern
adopted in #316.

## RE001-RE010 (PR #325) not included

PR #325 (ResearchExtensions) is still OPEN at the time of this PR.
Adding its alias here would break compilation on \`main\` for the window
between this PR landing and #325 landing. A follow-up commit will wire
\`ResearchExtensions\` once #325 merges — tracked inline at the alias
block as a \`# wired in follow-up\` comment.

## Test plan

- Existing 18-test \`BaselineHealth\` suite still passes (untouched).
- Module compile-checks pass for the four families + facade.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant