Skip to content

feat: close the Scorecard alert-lifecycle loop — #261 #262 #263 (#260)#264

Merged
hyperpolymath merged 3 commits into
mainfrom
fix/scorecard-loop-closer-260
May 16, 2026
Merged

feat: close the Scorecard alert-lifecycle loop — #261 #262 #263 (#260)#264
hyperpolymath merged 3 commits into
mainfrom
fix/scorecard-loop-closer-260

Conversation

@hyperpolymath
Copy link
Copy Markdown
Owner

@hyperpolymath hyperpolymath commented May 16, 2026

Lands all three structural fixes behind estate-wide recurring Scorecard findings. Parent: #260.

#262 — SLSA pin-exemption (was actively harmful)

Removed the slsa-github-generator → <sha> mappings (SHA-pinning it breaks SLSA provenance — it self-verifies github.ref). Added SecurityErrors.@pin_exempt registry; pin_action/1{:exempt, rationale}; check_unpinned_actions emits :pin_exempt_accepted (accept, :info).

#261 — Effective-vs-nominal SAST

ScorecardIngestor.check_sast now verifies the CodeQL matrix actually scans a language the repo has (or actions); nominal-only CodeQL → finding recommending language: actions. (Generalises modshells #72.)

#263Hypatia.ScorecardReconciler (the loop-closer)

The missing Sense(authoritative)→Classify→Act→Verify→Learn:

  • classify/2 pure 4-axis taxonomy → one unambiguous action; unknown rules escalate, never silently drop.
  • fingerprint/3 excludes line numbers → dismissals survive edits (the recurrence killer).
  • reconcile/3 fetches live alerts, dismisses non-actionable/exempt with rationale, threads registry via map_reduce, persists once.
  • verify/2 — a registry-dismissed finding open again is a recurrence defect, surfaced not re-dismissed.
  • Registry → ~/.git-private-farm/ (offline-survivable durability KPI).
  • mix hypatia.reconcile owner/repo [--dry-run|--verify].

Verification

mix compile clean (pre-existing unrelated warnings only). mix test reconciler+workflow_audit+security_errors: 47/47 green (+17 new).

Honesty note

While writing the reconciler I caught a real bug in my own first draft — Registry.record/3 is pure but the loop discarded its return, so every decision after the first would have been lost. Fixed with Enum.map_reduce before any test ran; regression covered by the registry round-trip test.

Follow-up (not blocking)

Auto-merge of CI-passing low-risk :fix PRs (per autonomy decision) — reconciler currently marks :fix_requested; wiring to the existing fixer/DirectGithubPr is the next increment.

Refs #260
Refs #261
Refs #262
Refs #263

🤖 Generated with Claude Code

hyperpolymath and others added 2 commits May 16, 2026 21:35
Two of the three structural defects behind estate-wide recurring
Scorecard findings (parent #260):

#262 — SHA-pinning slsa-framework/slsa-github-generator is HARMFUL:
it self-verifies its own github.ref and must stay on a semver tag,
or SLSA provenance is invalid. Removed the harmful SHA mapping from
SecurityErrors.@sha_pins and WorkflowAudit.@known_good_shas, added a
first-class @pin_exempt registry, and made pin_action/1 return
{:exempt, rationale}. check_unpinned_actions now emits
:pin_exempt_accepted (accept-with-rationale, severity :info) instead
of :pin_sha, so the reconciler dismisses the alert rather than
applying a provenance-breaking 'fix'.

#261 — ScorecardIngestor.check_sast only grepped for the string
'codeql' (presence != efficacy), so a codeql.yml pointed at a
language the repo lacks (modshells #72: javascript-typescript on an
Ada/Scheme repo) passed while Scorecard reported '0 commits checked'.
check_sast now verifies effectiveness: a CodeQL workflow whose
language matrix contains no language the repo has and lacks 'actions'
yields a Nominal-only SAST finding recommending 'language: actions'.

Tests: +7 (security_errors, workflow_audit); 37/37 green. Compiles
clean (pre-existing unrelated warnings only).

Refs #260
Refs #261
Refs #262

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds Hypatia.ScorecardReconciler + Registry + mix hypatia.reconcile —
the missing Sense(authoritative)->Classify->Act->Verify->Learn loop.

- classify/2: pure 4-axis taxonomy -> one unambiguous lifecycle action
  (:dismiss_info | :dismiss_accept | :fix | :open_escalate). Unknown
  rules escalate, never silently drop (escaped-defect KPI guard).
- fingerprint/3: repo|tool|rule|basename(path) — excludes line numbers
  so a dismissal survives edits and the alert does not re-open (the
  recurrence killer).
- reconcile/3: fetches LIVE open alerts (curl+GITHUB_TOKEN, same pattern
  as DependabotAlerts), dismisses non-actionable/exempt with rationale,
  threads the registry via map_reduce, persists once.
- verify/2: a registry-dismissed finding that is open again is a
  recurrence DEFECT, surfaced not silently re-dismissed.
- Registry persists to ~/.git-private-farm/ (offline-survivable
  'rebuild from DNA' substrate).

SLSA pin-exemption (#262) is wired into classify via
SecurityErrors.pin_exempt?/1 so PinnedDependenciesID on the SLSA
generator -> :dismiss_accept (never a provenance-breaking fix).

Tests: +10; 47/47 green. Compiles clean.

Refs #260
Refs #263

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@hyperpolymath hyperpolymath changed the title fix(rules): SLSA pin-exemption + effective-vs-nominal SAST (#260) feat: close the Scorecard alert-lifecycle loop — #261 #262 #263 (#260) May 16, 2026
@github-actions
Copy link
Copy Markdown

🔍 Hypatia Security Scan

Findings: 11 issues detected

Severity Count
🔴 Critical 2
🟠 High 2
🟡 Medium 7

⚠️ Action Required: Critical security issues found!

View findings
[
  {
    "reason": "Issue in quality.yml",
    "type": "missing_workflow",
    "file": "quality.yml",
    "action": "create",
    "rule_module": "workflow_audit",
    "severity": "high"
  },
  {
    "reason": "Issue in mirror.yml",
    "type": "missing_workflow",
    "file": "mirror.yml",
    "action": "create",
    "rule_module": "workflow_audit",
    "severity": "high"
  },
  {
    "reason": "believe_me undermines formal verification (2 occurrences, CWE-704)",
    "type": "believe_me",
    "file": "/home/runner/work/hypatia/hypatia/src/abi/RuleEngine.idr",
    "action": "flag",
    "rule_module": "code_safety",
    "severity": "critical"
  },
  {
    "reason": "Nickel file missing SPDX-License-Identifier header (1 occurrences, CWE-1104)",
    "type": "ncl_missing_spdx",
    "file": "/home/runner/work/hypatia/hypatia/configs/config.ncl",
    "action": "flag",
    "rule_module": "code_safety",
    "severity": "medium"
  },
  {
    "reason": "unsafe block -- requires SAFETY comment (22 occurrences, CWE-676)",
    "type": "unsafe_block",
    "file": "/home/runner/work/hypatia/hypatia/clients/rust/hypatia-client/src/ffi.rs",
    "action": "flag",
    "rule_module": "code_safety",
    "severity": "medium"
  },
  {
    "reason": "as_ptr exposes raw pointer that may dangle or alias unsafely (10 occurrences, CWE-676)",
    "type": "as_ptr",
    "file": "/home/runner/work/hypatia/hypatia/clients/rust/hypatia-client/src/ffi.rs",
    "action": "flag",
    "rule_module": "code_safety",
    "severity": "medium"
  },
  {
    "reason": "expect() in hot path (1 occurrences, CWE-754)",
    "type": "expect_in_hot_path",
    "file": "/home/runner/work/hypatia/hypatia/adapters/src/codeberg.rs",
    "action": "flag",
    "rule_module": "code_safety",
    "severity": "medium"
  },
  {
    "reason": "expect() in hot path (1 occurrences, CWE-754)",
    "type": "expect_in_hot_path",
    "file": "/home/runner/work/hypatia/hypatia/adapters/src/radicle.rs",
    "action": "flag",
    "rule_module": "code_safety",
    "severity": "medium"
  },
  {
    "line": 35,
    "reason": "Secret found: Password",
    "type": "secret_detected",
    "file": "/home/runner/work/hypatia/hypatia/.hypatia-exemptions.md",
    "action": "revoke_rotate_and_purge",
    "rule_module": "security_errors",
    "severity": "critical"
  },
  {
    "reason": "1 workflow(s) with tag-pinned (not SHA-pinned) actions in hypatia",
    "type": "DependencyPinning",
    "file": "/home/runner/work/hypatia/hypatia",
    "action": "auto_fix",
    "rule_module": "scorecard",
    "severity": "medium",
    "remediation": "Pin GitHub Actions and Docker base images by SHA hash.",
    "scorecard_check": "Pinned-Dependencies"
  }
]

Powered by Hypatia Neurosymbolic CI/CD Intelligence

hyperpolymath added a commit that referenced this pull request May 16, 2026
…264) (#266)

Two **pre-existing** CI reds on `main` (not introduced by #264) —
root-caused per the resolve-at-source rule:

1. **Build AsciiDoc**: `gem install asciidoctor --version ${{
env.ASCIIDOCTOR_VERSION }}` — `ASCIIDOCTOR_VERSION` is defined nowhere,
so the flag had no value → `Gem::OptionParser::MissingArgument`. Dropped
the dangling flag.
2. **Validate A2ML manifests**: external `a2ml-validate-action` had a
broken multi-line comment → bash ran `ame/project …` → **exit 127**,
failing every commit estate-wide. Fixed at source
(a2ml-validate-action#11, merged) and re-pinned here to `f8517bb`.

Shipped as a CI-fix PR separate from the #264 content per workflow
convention. Once merged, #264 is rebased and goes green.

Refs #260

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown

🔍 Hypatia Security Scan

Findings: 11 issues detected

Severity Count
🔴 Critical 2
🟠 High 2
🟡 Medium 7

⚠️ Action Required: Critical security issues found!

View findings
[
  {
    "reason": "Issue in quality.yml",
    "type": "missing_workflow",
    "file": "quality.yml",
    "action": "create",
    "rule_module": "workflow_audit",
    "severity": "high"
  },
  {
    "reason": "Issue in mirror.yml",
    "type": "missing_workflow",
    "file": "mirror.yml",
    "action": "create",
    "rule_module": "workflow_audit",
    "severity": "high"
  },
  {
    "reason": "believe_me undermines formal verification (2 occurrences, CWE-704)",
    "type": "believe_me",
    "file": "/home/runner/work/hypatia/hypatia/src/abi/RuleEngine.idr",
    "action": "flag",
    "rule_module": "code_safety",
    "severity": "critical"
  },
  {
    "reason": "Nickel file missing SPDX-License-Identifier header (1 occurrences, CWE-1104)",
    "type": "ncl_missing_spdx",
    "file": "/home/runner/work/hypatia/hypatia/configs/config.ncl",
    "action": "flag",
    "rule_module": "code_safety",
    "severity": "medium"
  },
  {
    "reason": "unsafe block -- requires SAFETY comment (22 occurrences, CWE-676)",
    "type": "unsafe_block",
    "file": "/home/runner/work/hypatia/hypatia/clients/rust/hypatia-client/src/ffi.rs",
    "action": "flag",
    "rule_module": "code_safety",
    "severity": "medium"
  },
  {
    "reason": "as_ptr exposes raw pointer that may dangle or alias unsafely (10 occurrences, CWE-676)",
    "type": "as_ptr",
    "file": "/home/runner/work/hypatia/hypatia/clients/rust/hypatia-client/src/ffi.rs",
    "action": "flag",
    "rule_module": "code_safety",
    "severity": "medium"
  },
  {
    "reason": "expect() in hot path (1 occurrences, CWE-754)",
    "type": "expect_in_hot_path",
    "file": "/home/runner/work/hypatia/hypatia/adapters/src/codeberg.rs",
    "action": "flag",
    "rule_module": "code_safety",
    "severity": "medium"
  },
  {
    "reason": "expect() in hot path (1 occurrences, CWE-754)",
    "type": "expect_in_hot_path",
    "file": "/home/runner/work/hypatia/hypatia/adapters/src/radicle.rs",
    "action": "flag",
    "rule_module": "code_safety",
    "severity": "medium"
  },
  {
    "line": 35,
    "reason": "Secret found: Password",
    "type": "secret_detected",
    "file": "/home/runner/work/hypatia/hypatia/.hypatia-exemptions.md",
    "action": "revoke_rotate_and_purge",
    "rule_module": "security_errors",
    "severity": "critical"
  },
  {
    "reason": "1 workflow(s) with tag-pinned (not SHA-pinned) actions in hypatia",
    "type": "DependencyPinning",
    "file": "/home/runner/work/hypatia/hypatia",
    "action": "auto_fix",
    "rule_module": "scorecard",
    "severity": "medium",
    "remediation": "Pin GitHub Actions and Docker base images by SHA hash.",
    "scorecard_check": "Pinned-Dependencies"
  }
]

Powered by Hypatia Neurosymbolic CI/CD Intelligence

@hyperpolymath hyperpolymath merged commit bd8f363 into main May 16, 2026
34 checks passed
@hyperpolymath hyperpolymath deleted the fix/scorecard-loop-closer-260 branch May 16, 2026 21:59
hyperpolymath added a commit that referenced this pull request May 18, 2026
…ion (#265) (#271)

Refs #260 / #263 / #265 — **do not auto-close**.

Follow-up increment to merged #264 (which explicitly deferred this).

### Problem (natsci-studio live calibration)
`BranchProtectionID` / `CodeReviewID` were `:open_escalate` — correct
(never silently dropped) but suboptimal: they are
*repository-configuration-actionable* via the GitHub settings API, not
code fixes and not non-actionable. natsci-studio dry-run: alerts #1
BranchProtection / #3 CodeReview should auto-remediate under full-auto.

### Change
Third, narrow action class between `:fix` (code) and `:open_escalate`:
- `classify/2`: `BranchProtectionID`/`CodeReviewID` → `:fix_settings`.
- `reconcile/3`: new `:policy` opt (default `:full_auto`);
`do_fix_settings/3` is an idempotent branch-protection PUT requiring ≥1
PR review on the default branch (satisfies both checks). `:conservative`
escalates instead.
- `mix hypatia.reconcile … --conservative`; summary gains
`settings_remediated`/`settings_actionable`.

### Verification
`mix compile` clean. Full suite **809/809** (+3 new tests).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant