Skip to content

rules: add 7 CI/CD weak-point detectors learned from 2026-05-26 affinescript session #333

@hyperpolymath

Description

@hyperpolymath

Superseded by per-pattern child issues (2026-05-27)

This umbrella was split into 7 focused per-pattern child issues. Each child carries the original pattern's full spec, detection algorithm, real-world example, landed-fix reference, and acceptance checklist.


Context

A long debugging + remediation session on 2026-05-26 across affinescript, hypatia, and standards surfaced seven concrete CI/CD anti-patterns that Hypatia could have detected before they bit. Filing them here as a structured rule-set for lib/rules/. Each pattern lists: detection logic, affected files in this session as worked examples, suggested severity, and remediation guidance.

Priority ordering reflects estate-wide impact (highest impact first). Implement in priority order; each is independently shippable.


Pattern 1 — Brittle regex matching of documentation headings inside inline-python workflow scripts

Severity: high (estate-wide silent gate failure)

Detection (Hypatia.Rules.WorkflowAudit):

  • Parse run: | blocks in workflow YAML.
  • Inside Python python3 << 'PYEOF' … PYEOF blocks, find re.search(r'...') and re.match(r'...') calls whose pattern contains the literal token sequence that names a markdown section heading (e.g. [A-Z][a-z]+ [A-Z][a-z]+).
  • Flag any such regex that is not anchored to ^# or ^#{1,4}\s+ — the absence of heading anchoring means the regex will also match prose mentions of the same phrase, which silently swap which table the parser walks.

Worked example (this session):

  • standards/.github/workflows/governance-reusable.yml:179re.search(r'TypeScript [Ee]xemptions', line) was unanchored. Repos with a heading variant like ### TypeScript / JavaScript Exemptions (Approved) (affinescript) didn't match, so the parser fell through to a prose mention of "the TypeScript exemptions above" inside a different section and parsed the wrong table. Every PR in affinescript failed the governance / Language / package anti-pattern policy check for weeks before this session caught it.
  • Fix: standards#183 anchored the regex with ^#{1,4}\\s+.*TypeScript.*[Ee]xemption.

Remediation guidance to emit:

Anchor heading-detection regexes to ^#{1,4}\\s+ so prose mentions don't trigger table parsing. Add an inline comment naming the heading shape you intend to match.


Pattern 2 — Required-check depends on a package only listed in optionalDependencies

Severity: high (contradiction: required check that can't be installed)

Detection (Hypatia.Rules.CicdRules):

  • For each workflow job listed in branches/main/protection.required_status_checks.contexts:
    • Scan its run: steps. If any step does require(\"X\") (or imports X) and X appears in a sibling package.json's optionalDependencies block but not in dependencies or devDependencies, flag it.
  • The combination of "required" + "optional install" is a structural contradiction.

Worked example (this session):

  • affinescript/editors/vscode/out/extension.cjs runtime-requires @hyperpolymath/affine-vscode.
  • affinescript/editors/vscode/package.json lists it in optionalDependencies (so npm install doesn't 404 when the package is missing).
  • vscode-smoke was NOT required (correct), but the same pattern in a required check would be silently broken. Was masked here by continue-on-error: true (see Pattern 5).
  • Fix: affinescript#380 embedded the adapter source at codegen time so the runtime no longer needs the optional dep at all.

Remediation guidance to emit:

Either move the dep out of optionalDependencies so installs are reliable, or remove the runtime require (embed the source / inline the adapter / use a different lookup path).


Pattern 3 — Workflow has multiple cron: entries firing on the same day-of-week

Severity: medium (runner-cost waste, log noise)

Detection (Hypatia.Rules.CicdRulescicd_waste_detection):

  • Parse on.schedule[].cron strings.
  • Group by the day-of-week field (5th cron position). If 2+ entries fire on the same day-of-week (incl. *), flag with the entries' minutes/hours.
  • Special case: if one entry uses * (every day) and another targets a specific day-of-week with the same hour:minute, the specific-day entry is a strict subset and should be removed.

Worked examples (this session):

  • hypatia/.github/workflows/tests.yml had 0 3 * * * (daily) + 0 3 * * 1 (Monday) — same hour, Monday is a subset. Fix: hypatia#331 dropped the Monday-only cron.
  • hypatia/.github/workflows/verify-proofs.yml had 0 5 * * 1, 30 4 * * 1, 0 4 * * 1 — three Monday triggers inherited from pre-consolidation source workflows. Fix: hypatia#331 collapsed to a single 0 4 * * 1.

Remediation guidance to emit:

Collapse to a single cron unless the multiple triggers serve distinct purposes (e.g. different if: gates per trigger). If they're a consolidation artefact, drop the redundant entries.


Pattern 4 — Workflow triggers reference branches that don't exist in the repo

Severity: low (dead config, but signals maintenance drift)

Detection (Hypatia.Rules.StructuralDrift):

  • For each workflow file, collect on.push.branches, on.pull_request.branches, on.push.branches-ignore, etc.
  • Resolve them against the repo's actual branch list (via the git index or GitHub API).
  • Flag any branch name that:
    • Doesn't exist as a ref, AND
    • Isn't a glob pattern, AND
    • Isn't the default branch name (some teams keep master/main for migration symmetry — exempt those).

Worked examples (this session):

  • affinescript/.github/workflows/{semgrep,spark-theatre-gate}.ymlbranches: [main, master] while the repo uses main exclusively. Fix: affinescript#379 dropped master.
  • hypatia/.github/workflows/{tests,verify-proofs}.ymldevelop and master were dead. Fix: hypatia#331 dropped them.

Remediation guidance to emit:

Drop branch refs in workflow triggers that don't resolve to actual branches in this repo. Confirms intent and avoids ambiguity.


Pattern 5 — continue-on-error: true on a job, paired with documentation that says "remove this when X lands"

Severity: medium (gate-quality erosion; the workflow drifts further from green main-line)

Detection (Hypatia.Rules.WorkflowAudit):

  • For each job with continue-on-error: true:
    • Scan the job's leading comments for phrases like until #N lands, until #N closes, remove when #N.
    • If a referenced issue is now closed or merged (via GitHub API), flag as "stale continue-on-error mask".
  • Bonus: detect continue-on-error on a job whose name matches an entry in branch-protection's required_status_checks.contexts (structural contradiction).

Worked example (this session):

Remediation guidance to emit:

If the gating issue has closed, drop continue-on-error: true and let the job report accurately. If the issue is still open but the workflow logic has changed (e.g. root cause fixed differently), also drop and update the comment.


Pattern 6 — Read-only check workflow missing concurrency: block

Severity: low (runner-cost waste on rapid-push PRs)

Detection (Hypatia.Rules.WorkflowAudit):

  • For each workflow file:
    • If on: includes pull_request or push, AND no top-level concurrency: block, AND permissions: is read-all or only has read scopes, flag.
    • The read-only condition is the safety gate — cancel-in-progress: true is only safe when the workflow doesn't publish/mutate.

Worked examples (this session):

  • affinescript/.github/workflows/{ci,semgrep,stdlib-naming,spark-theatre-gate,workflow-linter}.yml all lacked concurrency blocks. Fix: affinescript#379 added the estate-standard {group: workflow-ref, cancel-in-progress: true} pattern.

Remediation guidance to emit:

Read-only check workflows should add the canonical block:

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

Skip for workflows that publish (release, JSR, npm) — superseding a publish is unsafe.


Pattern 7 — Stale issue references in workflow / source comments

Severity: low (documentation rot)

Detection (Hypatia.Rules.HonestCompletion):

  • Scan workflow comments + source comments for #N issue/PR references with phrasing patterns like:
    • until #N, pending #N, gated on #N, awaits #N, blocked by #N, remove when #N
  • For each, query the GitHub API: is #N closed/merged?
  • If yes, flag the comment as stale.

Worked example (this session):

Remediation guidance to emit:

The referenced issue (#N) is closed/merged. Update or delete this comment so the next reader doesn't waste time looking it up.


Implementation notes

  • All seven patterns map to existing lib/rules/ modules — no new top-level rule families.
  • Severity ordering: 1, 2 = high; 3, 5 = medium; 4, 6, 7 = low.
  • Each pattern includes a worked example from this session — useful as a fixture for the rule's unit test.
  • The rules should emit Hypatia findings with rule_module set to the matching module and severity per above.

🤖 Filed by Claude Code after the 2026-05-26 estate CI remediation session.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions