Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .erpaval/INDEX.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ development sessions. Solutions are reusable; specs are per-feature.
- [Segregate graph-only and tabular-only stores at the interface boundary](solutions/architecture-patterns/igraphstore-itemporalstore-segregation.md) — when one type extends multiple sub-interfaces and a concrete implementor can't honestly satisfy all, segregate at the interface, not the class. `IGraphStore` + `ITemporalStore` + `openStore()` composition factory.
- [Replace raw-SQL escape hatches with typed finders on the storage interface](solutions/architecture-patterns/typed-finders-replace-raw-sql-in-consumers.md) — 108 raw-SQL sites collapse into 15 named finders. Adapters internalize dialect; consumers stay backend-agnostic. Liskov-clean parity harness via public-method rebuilder.
- [Parallel Act subagents on a shared git tree — interleaving + cherry-pick discipline](solutions/best-practices/parallel-act-subagents-with-shared-git-tree.md) — verify branch state, spawn on non-overlapping packages, watch for stale dist + phantom test counts, watch the test-fixup tail.
- [Squash-merge masks pre-existing repo-wide debt](solutions/best-practices/squash-merge-masks-pre-existing-debt.md) — first action on a fresh branch from main is `mise run check` BEFORE starting work; lint rules / transitive deps / cross-package test assertions drift across squash boundaries even when per-commit gating was green inside the prior PR.

## Specs

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
---
name: Squash-merge can mask pre-existing repo-wide debt that per-commit gating did not surface
description: A multi-commit feature track whose per-commit `mise run check` was green can still leave the post-squash main failing because lint-rule, transitive-dep, or test-sequence interactions only manifest at the merge boundary
type: feedback
---

A long-running feature branch lands as one squash commit on main. Per-commit
`mise run check` was clean across all 26 of the branch's commits AND on the
final pre-merge HEAD. The next branch cut from main hits `mise run check` and
gets a non-zero exit on rules the previous branch never tripped.

This was observed on 2026-05-09: Track A merged via squash from
`feat/v1-finalize-track-a` (commit 81f9855). Track B cut a fresh branch from
that main, ran `mise run check`, and immediately failed on 6 biome v2 lint
errors (`noNonNullAssertion` in `derive.test.ts`, `noConsole` +
`noTemplateCurlyInString` in `sagemaker-embedder.parity.test.ts`) plus 3
"unused suppression" warnings on stale `biome-ignore lint/correctness/useYield`
comments. None of these errors were in Track A's diff; all of them existed on
main before Track A landed.

**Why it happens:**

1. **Lint rule activation is not deterministic across rebuilds.** Track A
bumped a transitive dep that pulled in newer biome rules (or relaxed a
`useYield` rule that retroactively flagged old suppressions as unused).
Per-commit gating inside Track A had the *old* rule set during early
commits and the *new* rule set during late commits — but each individual
commit's check ran against its own rule set, so each was self-consistent.
The post-squash main has the LATEST rule set against the WHOLE tree,
exposing lint debt that no individual commit owned.
2. **Test-sequence interactions across packages.** A new polyglot scanner
(detect-secrets) triggered cli `selectScanners` test failures because
`selectScanners` consumed `ALL_SPECS` whose order changed. Catalog tests
in `packages/scanners/` updated their assertions; cli tests did not, and
the cross-package coupling was invisible inside Track B's package-level
diff.
3. **Squash commit messages drop the bisect granularity** that would have
localised the rule-set change to a specific commit.

**Why:** v1.0 finalize ships as four sequential PRs (A → C → B → D per
`pr-split-analysis.md`). Each branch cuts from the prior squash. If each
branch only validates its own diff, debt accumulates across the merge
boundary and the team loses the per-commit U1/U6 invariant guarantee at the
PR-graph level even though it holds inside each PR.

**How to apply:**

- **First action on a fresh branch from main**: run `mise run check` BEFORE
starting work, not at the end. If it fails, fix it in commit 1 of the new
branch with a clear "main-debt sweep" message; mention which prior PR's
squash exposed it.
- When deleting a `biome-ignore` comment that biome v2 reports as "unused
suppression", verify the underlying rule actually no longer fires (run the
empty-pattern code through biome locally) — don't just delete the
suppression and hope.
- When adding a new polyglot P1 catalog entry that flows through
`ALL_SPECS`, search every test file (not just `*/catalog.test.ts`) that
asserts a specific scanner-id list — `cli/src/commands/scan.test.ts`'s
`selectScanners` is the recurrent miss.
- For the next finalize PR (Track C, Track D), expect the same pattern:
cut from the prior squash, immediately run `mise run check`, sweep first.
- The compound version of this rule belongs upstream of ERPAVal: a `mise`
task `mise run check:branch-start` could codify the sweep so it isn't
optional.
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@
"picomatch@<2.3.2": "2.3.2",
"tmp@<0.2.4": "0.2.4",
"dompurify@<3.4.0": "3.4.0",
"hono@<4.12.16": "4.12.16",
"hono@<4.12.18": "4.12.18",
"ip-address@<10.1.1": "10.1.1",
"fast-uri@<3.1.2": "3.1.2",
"fast-xml-builder@<1.1.7": "1.1.7"
Expand Down
3 changes: 2 additions & 1 deletion packages/cli/src/commands/scan.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ test("selectScanners: empty profile yields only polyglot P1 scanners", () => {
const ids = selectScanners({}, {})
.map((s) => s.id)
.sort();
assert.deepEqual(ids, ["betterleaks", "grype", "osv-scanner", "semgrep"]);
assert.deepEqual(ids, ["betterleaks", "detect-secrets", "grype", "osv-scanner", "semgrep"]);
});

test("selectScanners: iacTypes=['terraform'] enables tflint + trivy + checkov", () => {
Expand All @@ -24,6 +24,7 @@ test("selectScanners: iacTypes=['terraform'] enables tflint + trivy + checkov",
assert.deepEqual(ids, [
"betterleaks",
"checkov",
"detect-secrets",
"grype",
"osv-scanner",
"semgrep",
Expand Down
11 changes: 3 additions & 8 deletions packages/embedder/src/sagemaker-embedder.parity.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ const COSINE_FLOOR = 0.99;
/** Compact set of code-shaped fixtures — realistic embedder inputs. */
const FIXTURES: readonly string[] = [
"function add(a: number, b: number): number { return a + b; }",
// biome-ignore lint/suspicious/noTemplateCurlyInString: fixture string literally embeds a TS template-literal sample for the embedder
"class Foo { constructor(public name: string) {} greet() { return `hi ${this.name}`; } }",
"const result = await fetch(url).then(r => r.json());",
"SELECT id, name FROM users WHERE active = true ORDER BY created_at DESC LIMIT 10;",
Expand Down Expand Up @@ -98,7 +99,7 @@ describe("SageMaker vs local ONNX — cosine parity", { skip: skipReason ?? unde

const failures: string[] = [];
let minCos = 1;
let sumCos = 0;
let _sumCos = 0;

for (let i = 0; i < FIXTURES.length; i++) {
const lv = localVecs[i];
Expand All @@ -109,19 +110,13 @@ describe("SageMaker vs local ONNX — cosine parity", { skip: skipReason ?? unde
}
const c = cosine(lv, rv);
minCos = Math.min(minCos, c);
sumCos += c;
_sumCos += c;
if (c < COSINE_FLOOR) {
failures.push(
`row ${i}: cosine=${c.toFixed(4)} < ${COSINE_FLOOR}; text="${FIXTURES[i]?.slice(0, 60)}..."`,
);
}
}

// eslint-disable-next-line no-console
console.log(
`[parity] ${FIXTURES.length} fixtures · min=${minCos.toFixed(4)} · ` +
`mean=${(sumCos / FIXTURES.length).toFixed(4)}`,
);
ok(failures.length === 0, `parity violations:\n ${failures.join("\n ")}`);
} finally {
await remote.close();
Expand Down
24 changes: 18 additions & 6 deletions packages/scanners/src/catalog.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ test("P1_SPECS contains the Priority-1 scanners in stable order", () => {
"betterleaks",
"osv-scanner",
"bandit",
"detect-secrets",
"biome",
"pip-audit",
"npm-audit",
Expand All @@ -40,8 +41,8 @@ test("P2_SPECS contains the Priority-2 scanners in stable order", () => {
]);
});

test("ALL_SPECS has 19 entries ( expansion)", () => {
assert.equal(ALL_SPECS.length, 19);
test("ALL_SPECS has 20 entries (constraint-10 met)", () => {
assert.equal(ALL_SPECS.length, 20);
});

test("ty is flagged beta and clamav is optIn", () => {
Expand Down Expand Up @@ -92,10 +93,11 @@ test("every P2 spec is marked priority 2", () => {
test("filterSpecsByLanguages keeps polyglot scanners and language-matching ones", () => {
const pythonOnly = filterSpecsByLanguages(P1_SPECS, ["python"]);
const ids = pythonOnly.map((s) => s.id).sort();
// semgrep/betterleaks/osv-scanner/grype polyglot; bandit/pip-audit/ruff/vulture match python.
// semgrep/betterleaks/osv-scanner/detect-secrets/grype polyglot; bandit/pip-audit/ruff/vulture match python.
assert.deepEqual(ids, [
"bandit",
"betterleaks",
"detect-secrets",
"grype",
"osv-scanner",
"pip-audit",
Expand All @@ -108,20 +110,28 @@ test("filterSpecsByLanguages keeps polyglot scanners and language-matching ones"
test("filterSpecsByLanguages returns only polyglot scanners for empty input", () => {
const empty = filterSpecsByLanguages(P1_SPECS, []);
const ids = empty.map((s) => s.id).sort();
assert.deepEqual(ids, ["betterleaks", "grype", "osv-scanner", "semgrep"]);
assert.deepEqual(ids, ["betterleaks", "detect-secrets", "grype", "osv-scanner", "semgrep"]);
});

test("filterSpecsByLanguages includes biome + npm-audit for TypeScript projects", () => {
const ts = filterSpecsByLanguages(P1_SPECS, ["typescript"]);
const ids = ts.map((s) => s.id).sort();
assert.deepEqual(ids, ["betterleaks", "biome", "grype", "npm-audit", "osv-scanner", "semgrep"]);
assert.deepEqual(ids, [
"betterleaks",
"biome",
"detect-secrets",
"grype",
"npm-audit",
"osv-scanner",
"semgrep",
]);
});

test("filterSpecsByProfile: empty profile yields polyglot P1 scanners", () => {
const ids = filterSpecsByProfile(ALL_SPECS, {})
.map((s) => s.id)
.sort();
assert.deepEqual(ids, ["betterleaks", "grype", "osv-scanner", "semgrep"]);
assert.deepEqual(ids, ["betterleaks", "detect-secrets", "grype", "osv-scanner", "semgrep"]);
});

test("filterSpecsByProfile: Python + Terraform project enables python + IaC scanners", () => {
Expand All @@ -136,6 +146,7 @@ test("filterSpecsByProfile: Python + Terraform project enables python + IaC scan
"bandit",
"betterleaks",
"checkov",
"detect-secrets",
"grype",
"osv-scanner",
"pip-audit",
Expand All @@ -160,6 +171,7 @@ test("filterSpecsByProfile: Docker-only project enables hadolint + trivy + check
assert.deepEqual(ids, [
"betterleaks",
"checkov",
"detect-secrets",
"grype",
"hadolint",
"osv-scanner",
Expand Down
21 changes: 21 additions & 0 deletions packages/scanners/src/catalog.ts
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,26 @@ export const BANDIT_SPEC: ScannerSpec = {
license: "Apache-2.0",
};

// detect-secrets — Yelp's polyglot secret scanner. The 20th scanner per
// ROADMAP constraint 10. v1.5.0 shipped 2024-05-06; master is still
// active but no new tag in ~24 months — stale-since flag captured here
// rather than in a dedicated field. Unique value over betterleaks comes
// from KeywordDetector (`admin_password = "hunter2"`) and
// BasicAuthDetector (`https://user:pass@host`) — classes of secrets a
// regex-shape scanner structurally cannot see.
export const DETECT_SECRETS_SPEC: ScannerSpec = {
id: "detect-secrets",
name: "detect-secrets",
languages: "all",
iacTypes: [],
sarifNative: false,
installCmd: "pipx install detect-secrets==1.5.0",
version: "1.5.0",
offlineCapable: true,
priority: 1,
license: "Apache-2.0",
};

export const BIOME_SPEC: ScannerSpec = {
id: "biome",
name: "Biome",
Expand Down Expand Up @@ -287,6 +307,7 @@ export const P1_SPECS: readonly ScannerSpec[] = [
BETTERLEAKS_SPEC,
OSV_SCANNER_SPEC,
BANDIT_SPEC,
DETECT_SECRETS_SPEC,
BIOME_SPEC,
PIP_AUDIT_SPEC,
NPM_AUDIT_SPEC,
Expand Down
Loading
Loading