Skip to content

test: Falsifiability A — skip-list ablation experiment (mache-3509aa)#350

Merged
jamestexas merged 1 commit into
mainfrom
test/falsifiability-a-skip-list-ablation
May 8, 2026
Merged

test: Falsifiability A — skip-list ablation experiment (mache-3509aa)#350
jamestexas merged 1 commit into
mainfrom
test/falsifiability-a-skip-list-ablation

Conversation

@jamestexas
Copy link
Copy Markdown
Contributor

Summary

ADR-0013 named two first-class falsifiability experiments. This is A — the headline.

Hypothesis: with `_lsp_refs` populated, dropping the `dead_code` skip-list flags zero NEW dead constructs that LSP indexed. Any new finding must be a method on a type LSP didn't see (genuine coverage gap — wedge case 1 from the ADR).

Procedure

  1. Run `dead_code` in production form (skip-list intact). Findings A.
  2. Run `dead_code` with the `skipped` CTE neutralized. Findings B.
  3. Diff = B \ A — newly flagged dead.
  4. For each row in diff, look up its `node_id` in `_lsp_refs`. If LSP HAS a reference to that def, the design is broken — the alive-check's binding arm should have made it alive.

Implementation: stripSkippedCTE

Walks the rule body with paren-depth balancing to locate the matching close paren of `skipped AS (...)` — the inline SQL has nested parens for the `IN ('main','init',...)` list, so naive regex matching fails. Replaces the whole CTE with an empty form: `skipped AS (SELECT '' AS node_id WHERE 0)`.

Tests

  • `TestStripSkippedCTE_RealRuleBody` — introspects the production registry, transforms the rule, asserts the skip-list (`'ServeHTTP'` marker) is gone AND both queries are still valid SQL. Future refactors that move the skip-list out of an inline CTE fail this test loudly.
  • `TestFalsifiabilityA_SyntheticHarness` — fixture with two methods on `MyType`: `Read` (LSP indexed) and `Quux` (LSP didn't). `Read` is in the skip-list and has an LSP binding row → not flagged in A; ablation drops the skip-list, alive-check's binding arm matches `target_node_id` → not flagged in B either. Diff doesn't contain `Read` — the headline. `Quux` is dead in both paths.
  • `TestFalsifiabilityA_IntegrationOnLeylineParse` — gated on `MACHE_FALSIFIABILITY_INTEGRATION=1` + `leyline` binary. Runs both variants on a real LLO-built mache `.db`, then for each newly-flagged def looks up `_lsp_refs` by token. If LSP HAS a reference but the rule flagged dead, the test fails with the suspect list.

Note

Tests pin `db.SetMaxOpenConns(1)` so TEMP views from `ensureCanonicalViews` remain visible to subsequent queries. Without the pin, `*sql.DB` pools fresh connections and the view-creation isn't seen by later `db.Query` calls.

ADR-0013 closeout

This is the last bead. All 5 migration steps + 2 falsifiability experiments now shipped:

Step Bead PR
1 — producer-side token extraction `ley-line-453f7e` LLO @`12330fc`
2 — `_index_coverage` table `mache-346cfe` #340
3 — canonical views (mention) `mache-346d0f` #341
3 — canonical views (binding) `mache-346d0f` #347
4 — rule rewrites `mache-346d1e` #346
5 — drop LIKE fallback `mache-346d2b` #348
Falsifiability B — π round-trip `mache-354464` #349
Falsifiability A — skip-list ablation `mache-3509aa` this

Closes `mache-3509aa`.

ADR-0013 named two first-class falsifiability experiments. This is
A — the headline. Hypothesis: with _lsp_refs populated, dropping
the dead_code skip-list flags zero NEW dead constructs that LSP
indexed. Any new finding must be a method on a type LSP didn't see
(genuine coverage gap — wedge case 1 from the ADR).

Procedure:
1. Run dead_code in production form (skip-list intact). Findings A.
2. Run dead_code with the skipped CTE neutralized. Findings B.
3. Diff = B \ A — newly flagged dead.
4. For each row in diff, look up its node_id in _lsp_refs. If LSP
   HAS a reference to that def, the design is broken — the alive-
   check's binding arm should have made it alive.

stripSkippedCTE walks the rule body with paren-depth balancing to
locate the matching close paren of `skipped AS (...)` (the inline
SQL has nested parens for the `IN ('main','init',...)` list, so
naive regex matching fails). Replaces the whole CTE with an empty
form: `skipped AS (SELECT '' AS node_id WHERE 0)`. The downstream
NOT IN filter then never excludes anything.

Tests:

- TestStripSkippedCTE_RealRuleBody: introspects the production
  registry, transforms the dead_code rule, asserts the production
  skip-list ('ServeHTTP' marker) is gone from the stripped form
  AND that both queries are still valid SQL. If a future refactor
  moves the skip-list out of an inline CTE, this fails LOUDLY —
  signaling the test needs to update.

- TestFalsifiabilityA_SyntheticHarness: fixture with two methods
  on MyType — 'Read' (LSP indexed it) and 'Quux' (LSP didn't).
  - 'Read' is in the production skip-list → not flagged in A.
    Ablation drops the skip-list; the alive-check's binding arm
    matches the LSP target_node_id → 'Read' is alive → not flagged
    in B either. Diff doesn't contain 'Read'. THE HEADLINE.
  - 'Quux' is not skip-listed and has no refs (textual or LSP) →
    flagged dead in BOTH runs. Same finding under either path.

- TestFalsifiabilityA_IntegrationOnLeylineParse: gated on
  MACHE_FALSIFIABILITY_INTEGRATION=1 + leyline binary. Runs both
  variants on a real LLO-built mache .db, computes the diff, then
  for each newly-flagged def looks up _lsp_refs by token. If LSP
  HAS a reference to that token but the rule still flagged it dead,
  the test fails loud with the suspect list.

TEMP-view persistence note: tests pin db.SetMaxOpenConns(1) so the
TEMP views ensureCanonicalViews creates remain visible to subsequent
queries. Without the pin, *sql.DB pools fresh connections and the
view-creation isn't seen by later db.Query calls.

Closes mache-3509aa. ADR-0013 closeout: all 5 migration steps + 2
falsifiability experiments now shipped.
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 8, 2026

find_smells (advisory)

Scoped to files changed in this PR. Rules below run on the standalone (no-LLO) cross-ref tables; _ast rules (cyclomatic_complexity, long_function, long_file, magic_int_in_comparison) are not exercised here.

fan_out_skew — 2 finding(s) in changed files
Source Node Metric
cmd/serve_lsp.go cmd/functions/enrichAndQueryDiagnostics/source 19
cmd/serve_lsp.go cmd/functions/enrichAndQueryTypeInfo/source 19

Rules: see docs/ARCHITECTURE.md for the full registry. Advisory only — these are heuristics, not gates.

@jamestexas jamestexas merged commit 0113cff into main May 8, 2026
15 checks passed
@jamestexas jamestexas deleted the test/falsifiability-a-skip-list-ablation branch May 8, 2026 22:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant