test: Falsifiability A — skip-list ablation experiment (mache-3509aa)#350
Merged
Merged
Conversation
ADR-0013 named two first-class falsifiability experiments. This is
A — the headline. Hypothesis: with _lsp_refs populated, dropping
the dead_code skip-list flags zero NEW dead constructs that LSP
indexed. Any new finding must be a method on a type LSP didn't see
(genuine coverage gap — wedge case 1 from the ADR).
Procedure:
1. Run dead_code in production form (skip-list intact). Findings A.
2. Run dead_code with the skipped CTE neutralized. Findings B.
3. Diff = B \ A — newly flagged dead.
4. For each row in diff, look up its node_id in _lsp_refs. If LSP
HAS a reference to that def, the design is broken — the alive-
check's binding arm should have made it alive.
stripSkippedCTE walks the rule body with paren-depth balancing to
locate the matching close paren of `skipped AS (...)` (the inline
SQL has nested parens for the `IN ('main','init',...)` list, so
naive regex matching fails). Replaces the whole CTE with an empty
form: `skipped AS (SELECT '' AS node_id WHERE 0)`. The downstream
NOT IN filter then never excludes anything.
Tests:
- TestStripSkippedCTE_RealRuleBody: introspects the production
registry, transforms the dead_code rule, asserts the production
skip-list ('ServeHTTP' marker) is gone from the stripped form
AND that both queries are still valid SQL. If a future refactor
moves the skip-list out of an inline CTE, this fails LOUDLY —
signaling the test needs to update.
- TestFalsifiabilityA_SyntheticHarness: fixture with two methods
on MyType — 'Read' (LSP indexed it) and 'Quux' (LSP didn't).
- 'Read' is in the production skip-list → not flagged in A.
Ablation drops the skip-list; the alive-check's binding arm
matches the LSP target_node_id → 'Read' is alive → not flagged
in B either. Diff doesn't contain 'Read'. THE HEADLINE.
- 'Quux' is not skip-listed and has no refs (textual or LSP) →
flagged dead in BOTH runs. Same finding under either path.
- TestFalsifiabilityA_IntegrationOnLeylineParse: gated on
MACHE_FALSIFIABILITY_INTEGRATION=1 + leyline binary. Runs both
variants on a real LLO-built mache .db, computes the diff, then
for each newly-flagged def looks up _lsp_refs by token. If LSP
HAS a reference to that token but the rule still flagged it dead,
the test fails loud with the suspect list.
TEMP-view persistence note: tests pin db.SetMaxOpenConns(1) so the
TEMP views ensureCanonicalViews creates remain visible to subsequent
queries. Without the pin, *sql.DB pools fresh connections and the
view-creation isn't seen by later db.Query calls.
Closes mache-3509aa. ADR-0013 closeout: all 5 migration steps + 2
falsifiability experiments now shipped.
find_smells (advisory)Scoped to files changed in this PR. Rules below run on the standalone (no-LLO) cross-ref tables; fan_out_skew — 2 finding(s) in changed files
Rules: see |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ADR-0013 named two first-class falsifiability experiments. This is A — the headline.
Hypothesis: with `_lsp_refs` populated, dropping the `dead_code` skip-list flags zero NEW dead constructs that LSP indexed. Any new finding must be a method on a type LSP didn't see (genuine coverage gap — wedge case 1 from the ADR).
Procedure
Implementation: stripSkippedCTE
Walks the rule body with paren-depth balancing to locate the matching close paren of `skipped AS (...)` — the inline SQL has nested parens for the `IN ('main','init',...)` list, so naive regex matching fails. Replaces the whole CTE with an empty form: `skipped AS (SELECT '' AS node_id WHERE 0)`.
Tests
Note
Tests pin `db.SetMaxOpenConns(1)` so TEMP views from `ensureCanonicalViews` remain visible to subsequent queries. Without the pin, `*sql.DB` pools fresh connections and the view-creation isn't seen by later `db.Query` calls.
ADR-0013 closeout
This is the last bead. All 5 migration steps + 2 falsifiability experiments now shipped:
Closes `mache-3509aa`.