Skip to content

v1.108.307 — A phase boundary drawn at the wrong place

Choose a tag to compare

@jgravelle jgravelle released this 29 Aug 12:41
· 36 commits to main since this release

One issue: #557, diagnosed by @Ticki84, who cloned the repo and instrumented his own long-running watch-all process after three of our hypotheses died on the thread.

The defect

_walk_tsconfigs descends into Rust's target/ on every watcher event. Measured by the reporter in his own process: 13.58s of a 13.75s reindex, against 0.27s once target was excluded. It fires even when the watcher reports no indexable changes, which rules out parsing and persistence by itself.

_TSCONFIG_SKIP_DIRS was the fourth copy of a skip list in this tree, and the only one deriving from nothing. security._SKIP_DIRECTORY_NAMES is the authority — our own docs say "add spellings there, never to a derived export", and two other exports already derive from it — and it already contained target. This set sat beside them, hand-maintained, and had never heard of it.

Adding "target" was the reported fix and would have been the wrong one — fix the call site, leave the mechanism. It derives now, so target, _build, .gradle, DerivedData and the eight dotted framework trees arrived at once, and the next one needs no edit there.

Union, never replacement. out is deliberately absent from the authority — it names a real source directory for the indexing walk — but has been skipped for tsconfig discovery for this function's whole life, and removing a skip is the one direction this change must not take.

The second half

index_folder evicted the tsconfig alias-map cache unconditionally, so every watcher-driven single-file re-index re-paid the discovery walk that _load_tsconfig_aliases' module-level cache exists to make once. A cache invalidated on every write is not a cache, and it hid behind the walk's own cost rather than showing up as one.

A targeted run knows which files it touched and keeps the map unless one of them is a tsconfig/jsconfig. A run that cannot know still evicts.

The part worth keeping

This corrects our own instrument. The v1.108.304 phase breakdown blamed save=9.906s and we believed it. save includes rebuilding the in-memory CodeIndex after the SQLite transaction, and that reconstruction is what triggers the walk.

A phase boundary drawn at the wrong place names the wrong subsystem confidently — which is worse than no breakdown at all. It sent us hunting lock contention that was never there, and v1.108.305's own process_locks.waited_seconds, shipped for exactly that hypothesis, correctly reported none.

Regressions

tests/test_tsconfig_walk_cost.py, 23 tests. The build-tree one asserts behaviour, not timing: a poisoned tsconfig.json inside target/ whose aliases must never reach the map, which holds on any machine at any speed. A separate test fails if the skip set is ever re-hardcoded — the derivation is the fix, and target being present is only its first visible consequence.

Measured here on a synthetic Rust target/ (9,200 entries): 0.617s → 0.003s, aliases intact. A lower bound on a synthetic tree, not his number.


Full detail in CHANGELOG.md.

pip install --upgrade jcodemunch-mcp