P0: silent wrong answer under the moving collector — stale heap-string deref in generated code (realistic interpreter workload)

[proggeramlug]

Summary

A realistic, 100%-static TypeScript program — a tree-walking interpreter for a small
functional language — produces a silently wrong answer on default settings. No
crash, no diagnostic, no TypeError: just an incorrect number.

node --experimental-strip-types interp.ts   → 1708840
scriptc 0.0.22 (fully static)               → 1708840
perry  0.5.1384 (a853135aa), defaults       → 1708662     ← WRONG, every run

Reproducible on 6/6 consecutive runs. The program is in
gc-handoff/apps/interp.ts (189 statements; scriptc coverage reports
"fully static — this program has no dynamic remainder", so it is ordinary
TypeScript, not an exotic construct).

This is a moving-collector correctness bug. Four independent knobs each restore
the correct answer:

knob	result
default	1708662 (wrong)
PERRY_GEN_GC=0	1708840 ✓
PERRY_GC_SCAVENGE=0	1708840 ✓
PERRY_CONSERVATIVE_STACK_SCAN=1	1708840 ✓
PERRY_WRITE_BARRIERS=0	1708840 ✓

All four share one property: they prevent evacuation/moving.

User-visible mechanism

The interpreter's variable lookup silently fails to find bindings that are present.
An instrumented lookup (counting the fall-through-to-default path) gives:

node                : checksum 437840  misses 0
perry (default)     : checksum 437836  misses 6      ← 6 silent lookup failures
perry PERRY_GEN_GC=0: checksum 437840  misses 0

Because naive fib is just a count of leaves returning 1, each miss subtracts
exactly 1 from the answer. Miss count varies with allocation timing (2–10 observed);
the wrongness does not.

The failing comparison is names[i] === name where both sides are heap strings
created by src.substring(i, j) in the lexer. A second symptom in the same run: a
concatenated diagnostic string came back empty.

Precise fault (the instrument did its job)

PERRY_GC_PROTECT_FROMSPACE=1 PERRY_GC_PROTECT_FROMSPACE_DEPTH=800 PERRY_GC_ZEAL=1:

[gc-fromspace-protect] FAULT: signal 10 at 0x2fa3bcbcc94
  This address is RETIRED FROM-SPACE. The evacuating minor moved or
  freed the object here and the holder kept the pre-collection address.
  block=0x2fa3bc80000 +248980 retired_bytes=1048568 retired_by_minor=#10
  last-known object: user_ptr=0x2fa3bcbcc88 obj_type=3 size=32
  The faulting instruction IS the stale use. Backtrace:
0  ...arena::quarantine::fromspace_fault_handler
2  perry_fn_iso_miss_ts__evalNode + 18260      ← the stale use, in GENERATED code
3  perry_fn_iso_miss_ts__evalNode + 5712

• obj_type=3 = GC_TYPE_STRING, size 32 (a small heap string)
• faulting address is user_ptr + 12
• the stale dereference is in generated code, not the runtime

Why this is NOT a missing codegen root

Both root lowerings fail, so it is not statepoint-specific:

lowering	result
PERRY_RS4GC=1 (statepoints, the default)	437832 / 437828 / 437837 — all wrong
PERRY_RS4GC=0 (shadow stack)	437837 / 437838 / 437836 — all wrong

(RS4GC is a compile-time choice — setting it on an existing binary does nothing.
These are separate compilations.)

And the static root-dominance checker, run against the shadow-stack lowering where
it is actually able to see roots, reports the program clean:

$ perry iso_miss.ts --trace llvm      # PERRY_RS4GC=0 PERRY_INLINE_SHADOW_SLOT=0
shadow_slot_bind: 381
$ python3 scripts/gc_root_dominance_check.py .perry-trace/llvm/iso_miss_ts.ll
=== checked 55 functions / 1 modules (1 .ll files, 380 root stores)
=== violations: 0   (moving-minor reachable: 0)

380 root stores — not a vacuous run. So codegen's root discipline is, by the
checker's model, complete for this program.

Per this repo's own rooting doc, that combination points at a table, not a
register (docs/src/internals/gc-rooting-invariant.md, echoed in CLAUDE.md):

A runtime-side cache of a raw heap pointer is a GC root, and the static checker
cannot see it … an unrooted register goes bad only when a collection lands in
its window, so it is intermittent; an unrooted cache goes bad at collection #0
and stays bad, so a perfectly reproducible GC bug means a table, not a
register.

This bug is perfectly reproducible. The prime suspect is therefore a runtime-side
cache holding a raw pointer to a heap string that is not rewritten on
evacuation — i.e. a missing entry in the gc_register_mutable_root_scanner
registry (gc/mod.rs, ~55 scanners), or a borrowed &str/&[u8] into a
StringHeader held across an allocation (see
changelog.d/7219-registry-gc-unrooted-caches.md and
changelog.d/7239-gc-unrooted-runtime-caches.md for the two prior instances of
this exact shape).

Candidates not yet excluded: any string-comparison or property-key fast path that
memoises a *const u8/usize string pointer. The intern table already has both
intern_table_root_scanner and intern_table_mutable_root_scanner registered, so
it is likely not that one.

Knobs that do not fix it (so these subsystems are excluded):
PERRY_SHAPE_LAYOUT_KEYED=0, PERRY_DISABLE_CLASS_FIELD_INLINE=1,
PERRY_GC_OLD_DEFRAG=0 — all still wrong.

Repro

cd gc-handoff/apps
export PERRY_RUNTIME_DIR=<repo>/target/release
PERRY_NO_AUTO_OPTIMIZE=1 <repo>/target/release/perry interp.ts -o p_interp
./p_interp                      # 1708662  (wrong)
PERRY_GEN_GC=0 ./p_interp       # 1708840  (correct)
node --experimental-strip-types interp.ts   # 1708840

# tighter: FIB alone, with lookup-miss instrumentation
PERRY_NO_AUTO_OPTIMIZE=1 <repo>/target/release/perry iso_miss.ts -o p_iso_miss
./p_iso_miss                    # checksum 437836 misses 6
PERRY_GEN_GC=0 ./p_iso_miss     # checksum 437840 misses 0

# the precise fault
PERRY_GC_PROTECT_FROMSPACE=1 PERRY_GC_PROTECT_FROMSPACE_DEPTH=800 \
  PERRY_GC_ZEAL=1 ./p_iso_miss 2>&1 | head -12

iso_miss.ts is interp.ts with only the FIB program, run 40×, plus a miss counter
in lookup. Both files are committed under gc-handoff/apps/.

Why the existing gates missed it

• The gap suite and the GC-ratchet probes assert on programs whose shape is much
  simpler than an interpreter: no long-lived recursive-union graph with heap-string
  keys walked under allocation pressure.
• PERRY_GC_VERIFY_MARK=1 reports OK (no marked->unmarked, 6210 marked / 20339
  edges checked) — marking is correct; it is the post-evacuation rewrite of a
  non-heap-resident holder that is missing, which mark verification cannot see.
• The static root-dominance checker is clean, as shown above.
• Under the shipped statepoint default the checker cannot run at all — it anchors on
  @js_shadow_slot_bind, which statepoint IR does not emit. It correctly reports
  its own vacuity (0 root store(s) in the corpus … The subject of this check never ran) rather than a false clean, but the practical effect is that this class has
  no static gate on the default lowering (engine-plan "what is left" item 5).

Suggested fix path

1. Audit for a runtime cache keyed on or storing a raw heap-string pointer that has
   no gc_register_mutable_root_scanner entry. PERRY_GC_FROMSPACE_SCAN=1 reports
   45 offending owners, all type=3 (string) with never_dirty=45 /
   not_in_snapshot=45 — no write barrier ever fired for them, consistent with a
   holder that is not a heap object at all.
2. Add the scanner (or copy the bytes off-heap, if it is a borrow — see the
   "borrowed heap slice" note: rooting cannot fix a &str into a moved
   StringHeader).
3. Add interp.ts to the GC-ratchet corpus as a correctness probe (assert exact
   stdout vs Node), not a timing probe. This program is a much better canary for
   this bug class than any existing probe.
4. Separately: the statepoint-side static checker (engine-plan item 5) would have
   made this a red build.