gc: forcing POINTER_FREE on a pointer-bearing object strands nothing — our zeal/protect instruments do not discriminate the layout-state hazard

[proggeramlug]

The problem

While auditing #7633 I sabotaged layout_finish_deferred_boxed_object(ptr, saw_pointer) → (ptr, false) — i.e. every JSON-parsed record claims
POINTER_FREE while holding heap pointers. That is the stranded-live-child
hazard #7630's own soundness note names, and the exact failure the layout state
exists to prevent.

It produces byte-identical correct output under every instrument we have:

probe	plain	ZEAL=1 PROTECT_FROMSPACE=1	FORCE_EVACUATE=1
children also held in a separate array	identical	identical	identical
children reachable ONLY via the record's slot	identical	identical	identical

The second probe was built specifically because the first was vacuous: 4,000
records × 2 pointer fields, 40 rounds of churn to force promotion, children read
back only after the churn, so the record's slot is the sole path to each string.
8 retired quarantine sets and 16 copying minors observed live — the
instruments were armed and the collector was moving.

Why this is not "the branch is safe"

The obvious benign explanation is refuted. Parse value strings are not
longlived or interned: string_storage_alloc (string/mod.rs:504) uses
arena_alloc_gc, the ordinary nursery arena — arena_alloc_gc_longlived is the
other function. My probe's strings are 6–10 chars, above
SHORT_STRING_MAX_LEN = 5, so they are real heap StringHeaders in the
nursery. They are collectable and movable, and a POINTER_FREE record should
therefore lose them.

So either the tracer does not honour POINTER_FREE on this path (in which case
the state's trace-skip is not buying what it claims anywhere), or something
re-marks these objects that neither I nor the PR author identified.

Why it matters

Every future change to layout-state bookkeeping — and #7630's family has more
queued — will cite "zeal + from-space protect: output identical" as evidence of
no stranded children. That sentence currently discriminates nothing: a
mutation guaranteed to strand children passes it. This is CLAUDE.md's hazard 4
(a gate that runs but whose subject never did), applied to the instrument rather
than the job.

#7633 merged on the strength of its argument — the materialiser owns each
object end-to-end, finalize is reached on every path (verified: zero early exits
between store and finalize in both parse functions), and the pointer case is
conservative — plus a clean gap suite. That was the right call. But the argument
is now the only thing holding it up, and the next such change may not have one
as tight.

What would close this

A probe where forcing POINTER_FREE on a pointer-bearing object faults or
corrupts. If one cannot be constructed, that is the finding: it means our
POINTER_FREE trace-skip is unobservable, and we should understand why before
building more optimizations on top of it. Either outcome is worth knowing, and
the probe becomes the regression test the whole family has been missing.

Starting points: check whether the full mark-sweep's conservative stack scan
(#7558) reaches these objects independently of their layout state; check whether
old-gen tracing consults _reserved layout bits at all after promotion; and try
a shape where the record is never promoted (small parse, forced minor only).

[proggeramlug]

Answer: the hazard is detectable, and the instruments were never at fault — the probe's subject never existed

PR #7643 ships the regression tests and the writeup. The mechanism, in order of
the questions in the issue.

1. Does the tracer honour POINTER_FREE on this path? Yes, unconditionally

heap_payload_slot_selection (gc/layout.rs) short-circuits on the state and
returns PointerFree with no further gate — no generation test, no size class,
no GC_OBJ_TYPED_LAYOUT_INTACT, no arena kind. Every collector pass funnels
through it via gc_child_slots → visit_gc_layout_slot_descriptors:

• marking — trace.rs::trace_heap_rewrite_slots → visit_gc_rewrite_slots;
• the evacuation rewrite — copying.rs:320/752, verify.rs;
• the remembered-set dirty scan — barrier.rs:133, barrier.rs:555.

All three drop the payload on GcMutableSlotDescriptor::PointerFreeRange => {}.
So the hazard is real at code level and there is no rescue path.

2. Does the conservative stack scan rescue them? No — and it never got the chance

The probe has no gc() call, so #7558's forced conservative scan is not in play.
The actual reason is much simpler and I confirmed it directly rather than
inferring it.

js_json_parse routes a top-level array of 1 KB–16 MB through the LAZY TAPE
(json_tape, default-on since #179, window LAZY_MIN_BLOB_BYTES = 1024 …
LAZY_MAX_BLOB_BYTES = 16 MiB in json/parse_api.rs). parse_object — the
function carrying the sabotaged finalize — therefore does not run at
JSON.parse time. It runs when an element is first read. Our probes read the
records only after the churn, so every misdeclared record was materialised
after the last collection.

Confirmed, not inferred. On the sabotaged build I added a temporary audit to
heap_payload_slot_selection that reports any object arriving in POINTER_FREE
state whose payload words decode as pointer-bearing:

• default (lazy) run: zero hits across the whole program, over four copying
  minors. Tracking one record by address through gc_child_slots showed it was
  never entered at all.
• PERRY_JSON_TAPE=0: hits immediately —
  [7635-audit] n=0 misdeclared POINTER_FREE objtype=2 slots=2 pointer_words=2.

Nothing was stranded because nothing was there. This is CLAUDE.md's hazard 4 —
"the gate runs but its subject never did" — applied to a probe rather than a job.

3. Shapes that do work

Sabotaged vs clean runtime, perry-dev, macOS arm64, PERRY_NO_AUTO_OPTIMIZE=1,
4,000 records × 2 string fields, 40 churn rounds:

arm	clean	sabotaged
default (lazy), read after churn	exit 0	exit 0, byte-identical, dangling=0 — the issue's result, reproduced
PERRY_JSON_TAPE=0, read after churn	exit 0	SIGSEGV (139)
default (lazy), records touched BEFORE the churn	exit 0	7,872 of 8,000 values read back wrong

So the lazy path is not immune either — deferring the reads was the whole
effect.

On the PERRY_JSON_TAPE=0 arm every instrument fires:

sabotaged:  [gc-fromspace-scan OFFENDERS] ... dangling=8000 owners=4000 | never_dirty=8000
clean:      [gc-fromspace-scan clean]     ... dangling=0    owners=0

sabotaged:  [gc-fromspace-protect] FAULT: signal 10 at 0x5f76ee69ee8
            The faulting instruction IS the stale use.

dangling=8000 owners=4000 is exactly 4,000 records × 2 pointer fields. (Both
arms carry a small pre-existing residue on later cycles — clean peaks at
dangling=156 — orders of magnitude below the signal, but worth a look
separately.)

The one instrument that IS structurally blind

PERRY_GC_VERIFY_EVACUATION walks the same enumeration the rewrite pass walks,
i.e. it asks this very layout state which slots exist. It cannot see a
misdeclaration, ever. gc/fromspace_scan.rs's module header already makes this
point about the verifier generally; it is worth knowing it applies with full
force to this hazard class.

PERRY_GC_FROMSPACE_SCAN=1 is the layout-independent one — whole-payload
word scan, no root enumeration, no layout state — and it is the knob to reach
for here. It was the one missing from the audit.

What shipped (#7643)

crates/perry-runtime/src/gc/tests/copying/deferred_finalize_7635.rs, four
tests that need no workload at all, so no lazy path, GC-timing accident, or
conservative-scan residue can defeat them:

1. the finalize's two exact outcomes (no pointer ⟹ keep the birth state; any
   pointer ⟹ GC_LAYOUT_UNKNOWN, no mask, no stranded descriptor);
2. the child-slot enumerator on a materialiser-built record + relocation across
   a copying minor, gated on copied_objects >= 3;
3. the same invariant through the real js_json_parse entry point, so the
   json/parser.rs call site is covered — the sabotage was applied there and
   tests 1–2 stay green through it;
4. a_misdeclared_pointer_free_record_strands_its_child — the sabotage arm made
   permanent, asserting a misdeclared record enumerates ZERO slots and that
   correcting the state alone makes the same record fully enumerable.

Sabotage-verified in both directions: #7635's exact parser mutation reddens (3);
neutering layout_finish_deferred_boxed_object reddens all four; restored, all
four green.

The doc comment on GC_LAYOUT_POINTER_FREE now records what can and cannot
verify a claim about this state, so the next PR in the family does not cite
"clean under zeal + protect" without first showing the subject existed during a
collection. #7633's changelog fragment carries the same qualification — its
argument was right, but its probe could not have shown it either way.

Worth a follow-up

PERRY_JSON_TAPE=0 + PERRY_GC_FROMSPACE_SCAN=1 over a parse-then-churn
workload, asserting dangling=0 and that a copying minor actually ran, is
now a known-good end-to-end gate for the whole layout-state family. Per
CLAUDE.md a new gate must be run once before being promoted to required, so
that is a separate change.

[proggeramlug]

Solved, with the probe you asked for — outcome (3), and the missing shape is now identified: the hazard is real, the instruments were never broken, and every probe in this investigation was measuring the lazy tape.

JSON.parse of a non-tiny blob is lazy by default (#7499). Parse → churn → read-back materializes the whole cohort at read-back, after the collections ran — there was nothing to strand while the instruments were armed. Your second probe's careful "children reachable only via the record" construction was defeated before it started: the records themselves didn't exist during the 16 copying minors you observed.

Two supporting measurements from an instrumented build (details in PR #7644):

• Under your exact sabotage with a lazy-shape probe, a trace-time counter in the POINTER_FREE selection arm saw only ~5 pointer-bearing skips across dozens of cycles × 4,000 records — the cohort wasn't there.
• The identical TS shape plus one pre-churn touch loop (sum += records[i].n over all records — defeats the tape) faults immediately: SIGSEGV under default cycles, and a precise [gc-fromspace-protect] FAULT: signal 10 with the named address under ZEAL=1 PROTECT_FROMSPACE=1. Your instruments discriminate perfectly once the subject exists.

PR #7644 commits the regression test at unit level (no env knobs, fromspace_protect planted-hazard convention): a lying saw_pointer=false finalize strands a field-only young string — asserted on both halves (slot unrewritten AND child in poisoned from-space) with the object-moved premise, plus the truthful green twin. It also records the standing rule this closes on: any parse-cohort GC probe must defeat the lazy tape or state that it exercised the lazy path — cohort-liveness is part of subject-liveness, and this was the third vacuity class in the family precisely because it hid one level deeper than the previous two.

[proggeramlug]

Closing — the hazard IS detectable, and this issue's premise was wrong. Resolved by #7643 (v0.5.1365).

I filed this after forcing every JSON-parsed record to POINTER_FREE while it held heap strings and getting byte-identical correct output under PERRY_GC_ZEAL=1 PERRY_GC_PROTECT_FROMSPACE=1, PERRY_GC_FORCE_EVACUATE=1 and PERRY_GC_VERIFY_EVACUATION=1, with copying minors and retired quarantine sets observed live — and concluded the instruments could not see this hazard class.

The instruments were never at fault. My probe's subject never existed. js_json_parse routes a 1 KB–16 MB top-level array through the lazy tape (json_tape, default since #179), so parse_object — the function I sabotaged — does not run at JSON.parse time. It runs when an element is first read. My probe read its records only after the churn, so every misdeclared record was materialised after the last collection. A temporary audit in heap_payload_slot_selection on the sabotaged build counted zero POINTER_FREE objects with pointer-bearing payload words ever handed to the collector. Nothing was stranded because nothing was there.

Re-run so the records live across a collection — reproduced independently on my side:

arm	clean	sabotaged
default (lazy), read after churn	exit 0	exit 0, byte-identical — the result I filed this on
PERRY_JSON_TAPE=0	exit 0	exit 138 = SIGBUS; dangling=8000 owners=4000 (4,000 records × 2 fields)
default (lazy), records touched before the churn	exit 0	7,872 of 8,000 values read back wrong

So the lazy path isn't immune either — deferring the reads was the entire effect.

Two things survive that are worth more than the bug I thought I had.

PERRY_GC_VERIFY_EVACUATION is genuinely blind to a misdeclaration, by construction: it walks the same enumeration the rewrite pass walks, i.e. it asks the layout state under test which slots exist. PERRY_GC_FROMSPACE_SCAN=1 is the layout-independent instrument — whole-payload word scan, no root enumeration, no layout state — and it is what should be reached for on this hazard class. That is now recorded on GC_LAYOUT_POINTER_FREE itself rather than in an issue thread.

And this was CLAUDE.md's own hazard 4 — "the gate runs but its subject never did" — applied to a probe rather than a job. A clean run under zeal proves nothing until you have shown the object you are hunting existed during a collection. #7643 ships four workload-free tests that cannot be defeated that way (child-slot enumerator; relocation gated on copied_objects > 0; the same invariant through the real js_json_parse so the call site is covered; and a permanent sabotage arm), and #7647 tracks the end-to-end gate.