json tape: full-array scans pay 2.3× the direct parser (field_access 2981ms vs bun 223) — decomposition + roadmap

[proggeramlug]

json_polyglot/field_access (parse → touch every element → stringify) is the lazy tape's worst case: 2981 ms vs bun's 223, while the same tape wins roundtrip outright (194 vs bun 221, node 384). This issue records tonight's decomposition and the two dead ends, so the next attempt starts from measurements instead of the plausible-but-wrong model I started with.

Decomposition (50 iterations, quiet host, same binary)

phase	tape on	tape off
parse only	210 ms	1220 ms
parse + full scan	3030 ms	1287 ms
parse + stringify	254 ms	1756 ms

Tape wins parse 6× (deferred materialization) and stringify 7× (unmutated-tape memcpy). The full scan is the entire loss: ~56 ms/iter for scan+materialize vs ~24 ms/iter for the direct parser's tree build. Both build the same 10k-record tree — the tape's per-element materialization path is ~2.3× the batch parser.

Two approaches tried, measured, and rejected

1. Earlier batch-flip triggers. Added distinct-element coverage (trip at half) and sequential-streak (trip at 16 consecutive) triggers alongside the existing walk-steps heuristic — the walk-steps trigger provably cannot fire on sequential access (1 step/element accumulates n, threshold is 2n). Both triggers verified firing (lldb breakpoint: 1 materialize per iteration). Result: time noise-equal (min 3050 old vs 3322 new), RSS regressed 180 → 213 MB — the full tree materializes while blob+tape+cache are still live, and total materialization work is unchanged because every element gets materialized either way. Reverted.

2. Re-parse the retained blob with the DirectParser inside force_materialize_lazy (lazy is top-level-array-only, so the blob is exactly the array's source; patch the few handed-out cached elements back over the fresh slots for identity). This is the structurally right fix — batch parse at 24 ms/iter instead of 56 — but it is blocked by #7477: the DirectParser's float parsing diverges from the tape's (and from node), so the prototype produced checksum 2552986400 where node says 2552985550. It also had an intermittent SIGSEGV in my rooting; both are why it was reverted rather than debugged into shape at 4 a.m.

The path that wins both

1. Fix JSON DirectParser parses floats differently from the tape materializer — and node agrees with the tape #7477 (DirectParser float divergence — a correctness bug in its own right).
2. Re-land the reparse-on-materialize with the identity patch loop (the prototype's shape, with the rooting done carefully: root the parsed result before re-reading the header, per-element handle across store_array_slot).
3. Keep a trip heuristic only as the decision of when to batch — the streak/coverage work is sound for that, it just cannot pay while the batch materializer is the same speed as the element-wise one.

Expected: field_access ≈ 210 + 50×(24 + ε) + memcpy-stringify ≈ ~1500 ms (from 2981), while roundtrip keeps its 194 ms win — the tape then wins or ties every JSON benchmark against the direct parser, and the remaining gap to bun (~223 ms) becomes a parser-speed workstream (shared with json_parse_1mb, currently 6.27× vs bun), not a tape-policy one.

One more datum

The per-element merge branch in force_materialize_lazy (fresh RuntimeHandleScope + safepoint per element, to honor a handful of cached slots) looked like the cost and is not: replacing it with root-batch + patch-cached-slots measured neutral. The copied-minor sabotage test (test_json_tape_force_materialize_sparse_cache_handles_survive_copied_minor_gc) hooks the ForceLazyArrayRooted safepoint — any rework of that function must keep firing it, or the instrument silently loses its subject.

[proggeramlug]

Step 2 (reparse-on-materialize) is up as #7499.

Root-caused the intermittent SIGSEGV from the prototype: DirectParser is only sound inside a no-move window in three separate ways — its input: &[u8] is derived from the blob StringHeader for the whole parse, it carries an unrooted raw-pointer shape cache (hot_shape_keys/hot_shape_array) across every allocation, and array_push_parse_fast skips the generational barrier through note_array_slot_layout_only explicitly on the strength of js_json_parse's suppression. The prototype called it without one. #7499 wraps the parse in a nesting-safe GcSuppressScope.

One correction to the plan's expectation, worth recording before step 3 is attempted: the reparse alone does not move field_access, and cannot. try_stringify_lazy_array force-materializes whenever any sparse-cache bit is set, so field_access does reach force_materialize_lazy — but with the bitmap full (all 10k elements were handed out by the scan). At that point the element-wise merge is already the cheap producer: it copies 10k cached JSValues and materializes nothing. All 2981 ms was spent in the 10k lazy_get calls before it. #7499 therefore gates the reparse on cached_count * 2 < cached_length so it does not turn a cheap merge into a full redundant parse.

That makes step 3 load-bearing rather than optional for this benchmark: the trigger has to fire early, while the cache is still mostly empty, so the reparse replaces the remaining per-element work instead of following it. The step-1 measurement (trigger neutral, RSS 180 -> 213 MB) was taken with the slow materializer on the other side; the RSS half of that cost is unchanged by #7499 and should be re-measured together with the time half.

Benefiting today, without any new trigger: full-array ops on a lazy array whose elements have not been individually walked (.map/.filter/spread/for…of/sort, i.e. the 1_json_pipeline filter-all-records shape), and the existing random-access cumulative_walk_steps trigger, which fires at a cache count of ~4 on a 10k array.

Also filed #7500 — PERRY_GC_VERIFY_EVACUATION=1 aborts on any PERRY_JSON_TAPE=1 workload (lazy sparse-cache slots report missing old->young edges). A/B'd against the merge-base: pre-existing, not from #7499.

[proggeramlug]

Quiet-host numbers from the public baseline at v0.5.1299 (pinned mini, AC power, CPU-quiet gate passed, node 22.23.1 / bun 1.3.14). These settle the direction.

field_access — the lazy tape is a NET NEGATIVE, not merely insufficient:

config	median	p95	σ	RSS
perry optimized (gen-gc + lazy tape)	2984 ms	3341	136.4	219 MB
perry idiomatic (mark-sweep, no lazy)	1350 ms	1351	0.7	61 MB
bun	218	218	0.4	91 MB
node	380	384	2.2	94 MB
rust serde_json (LTO)	183	199	4.7	10 MB

Our "optimized" configuration is 2.2× slower than our unoptimized one on this
workload, and carries 3.6× the RSS (219 MB vs 61 MB). Against bun it is 13.7×.

Two things worth pulling out beyond the headline:

1. σ = 136.4 against every other row's < 5. The lazy path's cost is not just
   high, it is variable — 2843…3341 ms across samples where bun varies by 1 ms.
   Whatever the materializer is doing has data-dependent cost that nothing else
   in the table exhibits. That is a lead in its own right.
2. The roundtrip arm shows the tape is worth keeping. Same binary, same run:
   192 ms optimized vs 1307 ms idiomatic — a 6.8× win, beating bun (216) and node
   (379) and landing within ~8% of Rust serde_json (178). So this is not "remove
   the tape"; it is "stop paying for it on scan-shaped access".

What this means for the fix. #7499 landed the reparse-on-materialize path but
is a deliberate no-op here, because field_access reaches force_materialize_lazy
with the cache bitmap already full — the 2984 ms was spent in the 10 000
lazy_get calls before materialization, not in the merge walk. So the remaining
work is the early batch-flip trigger: detect scan-shaped access and hand off to
the batch parser before paying those calls.

The idiomatic row is also a useful upper bound on what a trivial fix is worth:
if the tape simply declined this shape and fell back, we would be at ~1350 ms
(2.2× better) without any new machinery. The batch-flip should beat that, but
anything that does not is worse than switching off.

Acceptance should be both arms: field_access materially under 1350 ms and
roundtrip still at ~192 ms with the memcpy path intact.