perf(json): kill lex_number's per-call heap allocations

[mizchi]

Summary

Three related allocation removals along the JSON number lex chain,
all stemming from the same observation: the native target boxes every
returned struct / tuple / enum payload by default, so any per-number
intermediate value lives on the heap until the next GC sweep / refcount
drop.

1. Drop the (Double, StringView?) tuple

The number-lex chain (lex_zero / lex_decimal_* / lex_number_end /
lex_integer_end) returned (Double, StringView?), where the second
component is the source-text view that's only ever Some(_) on the
infinity-overflow path. Native boxes every tuple, so every parsed
JSON number cost one heap allocation per call just to carry a
value that was structurally None.

Side-channel the optional view through a new private
ParseContext.last_number_repr field. Each leaf return writes it
(usually None); lex_main.mbt reads + clears when constructing
the Number token. No public API change — ParseContext and these
helpers are all priv.

2. Stack-allocate JsonNumberScan via #valtype

scan_json_number builds + returns a five-field JsonNumberScan
struct on every JSON number, used immediately by lex_number_end
and discarded. Without #valtype the native target boxes that
struct as a ~32-byte heap object per call.

The annotation is the only change — same struct, same call sites.

3. NaN sentinel for try_fast_double

JsonNumberScan::try_fast_double returned Double? — Some(d) on
the fast path, None to fall back to strconv. The fast path can
only produce 0 or a finite Double (the checked_mul guard rules
out infinity), so NaN is a free sentinel. Returning a plain
Double (with NaN meaning "not handled by the fast path") skips
the boxed Option<Double> allocation on every JSON number that
hits this path.

Numbers

Measured on a native-target alloc profiler (mizchi/pprof-mbt's
memprofile-native) over the bench suite (--sample-rate 100):

bench	metric	before	after	Δ
json_numbers (10 k integers × 30)	allocs	1 200 200	600 200	−50 %
	bytes	18.31 MB	9.16 MB	−50 %
json_parse (1 000-obj × 50)	allocs	3 250 200	2 300 200	−29 %
	bytes	58.56 MB	46.93 MB	−20 %

#3 adds the last 100 k allocs / 681 kB on json_parse on top of
#1 + #2 (those handle the integer-only path; try_fast_double
fires for the mixed-content numbers).

After all three fixes, json_numbers's remaining allocations all
land in parse_value2 (the Number(_, _) token construction), which
is fundamental. The dominant lex_integer_end / scan_json_number /
try_fast_double attribution that previously stacked up to >18 MB on
that bench is gone.

Why this is safe

• The tuple change is an internal API rewrite; the Token surface
  produced by lex_main is identical (still Number(Double, String?)
  with the same string content on the overflow path).
• #valtype is the language's existing knob for opting a priv
  struct out of boxing — same semantics, just no heap roundtrip.
• The NaN sentinel relies on try_fast_double never legitimately
  producing NaN (it only returns 0, a finite scaled mantissa, or the
  sentinel itself). checked_mul rejects multiplications that could
  go through Infinity, and pow10_fast_path only indexes pre-computed
  finite doubles. Verified by the existing json test suite.

Test plan

• moon test --target native -p json — 171 / 171 pass
• moon test --target wasm-gc -p json — 171 / 171 pass
• moon fmt --check clean

Independent of #3632 (json lex_skip_whitespace) and #3634 (primitive
Hash::hash overrides). Same workflow each PR: native-target alloc
profile a real bench → attack the top site.

[Copilot]

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

This PR refactors JSON number lexing to avoid per-number tuple allocations by returning only Double and using a ParseContext side channel (last_number_repr) to carry the optional original string representation when needed (e.g., infinity fallbacks).

Changes:

• Replace (Double, StringView?) returns across number-lexing functions with Double and write the optional repr into ctx.last_number_repr.
• Update lex_value to read + clear last_number_repr when constructing Number tokens.
• Add last_number_repr state to ParseContext and initialize it in ParseContext::make.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File	Description
json/lex_number.mbt	Switch number lexing functions to return Double and set ctx.last_number_repr instead of returning tuples.
json/lex_main.mbt	Adapt number-token construction to read and clear ctx.last_number_repr.
json/internal_types.mbt	Add mut last_number_repr : StringView? to ParseContext and initialize it.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.