Skip to content

feat(l3): common-tail merge, scoped base clusters, and ranked live-range coalescing - #13

Merged
macabeus merged 6 commits into
mainfrom
match/UpdateHUDCounterDisplay-3
Aug 3, 2026
Merged

feat(l3): common-tail merge, scoped base clusters, and ranked live-range coalescing#13
macabeus merged 6 commits into
mainfrom
match/UpdateHUDCounterDisplay-3

Conversation

@macabeus

@macabeus macabeus commented Aug 3, 2026

Copy link
Copy Markdown
Owner

Third round on kleod:UpdateHUDCounterDisplay:agbcc. diff:60diff:18 (m2c: noncompile). Board: asmlift 350 vs m2c 340, 0 lost, 0 gained, 0 flips.

The row still does not match. What ships is three general L3 capabilities plus two bug fixes; the remaining gap is described at the end.

What ships

mergeCommonTails (l3/tailmerge.ts, unconditional in structureChecked). A statement that ends EVERY arm of an if moves below the if. SSA destruction puts the same merge-variable write at the end of each arm because that is where each edge's copy belongs; the source wrote it once. Both arms execute it last on their own path, so moving it below runs it on the same paths in the same order — no liveness or dominance analysis needed, and it holds for a side-effecting statement. The merge direction is what is unconditionally sound: a common HEAD would move across the condition's own evaluation. Placement is measured, not taste — below the if scores 33, peeling the same statements above it scores 48.

hoistScopedBases / /scopebase gains a deepestCluster fallback: when a base's uses have no common scope, hoist at the deepest cluster that does, rather than declining. Differ-refereed, so it can never cost a match.

coalesceCandidates / /coalesce (l3/coalesce.ts). Which pair of locals the register allocator coalesced is not derivable from the L3 tree, and first-fit gets it wrong — on this row the two legal merges score 18 and 40, and declaration order picks the 40. So every legal single merge is emitted as its own candidate and the differ referees, the same idiom /regcopy already uses.

fix(rank): a lever that throws was swallowed and reported as a decline — indistinguishable from "this lever had nothing to offer". It now reports via onLeverError.

fix(l3): exprEquals compares a field's dot-vs-arrow spelling. A CSE or dedup that treats p->f and p.f as equal keeps one node and discards the other, silently respelling the access.

Generality: mined against a whole game, not just the benchmark

Only ONE benchmark row changed, which is a fair thing to be suspicious of for a branch that adds an unconditional pass. Two measurements answer it.

First, instrumenting the passes over all 675 benchmark cases with the harness's own options: mergeCommonTails fires on exactly 1, and the exprEquals dot fix has 0 divergences corpus-wide (it is defensive, unit-tested only).

Second, the same instrumentation over the klonoa decomp — 464 per-function .s plus the built translation units, 368 of which lift and structure:

pass klonoa benchmark
mergeCommonTails 11 distinct functions, 7 of which change the emitted C 1
hoistScopedBases 29 functions wins 1 row
coalesceCandidates 19 candidates on 7 functions 2 candidates on 1 row

Firings outside the target row, verified by reading the before/after C: EntityCrushingBlock (a base assignment ending both arms), UpdateBootMinigame (two ifs, two statements merged out of each), UpdateUIElementAnimation (v7 = 0 merged out of a nested if and then its parent).

Why only one benchmark row moved: size. The klonoa functions where the shape occurs have a median of 198 asm lines; the benchmark's structured agbcc rows have a median of 33, a p90 of 65, and a maximum of 167 — exactly one row is ≥155 lines, and it is this one. A common tail needs an if whose arms each carry several statements plus a surviving merge phi, which essentially does not occur in a 33-line function.

Known weakness

constFed is doing 98% of coalesce's filtering. Relaxing that gate alone takes the klonoa corpus from 19 candidates on 7 functions to 1002 on 82 (one function yields 93). The gate is documented as a codegen heuristic that also bounds L(L-1)/2 growth, and both are true — but it means the pass enumerates ~2% of the real opportunity, and the surviving 2% is exactly the all-literal shape this row happens to have. That is the one place where "shaped by the target function" has real bite.

Also honest about the mining: 4 of the 11 tailmerge firings are output-neutral (they fire at the tree level but produce identical C after eliminateDeadStores and basecse); 3 are in m4a, which is old_agbcc territory and discounted above; and the mined population is biased toward what lifts (368 of 770 — the declines are computed jumps, sp-as-data, and overlapping struct fields).

Debt, deliberately not folded in

  • basecse, argbase and scopebase hold three divergent copies of the base-hoist policy. The end state is one l3/basepolicy.ts plus one hoist pass parameterized by placement.
  • l3/ has nine hand-rolled Stmt walkers; a stmtLists/mapStmt primitive in ast.ts would make a collector and a rewriter unable to disagree by construction. coalesce.ts's local rename duplicates regspell.ts's exhaustive renameInStmt, and worse.
  • Candidate growth is L(L-1)/2 with no budget. Visible already: this row's pre-existing broken raw-globals candidate now fails 3× per signedness instead of once, because the coalesce axis multiplies it.
  • Whether /scopebase-coalesce should compose as an axis rather than an enumerated re-spelling.

What is still missing for the match

~13 points are unaccounted. The bitfield member spelling (.dreamStones, worth ~5) is not built — it needs the layoutOf bitfield filter relaxed, bitOffset on the member interface, a map re-vendor, and an extract-shift recognizer. A /preinit lever was built and reverted: it scored worse AND a later audit found it unsound — 9 of 13 shapes silently miscompile, with a live inhabitant in the corpus (kleod:CopyBGScrollTiles, where the if sits inside a do/while, so hoisting the default makes the arm sticky). All core tests passed with it installed, which is its own finding: the repo has a byte differ but no semantic-equivalence gate for L3 rewrites.

Verification

pnpm bench run + pnpm bench regression → 0 lost, 0 missing, 0 gained, 0 other flips over 675 rows; stale-check green, meta.asmlift.dirty: false. pnpm test:offline 672 passing, apps/benchmark 139, apps/web 34. pnpm typecheck, pnpm lint, pnpm format:check clean. Each of the five code commits typechecks and passes the suite standalone.

🤖 Generated with Claude Code

macabeus and others added 6 commits August 3, 2026 15:12
Two `field` nodes differing only in `dot` compared EQUAL, so any dedup or CSE keeps
one and discards the other — silently respelling `p->field_4` as `p.field_4`, or the
reverse. Each form compiles only for the base type it belongs to, so collapsing them
turns a valid access into an invalid one, or into a valid one against a different
object.

`ast.ts` already documents this exact requirement for `lead` ("part of the ADDRESS,
so a CSE/dedup path must not collapse distinct accesses"); `dot` is the same class of
field fact and was simply missed. Not reachable today — `dot` is set at one site, for
a struct-VALUE global whose name cannot collide with a pointer local, and
`assertDerefsTyped` rejects the mismatch in both directions — but `exprEquals` is
shared by `basecse`, `tailmerge` and `coalesce`, so the guard belongs at the source.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`respell`'s bare `catch {}` made a lever that throws or fails a boundary contract
indistinguishable from one that correctly declined: both produce no candidate and no
trace. `dropped` records only spellings the SCORER refused, and its own docstring
says why that matters — "a scoring harness that shows only the surviving sibling
reports a clean win over a hidden failure". A lever failing inside enumeration was
the one path with no reporting at all.

`onLeverError` is optional on EnumerateOptions, so every caller is unaffected;
enumeration continues and the primary spelling is untouched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
SSA destruction puts the same merge-variable write at the end of each arm, because
that is where each edge's copy belongs:

    if (c) { v4 = 1; } else { g[594] = g[659]; v4 = 1; }

The source wrote it once. Both arms execute it LAST on their own path, so moving it
BELOW the `if` runs it exactly once, on the same paths, in the same order relative to
everything else — no liveness or dominance analysis needed, and it holds even for a
side-effecting statement. That is the merge direction that is unconditionally sound:
hoisting a common HEAD above the `if` would move it across the condition's own
evaluation, which is not.

Runs before `eliminateDeadStores`, whose empty-then peephole then flips the arm this
empties. On kleod:UpdateHUDCounterDisplay: 60 → 33, and the placement is not a matter
of taste — peeling the same statements to ABOVE the `if` scores 48.

SCOPE: only `assign`/`store`/`exprstmt` merge, compared through `exprEquals`. Control
flow is excluded (moving a `return` out of an arm changes what the arm can reach).
Nested if/loop/switch are excluded — comparing them needs a full `Stmt` congruence and
there is no second inhabitant for one. `switch` case bodies are never merged: a case
that falls through has no end of its own.

KNOWN INTERACTIONS, byte-level rather than soundness (a wrong merge changes recompiled
bytes and surfaces as a LOST match under the zero-lost gate, the same fallback
`dce.ts` and `basecse.ts` state for themselves): it defeats basecse's
scalar-fixed-offset gate by dropping a repeated offset's count from 2 to 1, and it
does not reach a fixpoint with `eliminateDeadStores`, so a differing DEAD statement at
the end of the arms hides the common tail.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`hoistScopedBases` required ONE scope to contain ALL of a base's uses and declined
otherwise. For an `addr`/`const` base that decline is right — basecse hoists those at
the function top. For a `var` base it was a hole: basecse cannot see a `var` base at
all, so nothing hoisted it and no candidate offered the named spelling.

That hole became live once address numbering removed the phi the lever keyed on: the
base's uses then span the whole function and the lever declined on the row it was
built for. The fallback takes the deepest scope still holding 2+ uses and names the
base for those only, leaving the rest as they were — the mixed form the compiler
produces when it materializes an address in one arm and re-derives it elsewhere.
kleod:UpdateHUDCounterDisplay 33 → 21.

Repointing is SCOPED: a plan entry may own only a subset of its key's uses, so a key
becomes active when the rewrite enters its scope and is restored on the way out.
Repointing by key alone would rewrite uses the hoist does not dominate.

Selection is by DEPTH with no size term, and that is a stated limitation rather than a
model of the compiler: a scope with four uses enclosing one with two names the two.
Only one cluster is served, and a depth tie goes to first appearance. Both are pinned
by tests.

Also in this pass: the hoist is placed immediately before the first use rather than at
the head of its scope (a call in between forces the pointer into a callee-saved
register, the failure this module exists to avoid); a compound `for` init/inc makes it
decline outright, because `collect` and `rewriteStmt` would otherwise disagree about a
tree they must both walk; and a global shadowed by a local is not eligible, since `&g`
would take the address of the local.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
SSA destruction names each merge independently — a non-loop block param adopts a name
only from an INCOMING carrier — so two unrelated phis get two locals where the
compiler held one register. On kleod:UpdateHUDCounterDisplay the reference calls both
`var_r5` for that reason.

WHICH pair was shared is not derivable from the tree, and committing to one gets it
wrong: the two legal merges on that row score 18 and 40 against a no-merge 21, and
declaration order picks the 40. So every legal merge is emitted as its own candidate
and the differ referees — the idiom `/regcopy` already uses for its
allocator-ambiguous tail choice. Row 21 → 18; the losing merge is present and loses.

GATES: neither local mentioned inside a loop BODY — sound-critical, it is what makes
preorder statement order a sufficient approximation of liveness; both constant-fed — a
codegen heuristic rather than soundness, and currently the only thing bounding
candidate growth, which is L(L-1)/2 in the local count with each merge a distinct
compile; the survivor written at its first mention by an assign that does not also
read it; same declared type; params never merged.

ACCEPTED, NOT FIXED: a survivor assigned on only SOME paths still absorbs the other's
value where that assignment is skipped. The original read an uninitialized local
there, so both spellings are ill-defined rather than one being wrong.

Labels name the merged pair rather than an enumeration index, so a change in local
ordering cannot silently re-point a recorded label at a different merge.

POLICY: rank.ts's rule is that re-spellings derive from the base spelling only. These
are enumerated as alternative OUTPUTS of the base hoist, in the one place that knows
the hoist just happened, and the un-coalesced spelling stays in the list. Every pass
invocation stays inside a `respell` thunk.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
kleod:UpdateHUDCounterDisplay:agbcc 60 → 18; asmlift 350 vs m2c 340 unchanged.
0 lost, 0 missing, 0 gained, 0 other flips across 675 rows.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@macabeus macabeus changed the title feat(l3): common-tail merge, scoped base clusters, and ranked live-range coalescing (UpdateHUDCounterDisplay 60→18) feat(l3): common-tail merge, scoped base clusters, and ranked live-range coalescing Aug 3, 2026
@macabeus
macabeus merged commit ba36dc7 into main Aug 3, 2026
2 checks passed
@macabeus
macabeus deleted the match/UpdateHUDCounterDisplay-3 branch August 3, 2026 21:32
macabeus added a commit that referenced this pull request Aug 5, 2026
* feat(structure): anchor constant merge copies at their def sites (/defsite axis)

SSA destruction places a merge copy where its CFG edge is, but the asm often
materialized the constant earlier — movs r9,#0 at entry ahead of a single-armed
overwrite, movs r5,#1 at the top of an arm ahead of a nested if. The new
anchorConstCopies structuring option emits such a copy at the const op's
original program position and suppresses the edge copies it replaces; where the
surviving arm empties, the existing empty-then peephole yields the single-armed
positive if the source wrote.

Sound by refusal, never by approximation: only unnamed const args; the def's
block must dominate every edge source; ANY in-loop shape declines outright
(block dominance is not per-iteration precedence — the /preinit sticky-arm
class from PR #13); the merge variable must name no other value; and a const
whose anchored sibling could clobber it on a path to its edge keeps the edge
placement. Off by default; rank.ts enumerates it as the differ-refereed
/defsite axis crossed with branch sense, so it can never cost a match.

kleod:UpdateHUDCounterDisplay:agbcc 18 -> 6 (the remaining 6 are the bitfield
signedness family, built separately).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(symbols): recover bitfield members — carry, spell, and declare them

The symbol map now carries bitfield members (bitWidth + bitOffset, LSB-first),
and three consumers learn what to do with them:

- structure.ts recognizes the `(x << a) >> b` extract of a struct global's
  loaded bytes as the declared bitfield at exactly those bits and spells
  `gSym.field` — whose `u32 f : n` declaration makes C's own promotion
  reproduce the signedness every downstream operator compiled with (a 7-bit
  unsigned field promotes to signed int, so sdiv renders `/` and recompiles
  to __divsi3 where the raw-shift spelling stayed unsigned). A load whose
  every use is a spelled extract is absorbed — its materialized temp would
  recompile to a second load. Exact, never approximate: position, width and
  signedness must all match, the window must lie inside the loaded bytes, a
  signless field never matches, and a volatile container refuses the fold.
- declaredFields seats members by BIT cursor (co-located bitfields are
  members, not union aliases; a plain member tied at one offset keeps winning
  as before), and drops any bitfield the `u32 f : n` no-straddle allocation
  model cannot reproduce. The exact (offset,size) scalar rules now exclude
  bitfields — a 7-bit field spans 2 bytes and would otherwise match a plain
  u16 read.
- declare.ts renders bitfield runs with `u32 asmlift_pad_N : k` bit padding;
  verified byte-faithful against agbcc (extract shifts, __divsi3 promotion,
  and a cross-byte field all reproduce from the synthesized decl alone).

The provider keeps bitfield facts only for LITTLE-ENDIAN ELFs — both the
extract equation and the layout model are LE-GCC semantics, so a big-endian
map carries no bitfield members at all (exactly today's behavior). A package
reporting bitWidth without bitOffset refuses loudly (the same key-presence
capability gate as every other fact).

kleod:UpdateHUDCounterDisplay:agbcc 6 -> 0: with /defsite this row is now a
MATCH (unsigned/defsite/scopebase-coalesce-v2-v4, exit 0).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore(bench): re-vendor symbol maps — bitfield facts land in the LE projects

kleod +40 bitfield members, pokeemerald +5099, sa3 +5; the big-endian
projects (af, marioparty3, snowboardkids2) correctly carry none — the
provider's EI_DATA gate at work.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(structure): close the adversarial round's two soundness holes + add the /no-bitfield axis

CRITICAL 1 — the bitfield fold moved a captured load past stores and calls:
`g.field` re-reads memory at the render position, but the asm captured the
bits at the load's. The fold now clears the same bar as every other memory
read in the structurer: it refuses whenever a call, or a store that may alias
the folded global, lies between the load and any position the member read
renders at (linear program position, the analysis.ts liveAcrossCall idiom —
a barrier on a disjoint path refuses harmlessly). A materialized shl refuses
too (its temp would still read the load).

CRITICAL 2 — an anchored const write in a switch-tree TEST block was
discarded with the block while its edge copy stayed suppressed (s(1)
returned 0). PRE4 now treats an anchored write as impurity: such a block is
never consumed as a discarded test, so it re-roots the chain instead and its
statements — the write included — are emitted before the dispatch.

MAJOR — the fold was unconditional, and the named read recompiles at the
DECLARATION's access width; where that diverges from the asm's load width
the honest shifts are the spelling that matches. The OFF spelling is now the
/no-bitfield axis, enumerated only when the map carries bitfield members.

Also: TargetDescription.capabilities.endianness gains its first consumer —
structureOptionsFor threads littleEndian into structure(), so a hand-built
BE map cannot reach the LSB-first extract equation (the provider's EI_DATA
gate, now enforced on core's side too). CAST_PATTERNS' width-8/16 shadowing
of the recognizer is documented in both files and pinned by a test.

kleod:UpdateHUDCounterDisplay:agbcc stays a MATCH
(unsigned/defsite/scopebase-coalesce-v2-v4: 0).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(structure): path-based ordering gate for the bitfield fold — block order is not path order

The second adversarial pass broke the first fix: fn.blocks follows ADDRESS
order, so a store block laid out after the render block but executing between
the load and the render on the taken path slipped past the linear-position
scan, and the fold emitted a member read of post-store bits.

The gate now reuses the materialization model's own machinery, newly exported
from analysis.ts: `emitPos` resolves where each extract actually renders
(transitively through its inlining consumers; unresolvable refuses), and
`memWriteBetween` walks every def-avoiding load-to-render PATH — the same
cycle-aware discipline the materialize decisions use — for a call, an opaque,
or a store not provably to a different named global. The kleod row's fold
still clears the gate (its astores hit gBgTilemapBufs, a different global,
on disjoint or later positions) and the row stays a MATCH.

Regression pin: the non-topological layout shape from the audit is now a
test; the linear scan fails it, the path walk refuses it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* bench: refresh results — asmlift 354 -> 360 vs m2c 342 (6 gained, 0 lost)

kleod:UpdateHUDCounterDisplay:agbcc flips nonmatch(diff:18) -> MATCH on
unsigned/defsite/scopebase-coalesce-v2-v4, and the /defsite axis also flips
five synthetic agbcc rows carrying the same pre-init shape: fcmp, iszero,
lor, notb, signum. regression: 0 lost, 0 missing, 6 gained, 0 other flips
over 743 committed rows. Provenance 63f02b6, clean tree.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant