Skip to content

bench: 8 more real functions per project — close the size/feature blind spot (192 → 240 rows) - #15

Merged
macabeus merged 6 commits into
mainfrom
bench/real-tier-240
Aug 4, 2026
Merged

bench: 8 more real functions per project — close the size/feature blind spot (192 → 240 rows)#15
macabeus merged 6 commits into
mainfrom
bench/real-tier-240

Conversation

@macabeus

@macabeus macabeus commented Aug 4, 2026

Copy link
Copy Markdown
Owner

Adds 8 real-tier functions per project — 48 rows, 192 → 240 real, 723 total. Board: asmlift 350 match, m2c 342; no pre-existing row moved.

Why

PR #13 shipped an unconditional L3 pass and exactly one benchmark row changed. Instrumenting the pass over all 675 rows showed why: it fires on 1. The same instrumentation over a whole game's worth of klonoa functions showed it fires on 11, seven of which change the emitted C. The difference was size — the shape needs an if whose arms each carry several statements, and the real tier's medians were 9–15 source lines with maxima of 17–71. A capability could be right or wrong and the suite would report the same number either way.

How the rows were chosen

Every candidate was mined from the checkout, then compiled standalone and run through the raising tower before selection, so each entry's build status and outcome were known before it landed. Hit rates were low enough that this mattered — roughly 60/93 for kleod, 44/392 for sa3, 33/150 for pokeemerald, 15/160 for af, 16/70 for marioparty3, 13/70 for snowboardkids2. These counts come from a throwaway screening script that is not in the repo, so take them as reported, not verified; what is verifiable is every row's landed outcome, below.

Where a project had more compiling candidates than slots, the picks are balanced between functions asmlift emits (so gap size can move) and functions that expose distinct capability gaps. On the three MIPS projects that balance was not available: screening ~300 candidates produced zero that asmlift lifts, because essentially every real MIPS function contains a call or a branch-likely. (Same caveat — the screen is not committed. The consequence is visible in the tables: all 24 MIPS rows decline, and every decline is a jal/jalr or a branch-likely.)

Every prependC is verbatim project text — a source file's own #include block, its declaration preamble, or a struct copied from where the project declares it. No authored types.

All 24 new GBA rows were verified to reproduce their project's own build object byte-for-byte (relocations and Thumb bl operands masked, since a call to a same-TU symbol is already resolved in the project's object).


The 48 new rows

Outcome key: diff:N = compiles, objdiff distance N. declined = explicit incompleteness markers. noncompile = marker-free output that fails to build. failed = no usable output.

kleod — agbcc (prior rows: max 58 lines)

function loc asmlift m2c why this function
ProcessInputAndUpdateEntities 136 declined — jump table failed Largest kleod pick, and the largest kleod row in the manifest; switch dispatch over nested loops. Pure size coverage — 2.3× the previous kleod maximum.
UpdateCameraScroll 99 declined — pool word is a symbol+offset noncompile Passes a struct by value to ScrollBGLayer, a shape no row had, and it is the only row that declines on a symbol+offset pool word. (It was also picked as a /coalesce inhabitant, but it declines at lift, so it never reaches L3 — that motivation did not pan out.)
ConfigureEntityBehavior 94 declined — jump table failed A mov pc, r0 jump-table switch at 94 lines. kleod's one prior switch row (TransformSingleEntityToScreen) is also a jump table with the same decline, so this adds size, not a new gap.
CountCollectedGems 85 declined — sp-as-data noncompile 2-D array indexing under nested loops. Declines on sp-as-data (spill slots — the function takes no addresses).
UpdateWorldMapNodeAnim 76 declined — jump table failed Switch + soft-division + shifts in one function; the densest arithmetic mix available.
EntityItemDrop 64 diff:148 noncompile Switch over entity types reaching a genuine bitfield (u32 unk_204_2 : 2) — the only real bitfield row in the set.
SetupBG3WindowOverlay 56 noncompile declined Picked as a /coalesce inhabitant from the klonoa mining (candidate counts not recorded in the row), plus array writes into VRAM-shaped memory. Lands as noncompile — see the findings below.
EntityDeathAnimation 50 noncompile declined A real multi-statement do-while loop. kleod already had one do-while row (StrCpy), so this is depth, not a new feature — and the noncompile it produces is the useful part.

pokeemerald — agbcc (prior rows: max 25 lines)

function loc asmlift m2c why this function
TrySetCantSelectMoveBattleScript 105 diff:303 noncompile The largest pokeemerald function that both compiles standalone and asmlift emits — 4× the previous maximum, and it keeps a gap-size signal at that size.
GetMoveTarget 84 declined — jump table failed Switch → jump table and a do-while in the same function.
SetMauvilleOldManLanguage 64 diff:59 noncompile Takes a union OldMan * and switches on its tag — no prior row on any project (real or synthetic) mentions union. Two other new rows reach unions too (af:_MtxF_to_Mtx, sa3:sub_804DC38); this is the one that both compiles and scores.
Cmd_tryconversiontypechange 60 declined — unrecovered back-edge noncompile A do-while whose loop shape asmlift's recovery declines — a structuring gap rather than a lifting one.
UpdateShoalTideFlag 39 declined — sub-word .byte table declined Reads a static-local .byte table; the only row of its kind.
AcroBikeHandleInputTurning 38 diff:44 diff:2 u8-returning static callees whose return truncations are load-bearing; also the closest head-to-head on the new rows.
AnimTask_FlashHealthboxOnLevelUp_Step 36 declined — union overlap noncompile asmlift's recovered struct has overlapping fields (an artifact of s16 data[] in struct Task), which its union modelling declines. Nothing in the source is a bitfield.
DoForcedMovement 29 diff:31 declined Indirect call through a function-pointer parameter (void (*moveFunc)(u8)), compiled to bl _call_via_r9.

sa3 — agbcc (prior rows: no switch at all, max 71 lines)

function loc asmlift m2c why this function
PackSaveSector 117 declined — sp-as-data noncompile Largest sa3 pick; three switches driving a field-by-field save-sector copy. Declines on sp-as-data — from spill slots, not an address-taken local.
sub_804DC38 92 declined — union overlap noncompile Switch + soft-division over a struct whose fields overlap (a union asmlift cannot model).
Sio32MultiLoadMain 77 declined — jump table failed Switch → jump table driving MMIO serial registers.
sub_802DFC8 65 declined — branch target is a data label match The only row whose decline is branch target is a data label. Also the clearest m2c-wins row on the new set. (Selected believing it had bitfields; it does not — see the correction note below.)
sub_806132C 54 noncompile diff:171 Mid-size entity code asmlift emits. Picked so gap size could move; it ends up noncompile instead, which is the finding below.
sub_804D360 47 diff:57 diff:6 A switch asmlift does lift, so sa3 finally has a switch row with a score.
Sio32MultiLoadIntr 38 diff:76 noncompile MMIO register work in an interrupt handler, and one of the four sa3 picks that produce a score.
sub_803213C 30 noncompile match Small enough to isolate one behaviour, and it produces a noncompile. (Also selected on a false bitfield reading — see below.)

af — ido7.1 (prior rows: max 31 lines)

Every af pick declines, with six distinct reasons. af's asm is FP-heavy and call-heavy, which is exactly what asmlift's MIPS frontend does not model yet.

function loc asmlift m2c why this function
Skin_Matrix_MulMatrix 122 declined — sdc1 store noncompile Largest af pick; 4×4 float matrix multiply, maximum FP register pressure.
_MtxF_to_Mtx 71 declined — lwc1 load noncompile MtxF and Mtx are both unions, indexed as 2-D arrays, with float↔fixed bit twiddling.
suMtxMakeSRT 67 declined — jal declined Float matrix construction that also calls out.
add_calc_a 53 declined — bc1tl diff:98 FP compare feeding a branch-likely — a distinct decline from the store/load ones.
get_percent_forAccelBrake 51 declined — bc1fl diff:78 Float division plus the opposite branch-likely sense.
Skin_Matrix_SetQuaternion 47 declined — sdc1 noncompile Quaternion math with double-word FP stores.
sAdo_Get_KokeruLabel 43 declined — jal diff:4 The only switch available in af's compiling set; af had none.
atans_table 36 declined — bc1f diff:27 Quadrant dispatcher over float compares; the only af row whose decline is a plain bc1f rather than a branch-likely. Despite the name it reads no table — it calls U_GetAtanTable.

marioparty3 — gcc2.7.2 (prior rows: max 17 lines, six features total)

The narrowest project in the suite: its 32 rows only ever exercised struct, pointer, branch, global, arithmetic and loop.

function loc asmlift m2c why this function
ContDataUpdate 50 declined — blezl diff:37 Largest marioparty3 pick — 2.9× the previous maximum — a controller-state loop.
func_80012B14_13714 41 declined — jal noncompile float 3-D projection with division; marioparty3 had no float row.
Hu3DCam3DToScreen 37 declined — jal noncompile The sibling projection path, 2-D array indexed.
func_800230F8_23CF8 36 declined — beql noncompile Struct/pointer loop hitting a different branch-likely form.
HuSprGrpCreate 35 declined — jal declined Allocation + link loop over a sprite group.
DecodeData 34 declined — jal diff:13 Contains a MIPS jump table (jr v0), a shape this project never had — but asmlift declines earlier, on the jal at 0x40, so the table is not what this row currently exercises.
FileSeek 25 declined — beqzl noncompile Switch + ternary in one small function.
HuMemAllocTag 24 declined — jalr noncompile Indirect call through a function pointer — distinct from the direct-jal declines.

snowboardkids2 — gcc2.7.2kmc (prior rows: max 52 lines)

function loc asmlift m2c why this function
func_8005DF10_5EB10 125 declined — jal noncompile Largest; 2.4× the previous maximum.
func_8001E104_1ED04 114 declined — bnezl noncompile Branch-likely–heavy loop, a different decline from the call-blocked ones.
func_8005AE8C_5BA8C 109 declined — jal declined do-while + int64 + goto in one function.
func_800B2C18_A2AC8 98 declined — jal declined goto-based control flow.
func_8005AB58_5B758 82 declined — jal noncompile 64-bit arithmetic over a memory buffer.
func_800B3F2C_A3DDC 80 declined — jal declined Switch dispatch.
func_800B3784_A3634 79 declined — beqzl declined A second switch of similar size that blocks on a branch-likely rather than a call, so the pair separates the call gap from the branch-likely gap. Neither reaches the switch.
func_80038090_38C90 79 declined — jal declined Struct/pointer loop from a fifth file — the eight picks span five source files.

Corrections after an adversarial review of this description

A hostile fact-check of the table above (48 rows, every claim checked against the manifests, the project checkouts and results.json) found six false rationales. They are fixed above, and one of them was a dataset bug, not just prose:

  • Three rows were tagged bitfield and are not. The selection-time detector's regex matched ternaries (? 6 : 2;). struct Task is all scalars, and sa3's SPRITE_FLAG is ((value) << SF_SHIFT(name)) — a shift into a plain u32. Retagged in a separate commit; conversely kleod:EntityItemDrop reads a genuine u32 unk_204_2 : 2 and was not tagged, so it now is. Outcomes and scores are unchanged.
  • kleod already had a do-while row (StrCpy) and its prior switch row (TransformSingleEntityToScreen) is already a mov pc, r0 jump table with the identical decline marker. Those two picks add size and depth, not new gaps.
  • marioparty3:DecodeData declines on jal, not on its jump table — the table is there, but asmlift never reaches it.
  • pokeemerald:DoForcedMovement dispatches through a function-pointer parameter, not a table.
  • kleod:UpdateCameraScroll cannot exercise PR feat(l3): common-tail merge, scoped base clusters, and ranked live-range coalescing #13's coalescing lever — it declines at lift, so it never reaches L3.
  • Two sp-as-data rows were described as having address-taken locals; neither takes an address. The marker's parenthetical is a disjunction and the real cause is spill slots.

What this immediately found

asmlift emits calls that do not type-check against the project's own declarations. Four new rows are noncompile — a first for asmlift on this board:

  • sa3:sub_803213Ctoo many arguments to function VramMalloc
  • kleod:SetupBG3WindowOverlaytoo many arguments to function m4aSoundVSyncOff
  • sa3:sub_806132Cassignment makes integer from pointer without a cast (VramMalloc returns void *)
  • kleod:EntityDeathAnimation — surfaced only after the row was given its project's real u8-returning callee declarations

asmlift recovers call arity from register usage; where the project declares the real prototype and the recovered arity disagrees, the candidate fails to compile in the project's own context. The previous 192 rows never surfaced this.

11 pre-existing rows do not reproduce their project's own build. Auditing all 120 agbcc rows against the projects' build objects: kleod 40/40 equal, pokeemerald 32 equal / 8 differ, sa3 37 equal / 3 differ. Ten differ by -fprologue-bugfix (the canonical flag set has it; pokeemerald never enables it, and sa3 only for BUILD_NAME == sa1), and sa3:VerifyFlashSector_Core because the project builds lib/agb_flash at -O1. For those rows "match" means reproducing our canonical re-compile rather than what the project's build emits. That is the documented design, so changing the canonical flags — which would move rows board-wide — is left as a decision rather than folded in here.

On old_agbcc: m4aSoundVSyncOff is an old_agbcc function (defined in src/m4a.c, which kleod builds with old_agbcc -mthumb-interwork -O2), but its caller SetupBG3WindowOverlay is byte-identical to kleod's own build — the callee's compiler never reaches the caller, which only emits a bl. Only the TU a function is defined in matters for eligibility.

Verification

pnpm bench run + merge → 723 rows, meta.asmlift.dirty: false, stale-check green. pnpm bench fidelity: 1446 script runs — 1446 ok, 0 warn, 0 fail. Only one pre-existing row differs from the pre-#13 snapshot, and that is #13's own effect. pnpm typecheck, format:check, lint clean; 139 benchmark tests and 34 web tests (including the summary-consistency provenance gate) pass. A second full run on this branch reproduced every row measurement-identically.

Supersedes #14 (same content, clean two-commit history).

🤖 Generated with Claude Code

macabeus and others added 2 commits August 4, 2026 01:31
The real tier's blind spot was SIZE: median 9-15 source lines per project,
maxima 17-71. A pass that only fires on a multi-statement if/else had exactly
one inhabitant in 675 rows, so a whole capability moved one number.

Eight functions per project, chosen for features the 192 existing rows never
exercised — jump-table switches, do-while, goto, unions, bitfields, 64-bit
arithmetic, varargs, function pointers, static-local data tables, 2-D array
indexing — with a deliberate mix of sizes (24-136 lines, median 64).

Selection was screened, not guessed: every candidate was compiled standalone
against its own project context and run through the raising tower, so each
entry is known to build and its outcome is known before it lands. Where a
project had more compiling candidates than slots, the picks are balanced
between functions asmlift emits (so gap size can move) and functions that
expose distinct capability gaps.

Every prependC is VERBATIM project text — a source file's own #include block,
its declaration preamble, or a struct definition copied from where the project
declares it. No authored types. Three rows needed context their real
translation unit has and the manifest headers do not: data/trig.h for
UpdateCameraScroll (SIN/COS are macros over gSineTable — without them agbcc
emits calls where the project inlines a table lookup), the u8-returning callee
declarations at code_1.c:426-428 for EntityDeathAnimation, and bike.c's own
static prototype block for AcroBikeHandleInputTurning. kleod's per-file
#include blocks cannot be copied wholesale: include_asm.h pulls raw .s files
into the TU, and engine.c's preamble carries INCLUDE_ASM(...) lines.

GBA rows are canonical-agbcc only: pokeemerald's libc.c/m4a.c (old_agbcc),
librfu_intr.c (ARM mode), sa3's lib/m4a + agb_flash (-O1) and kleod's
m4a/eeprom are excluded. All 24 new GBA rows were verified to reproduce their
project's OWN build object byte-for-byte (relocations and Thumb bl operands
masked).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
723 rows (483 synthetic + 240 real). asmlift 350 match, m2c 342.

The 48 new rows land as 0 match / 7 nonmatch / 37 declined / 4 noncompile for
asmlift, and 2 / 9 / 11 declined / 21 noncompile / 5 failed for m2c — hard for
both, and the decline reasons are the point: MIPS calls, branch-likely, jump
tables, sp-as-data, unions, sub-word data tables.

No pre-existing row moved.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
macabeus and others added 2 commits August 4, 2026 10:17
The 'bitfield' tag on three new rows came from a selection-time detector whose
regex matched TERNARIES (`? 6 : 2;`), not bitfield declarations. None of the
three touches a C bitfield:

  pokeemerald:AnimTask_FlashHealthboxOnLevelUp_Step  only reaches
      gTasks[].data[] — struct Task is all scalars
  sa3:sub_802DFC8, sa3:sub_803213C                   SPRITE_FLAG is
      `((value) << SF_SHIFT(name))` (sprite.h:399) — a shift/OR into a u32

Retagged to what they actually exercise. Conversely kleod:EntityItemDrop reads
a genuine bitfield (`u32 unk_204_2 : 2` in its preamble) and was NOT tagged;
it is now, and it is the only real bitfield row in the set.

The rows stand on their other merits — sub_802DFC8 is still the only
branch-target-is-a-data-label decline — but the dataset should not claim
coverage it does not have.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Tags only — every outcome, score and gap size is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
macabeus and others added 2 commits August 4, 2026 12:15
…TU has

The row's prependC copied code_1.c:404-428 (the EntityDeathStruct typedef plus
the three callee prototypes just above the function), but `m4aSongNumStart` is
declared at code_1.c:30 — outside that span. So the row compiled with an
implicit declaration, and agbcc's warning about it became the FIRST diagnostic
line, which is the one that survives to the published marker.

That made the row's failure read as `warning: implicit declaration of function
m4aSongNumStart`, blaming a benign warning the reference build also emits (and
exits 0 on). With the prototype present the marker names the actual error:
`too many arguments to function m4aSongNumStart` — the same call-arity defect
the other three noncompile rows show.

Only the one prototype this row calls: the sibling m4a declarations reference
`struct MP2KPlayerState`, which this context does not declare, and would trade
one spurious warning for another.

Byte-verified unchanged against the project's own build (384 bytes, equal).
m2c moves declined -> noncompile on the same context.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Only that row moves: its marker now names the real error, and m2c goes
declined -> noncompile on the same richer context (m2c declined 33 -> 32,
noncompile 144 -> 145). asmlift's board is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@macabeus
macabeus merged commit 36c2aab into main Aug 4, 2026
2 checks passed
@macabeus
macabeus deleted the bench/real-tier-240 branch August 4, 2026 12:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant