Skip to content

v0.11.0 — Klonoa's last declines, loops and switches, and callee bodies

Latest

Choose a tag to compare

@macabeus macabeus released this 10 Nov 00:50
· 17 commits to main since this release
5397b76

Upgrading

There are no flag changes. Four behaviours differ:

  • A linked ELF now bounds what a guessed callee is passed (#378, #382). When decomp.yaml sets tools.asmlift.elf, asmlift reads the body of each undeclared callee that the input does not define from that ELF. It reads the ARM body behind a Thumb interworking veneer the same way. It then passes the callee only the arguments its body reads. A callee defined in the same input is bounded by its own body (#375). The stderr arity note names each body it used.
  • Stack arguments of an undeclared call are sized by the callee's body (#424), on agbcc and the O32 compilers (IDO, gcc 2.7.2, KMC). The call passes the incoming stack words its body loads, rather than guessing or refusing.
  • On CodeWarrior 2.4.2 and 2.3.3, the code after an if or switch with early returns is now written once by default (#418, #423). It reads as one join where it used to be copied into each return path. Every other compiler keeps the per-return copies by default. Both readings are still ranked when you score against an object.
  • Some lifts that used to print wrong C now refuse.
    • A function that spells one byte two ways, through a symbol and through a raw address, where the compiler would forward between them (#387).
    • A structuring whose tail copies would pass a budget (#418). This used to overflow.

What the decompiler learned

  • Switches and loops. Most of this release went into the control flow Klonoa: Empire of Dreams' last declines needed.
    • Cross-jumped tails are written back into each switch arm or loop arm that enters them:
      • a sibling's body (#374);
      • a loop's return tail (#380, #396);
      • a default's tail under a PRE dispatch (#392);
      • a break tail holding a whole loop (#398).
    • Two-level exits out of a search loop are read as a flag break, one loop at a time (#391, #395, #399). Where nothing else structures, they are read as a narrow goto L; … L: break; (#415, #416).
    • Two jump-table arms entering one case's body are read as a fall-through plus a goto to the case label (#406).
    • A switch inside a loop body ends at the body's own join (#390).
    • Counted loops are read as for, where it used to be while, on:
      • agbcc, including a for whose continue runs the update (#403) and exits through a relay that carries their values (#412);
      • gcc 2.7.2 with strength reduction (#409);
      • IDO with a != latch and a pointer stride (#413);
      • KMC around a jump table (#419).
    • The join of an if or switch past its early returns is read once (#418).
    • A condition whose second test reads several members off one struct no longer writes the code after it twice (#420).
  • Local arrays and the frame.
    • Two copied local arrays split into separate objects when the runtime index provably stays inside one of them (#402). That proof can come from:
      • a mask;
      • the loop counter's range (#405);
      • a signed remainder by a power of two (#408).
    • Elements of a proven frame part are spelled bare (#410), and IDO's branching signed remainder folds back to % (#410).
    • A frame's end words that only a scalar uses, and a released by-value struct home, go back to the per-slot model (#376, #383).
    • The frame base is taken as a capture wherever its address escapes and no call takes a stack word (#389).
    • A struct-return temp at agbcc's frame base proves that no call takes a stack word (#379).
  • Calls.
    • A guessed call's leftover argument register is resolved per reaching write (#386).
    • Callee arity is bounded by the callee's body, in the same input (#375) or in the linked ELF (#378, #382).
    • An undeclared call's stack arguments are sized by the callee's body (#424).
    • A switch case's parameter that every edge passes as one pure expression is recomputed in the case (#377).
  • Narrow values.
    • A Thumb narrow reload into another register is read as the word slot, narrowed (#381).
    • A narrow reload's word is bounded by its width, so a cmn against a reloaded halfword lifts (#394).
    • A declared scalar global is spelled through its address where its one access disagrees with the declaration's width or sign (#407).
  • Speed. The plain lift of a very large function no longer redoes work whose answer has not changed (#426). Output is byte-identical. Klonoa's sub_0800D188 (about 10,000 lines of C) went from over an hour to 84 s, and the 396 Klonoa functions take 30% less time in total.
  • Symbol maps are read with @gba-kit/debug-info 0.9.0 (#373). A file static that the symbol table places at several addresses gets no borrowed DWARF shape (#387).

The honest numbers

0.10.0 0.11.0
corpus 2,368 rows 2,963
asmlift matches 1,410 1,627
m2c matches 614 619
asmlift match rate 59.5% 54.9%
asmlift declines 301 338
  • On the 2,368 rows shared with 0.10.0, asmlift goes from 1,410 → 1,418 matches and from 301 → 300 declines. m2c is unchanged at 614.
    • The 8 new matches by compiler: gcc 2.7.2 3, IDO 2, agbcc 2, KMC 1.
    • Two of them are real functions, both from Klonoa: Empire of Dreams. DrawLevelHud_DreamStones went from 40 rows off to a match, and WorldMapScreenCheckNewWorldUnlocked from 62 rows off.
    • On the 421 real functions, asmlift goes 141 → 143 matches and 120 → 119 declines. snowboardkids2:func_800B3F2C_A3DDC now lifts.
  • The 595 new rows are all synthetic. asmlift matches 209 of them and m2c 5. asmlift declines 38. They are not a random sample, which is why the overall rate drops:
    • 185 are stress sources for the return-tail join (#423), written to find where that reading loses;
    • most of the rest pin the causes behind Klonoa's declines (#372, #388, #393, #397, #400).
  • No row stopped matching, and no row was removed. No row moved from a decline to wrong C, and no row moved from a lift to a decline.

Klonoa: Empire of Dreams census: the 396 agbcc functions, lifted with each unit's preprocessed context.

lift decline
0.10.0 381 15
0.11.0 396 0

Lifting is not matching. All 15 functions that declined in 0.10.0 now lift.

Release check, npm 0.10.0 vs this release, the same inputs to both:

  • Real benchmark functions: 300 → 302 of 421 lift.
  • Klonoa functions: no function that lifted in 0.10.0 stops lifting.

Full list

  • Make the plain lift of a large function finish by removing quadratic and repeated CFG work, with output unchanged (#426)
  • Size an undeclared call's stack arguments by the incoming stack words its callee's body loads, on agbcc and the O32 compilers (#424)
  • Read the join past early returns as the default where held-out and generated functions lose no match, and rank the other reading beside it (#423)
  • Fold a condition whose second test reads several members off one struct base, so its || no longer writes the code after it twice (#420)
  • Read KMC's strength-reduced for around a jump table (#419)
  • Read the join of an if or switch past its early returns so the code below is written once, decline past a copy budget instead of overflowing, and stop the printers spreading large arrays (#418)
  • Spell a search loop's hit that skips the rest of its case as goto L; … L: break; (#416)
  • Read a search loop's hit that runs code on its way to the case's end as a two-level exit (#415)
  • Read IDO's counted loop as a for (#413)
  • Read a counted for whose exit takes params, through an agbcc relay that carries them (#412)
  • Spell a runtime-indexed element of a proven frame part bare, and fold IDO's branching signed remainder to % (#410)
  • Read gcc 2.7.2's strength-reduced counted loop as a for (#409)
  • Bound a signed remainder by a power of two, and a remainder op by its divisor, in a value's range (#408)
  • Spell a declared scalar global through its address where its one access disagrees with the declaration's width or sign (#407)
  • Spell two jump-table arms entering one case's body as a fall-through and a goto to that case's label (#406)
  • Prove a loop counter's range from its latch, its start and the branches above the access (#405)
  • Split the one frame object where a runtime index provably stays inside one copy (#402)
  • Hand a goto out of both loops out one loop at a time as a flag (#399)
  • Let a loop's exit arms each copy a search loop in a tail they share (#398)
  • Let a loop's early-return arm copy a return tail a path from outside the loop also enters (#396)
  • Spell a loop's two-level exit to its switch arm's end as a flag break (#395)
  • Bound a narrow reload's word by its width (#394)
  • Copy a cross-jumped default's tail into its entrants (#392)
  • Spell an inner loop's goto past the code after it as a flag (#391)
  • End a switch in a loop body at the body's own join where the epilogue merge declines (#390)
  • Take the frame base as a capture wherever its address escapes and no call takes a stack word (#389)
  • Give a file static the symtab places at several addresses no borrowed DWARF shape, and decline a lift that spells one byte two ways where the compiler forwards (#387)
  • Ask a guessed call's leftover argument register of each write that reaches it (#386)
  • Release a by-value struct parameter's spilled home to the slot model (#383)
  • Hand the benchmark harness the linked-ELF callee bodies the CLI reads (#382)
  • Read a Thumb narrow reload into another register as the word slot narrowed (#381)
  • Write a loop's cross-jumped return tail in each arm (#380)
  • Read a struct-return temp at agbcc's frame base as proof that no call takes a stack word (#379)
  • Bound a guessed callee's arity by its body in the linked ELF, and by the ARM body a Thumb interworking veneer enters (#378)
  • Recompute a switch case's parameter that every edge hands one pure expression (#377)
  • Hand a frame's end words that only a scalar uses back to the slot model (#376)
  • Bound a K&R callee's arity by its own body in the same input (#375)
  • Write a cross-jumped sibling's body back into the switch arms that enter it (#374)
  • Bump @gba-kit/debug-info to 0.9.0 (#373)
  • Benchmark rows for the causes behind Klonoa: Empire of Dreams' declines (#372, #388, #393, #397, #400)
  • Tooling: re-vendored symbol maps with a staleness check (#384), a faster bench fan tally (#404), deterministic load-sensitive tests and a platform-skip gate (#421), and a harness orphan check and agent brief template (#422)