v0.10.0 — The frame, structs and switches on every toolchain

Upgrading

There are no decomp.yaml or flag changes. Three behaviours differ:

• Exit status 3 is gone, and so is the [candcache] line (#310). The candidate-object cache now serves only the benchmark's own compile commands. A project's tools.asmlift.compiler command compiles every candidate, and the ASMLIFT_CANDCACHE* variables no longer affect it. A wrapper that checked for 3 can drop that branch.
• A typed --proto entry for the decompiled function itself now bounds its argument registers (#344). A register the body reads at or past the words its params fill is an uninitialised local, not a parameter. An incoming stack argument past them refuses. A bare count bounds nothing. A variadic function's own entry should state its fixed parameters as "variadicParams", because a params list or count now also bounds what ... takes.
• --context reads the function's own header entry (#349, #350). It is a soft bound: an argument register past it that the body never reads is dropped. One the body does read stays a parameter, and an [own-arity] note on stderr names it. A struct returned through a hidden pointer lifts as that struct where the pointer is provably only its storage. Otherwise you get an [own-return] note. A --proto own entry marked "fromContext": true is read the same way.

What the decompiler learned

• The stack frame on MIPS and PowerPC. Address-taken locals and incoming stack arguments now lift on O32 (#323) and on CodeWarrior (#327). mwcc's out-of-line register save routines count as frame bookkeeping (#324). An uninitialised local in a register or frame word reads as a local, not as a made-up parameter: on PowerPC (#324), MIPS (#337), and agbcc (#335).
• More of the frame on every target.
  • A handed-out frame address and a stack argument outside the entry block (#315).
  • Wider and literal-moved runtime indexes into a frame object (#318, #368).
  • A frame nothing escapes from kept as one object (#321).
  • Two brace-initialised local arrays kept apart (#369).
  • A DMA read window bounded past a store that misses its control halfword (#370).
• Initialisers. A local array's constant initializer is spelled from the image agbcc copies it from (#314). A static's relocated elements are spelled as the address constants they hold (#316).
• Structs.
  • A small struct returned through a hidden pointer is read by its members, and agbcc passes structs by value (#331), as MIPS now does too (#336).
  • The hidden return pointer travels in a0 on every O32 call (#340).
  • A function's own struct return lifts as that struct (#350).
  • A struct a declared callee returns in registers is read off its local, on agbcc and mwcc (#354).
  • A member past the displacement range is recovered as a member (#352), and a union agbcc would round over its neighbour is declared as one field (#358).
• Calls.
  • A function's own declaration bounds its argument registers (#344, #349).
  • An undeclared call is passed the incoming arguments only it reads (#351), and lifts past a frame word the function loads (#348).
  • A gap in an undeclared call's Thumb argument registers refuses rather than guesses (#341).
  • Every MIPS and PowerPC return is proved to go through the caller's return address (#338), and agbcc's far-jump relay is read as the branch it stands for (#329).
  • A header's register return survives a symbol map that sizes the parameters (#319).
  • A callee-saved register no prologue saves lifts as a file-scope global register variable, and refuses where the compiler cannot declare one (#367).
• Floating point.
  • agbcc's single-precision helpers fold to float arithmetic (#341).
  • int↔float conversions and NaN-correct float compares lift on agbcc, MIPS and PowerPC (#355).
  • A soft-float callee's declared float return is read only as that float (#363).
• Loops. Exitless loops, breaks that carry a value or run code, else-breaks in tested loops, and agbcc's peeled loop test (#330). A top test the compiler folded away is restored (#353). A jump-threaded exit points back at the re-test it skipped (#362). Loop exit tails shared by two arms are copied into each (#365).
• Switches.
  • Jump tables that compilers schedule around, bound by bgtlr, or base outside the dispatch (#326).
  • Tables that end where non-returning arms meet (#317), or at the default block (#334).
  • agbcc's PRE code inside a Thumb dispatch (#361), and what an elided dispatch leaves at each case (#364).
  • A MIPS table whose %lo folds into the table load (#366).
  • A case that breaks on one path and falls through on another (#359).
• Instructions.
  • mwcc's branch-free compares and PowerPC's carry bit (#342).
  • PowerPC loads and stores with update (#345).
  • agbcc's cmn as a compare against the negation (#357).
  • A MIPS global indexed between its %hi/%lo halves (#343).
  • A MIPS delay slot holding the target's stolen first word (#347).
• Volatile. A memory read nobody uses is kept as a volatile read (#333), including the ones the compiler discards into r0 or $zero at -O0 (#339). Context-declared volatile globals use a qualified cast or the bare name as each compiler needs (#322, #325), and mwcc 2.3.3's function-scope statics lift (#325).
• Spelling.
  • Pointer arithmetic is decided from operand roles (#299–#302).
  • Narrow loads, parameters and stores are declared at their width and signedness (#301, #309, #311, #363).
  • Counter-carried walks are re-spelled as indexes (#312).
  • An unsigned compare is spelled unsigned unless the signed spelling matches byte for byte (#332).

The honest numbers

	0.9.0	0.10.0
corpus	1,334 rows	2,368
asmlift matches	785	1,410
m2c matches	467	614
asmlift match rate	58.8%	59.5%
asmlift declines	227	301

• On the 1,334 rows shared with 0.9.0 (every 0.9.0 row), asmlift goes from 785 → 831 matches (+46: gcc 2.7.2 KMC 19, IDO 11, CodeWarrior 7, agbcc 6, GCC 2.7.2 3). m2c stays at 467. Of the 46, 19 are real functions: snowboardkids2 7, pokeemerald 4, marioparty3 3, marioparty4 2, and one each from kleod, sa3 and Animal Forest. On the 421 real functions, asmlift goes 122 → 141 and m2c stays at 80. asmlift's declines on these rows drop 227 → 186.
• On the 1,034 new rows, all synthetic, asmlift matches 579 and m2c 147, and asmlift declines 115. Most were written alongside the capability they test, so they are not a random sample. #328 and #356 add rows for the most common reasons Klonoa functions still declined. No real functions were added.
• No row stopped matching, and no row was removed.
• Fourteen rows moved from wrong C to a decline on purpose.
  • Twelve 64-bit MIPS rows (i2ll, lladd, llorr, llsub, u2ull on IDO and KMC, plus llshl and llshr on KMC) printed an s32 function for a long long return. Since #323 they decline.
  • llorr on CodeWarrior dropped the low word of a 64-bit OR. #342 refuses it.
  • strcmp1 on IDO scored 31 of 31 (nothing matched) and now declines at structuring (#347).
• Eight agbcc float rows still match, now with honest C. In 0.9.0, fadd, fsub, fmul, fdiv, fma1, f2i, i2f and fcmp matched by calling the soft-float helper with no arguments (return __fixsfsi();). Now they print the float arithmetic, cast or compare (#341, #355).

Klonoa: Empire of Dreams census, the 396 agbcc functions lifted with each unit's preprocessed context. 0.9.0 declined 49 of them. 0.10.0 declines 15, and 36 of the original 49 now lift. Lifting is not the same as matching.

Two functions that lifted on 0.9.0 now decline on purpose: sub_0801F648 and sub_08021AD4. Their calls to the unprototyped thunk_GetRandomValue() had a guessed arity and passed made-up arguments. With the arity declared, they lift:

asmlift … --proto '{"thunk_GetRandomValue":{"params":0}}'

Also

• New notes on stderr.
  • [own-arity] names an argument register kept past the function's own context declaration.
  • [own-return] says why an own struct return stayed a pointer.
  • The guessed-arity [proto] note no longer lists runtime helpers or C-standard functions (#341). It names callees by their relocation, and only those of the function being lifted.
• The benchmark runs faster and is easier to run.
  • Native 32-bit compilers, cheap spawns and faster structuring (#307).
  • Cached container compiles for KMC GCC, GCC 2.7.2 and CodeWarrior (#310).
  • One shard per 2.5 GB of memory when no --jobs is given (#320).
  • A memory check before a heavy job starts (#360).
  • Checkouts that build on a Linux host (#303, #304, #305).
• The website's Overview tab is about asmlift alone (#346).

Breaking, for programmatic use

@asmlift/core and @asmlift/cli export their sources (./*), so these changes reach anyone importing them:

• @asmlift/core/structure/loops moved to @asmlift/core/ir/loops.
• @asmlift/core/proto's wordsOf takes the target as a second argument, since even-word alignment of arguments depends on it.
• macroDefinesOf is gone from @asmlift/core/declare and @asmlift/cli/declare. The CLI's declare now re-exports declarationsKeptWithHeaders.
• @asmlift/cli/main no longer exports rankedExitCode. Several @asmlift/cli/candcache helpers are gone (cacheMode, cacheSampleNote, candidateCacheRefusal, COMPILE_ENV).
• @asmlift/cli/callees: guessedArityCallees and guessedArityNote take a full TargetDescription and an optional relocation list. calleeNames takes the relocations too.
• decompile and the ranked entry points accept contextGlobals and ownContextProto. @asmlift/core/pipeline adds ownArityNotesOf.

Install

npm install -g @asmlift/cli

Full changelog: v0.9.0...v0.10.0