Skip to content

Releases: macabeus/asmlift

v0.10.0 — The frame, structs and switches on every toolchain

Choose a tag to compare

@macabeus macabeus released this 22 Oct 22:40
2eeaf8c

Upgrading

There are no decomp.yaml or flag changes. Three behaviours differ:

  • Exit status 3 is gone, and so is the [candcache] line (#310). The candidate-object cache now serves only the benchmark's own compile commands. A project's tools.asmlift.compiler command compiles every candidate, and the ASMLIFT_CANDCACHE* variables no longer affect it. A wrapper that checked for 3 can drop that branch.
  • A typed --proto entry for the decompiled function itself now bounds its argument registers (#344). A register the body reads at or past the words its params fill is an uninitialised local, not a parameter. An incoming stack argument past them refuses. A bare count bounds nothing. A variadic function's own entry should state its fixed parameters as "variadicParams", because a params list or count now also bounds what ... takes.
  • --context reads the function's own header entry (#349, #350). It is a soft bound: an argument register past it that the body never reads is dropped. One the body does read stays a parameter, and an [own-arity] note on stderr names it. A struct returned through a hidden pointer lifts as that struct where the pointer is provably only its storage. Otherwise you get an [own-return] note. A --proto own entry marked "fromContext": true is read the same way.

What the decompiler learned

  • The stack frame on MIPS and PowerPC. Address-taken locals and incoming stack arguments now lift on O32 (#323) and on CodeWarrior (#327). mwcc's out-of-line register save routines count as frame bookkeeping (#324). An uninitialised local in a register or frame word reads as a local, not as a made-up parameter: on PowerPC (#324), MIPS (#337), and agbcc (#335).
  • More of the frame on every target.
    • A handed-out frame address and a stack argument outside the entry block (#315).
    • Wider and literal-moved runtime indexes into a frame object (#318, #368).
    • A frame nothing escapes from kept as one object (#321).
    • Two brace-initialised local arrays kept apart (#369).
    • A DMA read window bounded past a store that misses its control halfword (#370).
  • Initialisers. A local array's constant initializer is spelled from the image agbcc copies it from (#314). A static's relocated elements are spelled as the address constants they hold (#316).
  • Structs.
    • A small struct returned through a hidden pointer is read by its members, and agbcc passes structs by value (#331), as MIPS now does too (#336).
    • The hidden return pointer travels in a0 on every O32 call (#340).
    • A function's own struct return lifts as that struct (#350).
    • A struct a declared callee returns in registers is read off its local, on agbcc and mwcc (#354).
    • A member past the displacement range is recovered as a member (#352), and a union agbcc would round over its neighbour is declared as one field (#358).
  • Calls.
    • A function's own declaration bounds its argument registers (#344, #349).
    • An undeclared call is passed the incoming arguments only it reads (#351), and lifts past a frame word the function loads (#348).
    • A gap in an undeclared call's Thumb argument registers refuses rather than guesses (#341).
    • Every MIPS and PowerPC return is proved to go through the caller's return address (#338), and agbcc's far-jump relay is read as the branch it stands for (#329).
    • A header's register return survives a symbol map that sizes the parameters (#319).
    • A callee-saved register no prologue saves lifts as a file-scope global register variable, and refuses where the compiler cannot declare one (#367).
  • Floating point.
    • agbcc's single-precision helpers fold to float arithmetic (#341).
    • int↔float conversions and NaN-correct float compares lift on agbcc, MIPS and PowerPC (#355).
    • A soft-float callee's declared float return is read only as that float (#363).
  • Loops. Exitless loops, breaks that carry a value or run code, else-breaks in tested loops, and agbcc's peeled loop test (#330). A top test the compiler folded away is restored (#353). A jump-threaded exit points back at the re-test it skipped (#362). Loop exit tails shared by two arms are copied into each (#365).
  • Switches.
    • Jump tables that compilers schedule around, bound by bgtlr, or base outside the dispatch (#326).
    • Tables that end where non-returning arms meet (#317), or at the default block (#334).
    • agbcc's PRE code inside a Thumb dispatch (#361), and what an elided dispatch leaves at each case (#364).
    • A MIPS table whose %lo folds into the table load (#366).
    • A case that breaks on one path and falls through on another (#359).
  • Instructions.
    • mwcc's branch-free compares and PowerPC's carry bit (#342).
    • PowerPC loads and stores with update (#345).
    • agbcc's cmn as a compare against the negation (#357).
    • A MIPS global indexed between its %hi/%lo halves (#343).
    • A MIPS delay slot holding the target's stolen first word (#347).
  • Volatile. A memory read nobody uses is kept as a volatile read (#333), including the ones the compiler discards into r0 or $zero at -O0 (#339). Context-declared volatile globals use a qualified cast or the bare name as each compiler needs (#322, #325), and mwcc 2.3.3's function-scope statics lift (#325).
  • Spelling.
    • Pointer arithmetic is decided from operand roles (#299–#302).
    • Narrow loads, parameters and stores are declared at their width and signedness (#301, #309, #311, #363).
    • Counter-carried walks are re-spelled as indexes (#312).
    • An unsigned compare is spelled unsigned unless the signed spelling matches byte for byte (#332).

The honest numbers

0.9.0 0.10.0
corpus 1,334 rows 2,368
asmlift matches 785 1,410
m2c matches 467 614
asmlift match rate 58.8% 59.5%
asmlift declines 227 301
  • On the 1,334 rows shared with 0.9.0 (every 0.9.0 row), asmlift goes from 785 → 831 matches (+46: gcc 2.7.2 KMC 19, IDO 11, CodeWarrior 7, agbcc 6, GCC 2.7.2 3). m2c stays at 467. Of the 46, 19 are real functions: snowboardkids2 7, pokeemerald 4, marioparty3 3, marioparty4 2, and one each from kleod, sa3 and Animal Forest. On the 421 real functions, asmlift goes 122 → 141 and m2c stays at 80. asmlift's declines on these rows drop 227 → 186.
  • On the 1,034 new rows, all synthetic, asmlift matches 579 and m2c 147, and asmlift declines 115. Most were written alongside the capability they test, so they are not a random sample. #328 and #356 add rows for the most common reasons Klonoa functions still declined. No real functions were added.
  • No row stopped matching, and no row was removed.
  • Fourteen rows moved from wrong C to a decline on purpose.
    • Twelve 64-bit MIPS rows (i2ll, lladd, llorr, llsub, u2ull on IDO and KMC, plus llshl and llshr on KMC) printed an s32 function for a long long return. Since #323 they decline.
    • llorr on CodeWarrior dropped the low word of a 64-bit OR. #342 refuses it.
    • strcmp1 on IDO scored 31 of 31 (nothing matched) and now declines at structuring (#347).
  • Eight agbcc float rows still match, now with honest C. In 0.9.0, fadd, fsub, fmul, fdiv, fma1, f2i, i2f and fcmp matched by calling the soft-float helper with no arguments (return __fixsfsi();). Now they print the float arithmetic, cast or compare (#341, #355).

Klonoa: Empire of Dreams census, the 396 agbcc functions lifted with each unit's preprocessed context. 0.9.0 declined 49 of them. 0.10.0 declines 15, and 36 of the original 49 now lift. Lifting is not the same as matching.

Two functions that lifted on 0.9.0 now decline on purpose: sub_0801F648 and sub_08021AD4. Their calls to the unprototyped thunk_GetRandomValue() had a guessed arity and passed made-up arguments. With the arity declared, they lift:

asmlift … --proto '{"thunk_GetRandomValue":{"params":0}}'

Also

  • New notes on stderr.
    • [own-arity] names an argument register kept past the function's own context declaration.
    • [own-return] says why an own struct return stayed a pointer.
    • The guessed-arity [proto] note no longer lists runtime helpers or C-standard functions (#341). It names callees by their relocation, and only those of the function being lifted.
  • The benchmark runs faster and is easier to run.
    • Native 32-bit compilers, cheap spawns and faster structuring (#307).
    • Cached container compiles for KMC GCC, GCC 2.7.2 and CodeWarrior (#310).
    • One shard per 2.5 GB of memory when no --jobs is given (#320).
    • A memory check before a heavy job starts (#360).
    • Checkouts that build on a Linux host (#303, #304, #305).
  • The website's Overview tab is about asmlift alone (#346).

Breaking, for programmatic use

@asmlift/core and @asmlift/cli export their sources (./*), so these changes reach anyone importing them:

  • @asmlift/core/structure/loops moved to @asmlift/core/ir/loops.
  • @asmlift/core/proto's wordsOf takes the target as a second argument, since even-word alignment of arguments depends on it.
  • macroDefinesOf is gone from @asmlift/core/declare and @asmlift/cli/declare. The CLI's declare now re-exports declarationsKeptWithHeaders.
  • @asmlift/cli/main no longer exports rankedExitCode. Several @asmlift/cli/candcache helpers are gone (cacheMode, cacheSampleNote, candidateCacheRefusal, COMPILE_ENV).
  • @asmlift/cli/callees: guessedArityCallees and guessedArityNote take a full TargetDescription and an optional relocation list. calleeNames takes the relocations too.
  • decompile and the ranked entry points accept contextGlobals and ownContextProto. @asmlift/core/pipeline adds ownArityNotesOf.

Install

npm install -g @asmlift/cli

Full changelog: v0.9.0...v0.10.0

v0.9.0 — 64-bit values, floating point and the stack frame

Choose a tag to compare

@macabeus macabeus released this 07 Oct 00:21
f46f352

Upgrading: two changes to decomp.yaml

Both refuse the run with a message naming the problem, so a config that needs them cannot be misread.

  • A decomp.yaml must meet the decomp_settings spec (#274). It needs name, platform, and versions, each version with its fullname (and its paths). A config with only platform and tools now exits 66:
    asmlift: decomp.yaml: name: Invalid input: expected string, received undefined
    decomp.yaml: versions: Invalid input: expected array, received undefined
    
    A project that builds no versions can write versions: [].
  • The flags placeholder in tools.asmlift.compiler is {{flags}} (#291), replacing {{cflags}}. A command still writing {{cflags}} is refused as an unknown placeholder. The --cflags flag keeps its name and fills {{flags}}.

tools.asmlift is also checked now: a key of the wrong type or a typo such as targt: exits 66 and names the key, where before it was ignored.

What the decompiler learned

  • 64-bit integers (#244, #249, #250, #255). A 64-bit value is one value across its register pair: a carry chain is one 64-bit add, a 64-bit argument crosses a call when a prototype states its width, and a 64-bit return comes back in a register pair.
  • Floating point. The first hardware-float layer for MIPS and PowerPC single precision (#262), agbcc's soft-double ABI (#275), and double arguments and literals (#279).
  • Structs returned by value (#257, #277), through a hidden pointer where each compiler uses one.
  • The stack frame. Address-taken locals at a frame offset (#270), a frame object above the outgoing argument block (#288), several captured addresses a callee writes through kept as one object (#296), and a frame word that survives a call (#276).
  • Calls. One call plan for every frontend, including MIPS calls (#285, #286), and a call through a register, as agbcc's _call_via_rN thunks make (#290).
  • Spelling. A global's pointer value plus a byte offset is no longer scaled by its pointee size when no declaration reaches it (#292); a local narrowed at its reads or at its declaration (#282, #289); a static defined in the lifted body (#273); loops no other form takes as while (1) (#271).

The honest numbers

0.8.1 0.9.0
corpus 1,198 rows 1,334
asmlift matches 654 785
m2c matches 446 467
asmlift match rate 54.6% 58.8%
asmlift declines 303 227
  • On the 1,194 rows shared with 0.8.1, asmlift moves 652 → 719 (+67: gcc 2.7.2 KMC 24, CodeWarrior 20, agbcc 18, IDO 7). m2c holds at 444.
  • On the 140 new rows (94 synthetic, 46 real GBA functions from pokeemerald, kleod and sa3), asmlift matches 66 and m2c 23. Most real ones were chosen because asmlift's own work touched them, or from a random sample; the dataset PRs (#287, #294, #295) say which.
  • Two rows stopped matching on purpose. llshl and llshr on CodeWarrior matched in 0.8.1 by emitting return __shl2i();: the bytes matched, and the C dropped both 64-bit operands. Since #244 they decline ("no model for the runtime helper") rather than publish that.
  • Four 0.8.1 rows are gone: three kleod functions upstream renamed or moved, and one retired synthetic cell.

Also

  • --help and --version answer on stdout with exit 0 (#297).
  • A GameCube or N64 project needs no tools.asmlift.target when its objdiff.json names the compiler of the unit defining --name (#297).
  • Compiles go through @match-kit/compiler (#291), which asmlift shares with Transmuter. Compiler output goes to files rather than pipes, which fixes compilers run under Wine. Failure messages name what happened (crashed (SIGSEGV), did not run (exit 127)), and the compile directory reads <compile-dir>.
  • Scoring goes through @match-kit/scoring (#263).

Breaking, for programmatic use

@asmlift/cli publishes its sources, so these reach anyone importing them:

  • decompileRanked, scoreSource, scoreCandidate and decompileWithReport are async, and CandidateCompiler returns a promise (#291). Core's tracer hook is probeSource.
  • @asmlift/cli/objdiff and @asmlift/cli/objdiff-version are gone; scoring lives in @match-kit/scoring. @asmlift/cli/score keeps scoreObjects, releaseTarget and the score types (#263).
  • @asmlift/cli/config no longer exports loadDecompConfig, DecompConfig, DecompVersion or LoadedConfig; read a config with @match-kit/decomp-yaml and check it with asmliftBlock (#274).

Install

npm install -g @asmlift/cli

Full changelog: v0.8.1...v0.9.0

v0.8.1 — two fixes for dockerized compilers

Choose a tag to compare

@macabeus macabeus released this 19 Sep 22:45
54b1eb7

A patch release. Both fixes predate 0.8.0 — they are not regressions from it. They surfaced
while sanity-checking the published 0.8.0 binary against real rows, and the published 0.7.0 binary
fails on them identically.

Both are hit by a project whose tools.asmlift.compiler runs the compiler in a container.

A candidate's scratch directory was a recycled path

Every candidate compile reused one directory, deleted and recreated under the same name. A
container reaches that directory through a bind mount, and a recycled path does not survive one:
every candidate failed with FATAL: Can't create /work/cand.o: Invalid argument, and the row
scored nothing at all. Each candidate now gets a fresh directory; one stays live at a time, as
before.

The repo had already measured this class in its own harness — 50 of 160 compiles failing on a
reused path against 0 of 160 with a fresh one — and its dockerized toolchains already avoided it.
The helper a real project's decomp.yaml goes through was the one still recycling.

…and the error said nothing

That failure reported the compile command and stopped, because the cause was summarised to its
first line — and a compile failure is the command on line one with the compiler's own complaint
after it. The only sentence saying what broke was the one being dropped.

Visible in the benchmark too: rows that published a single incidental warning, or a bare
# Error: caret with no text beside it, now carry the diagnostic that names the problem.

An unbounded syntax probe does not fail, it hangs

The vendoring-time syntax check spawned the host compiler with no deadline. spawnSync waits on
the stdio pipes rather than merely on the child, so a compiler that exits promptly still blocks for
as long as anything it forked holds stderr — measured once at two and a half hours, with the suite
reporting nothing throughout. It is bounded now, and a timeout is a loud error.

While there: a non-zero exit from that probe was being read as its passing answer, since a
compiler that rejects the unit reports no implicit declarations. A wrong ASMLIFT_CC made the gate
a no-op. It refuses now.

Also

A playground example — a sprite-list walk whose recovery is byte-exact and needs a struct recovered
from offsets alone, a guarded do-while, a short-circuit && and a pow2 divide to all hold at once.
And the documentation the corpus and the CLI had outgrown: ranked-repro.md's section about the
compiler flags now mentions them, asmlift-101.md lists the seventh --target id, and two censuses
the corpus has outgrown are dated rather than quietly left in the present tense.

Install

npm install -g @asmlift/cli

Full changelog: v0.8.0...v0.8.1

v0.8.0 — GameCube and Wii

Choose a tag to compare

@macabeus macabeus released this 19 Sep 18:09
16f09fd

GameCube and Wii arrive, every benchmark row compiles at its own build's flags, and the corpus
grows to 1,198 rows across nine decomp projects.

GameCube and Wii

asmlift lifts PowerPC for CodeWarrior, and the benchmark's real tier now includes three dtk
projects — Animal Crossing, Mario Party 4 and Pikmin — alongside the six it already had.

  • Three CodeWarrior targets: mwcc_233_163n (GC/1.2.5n), mwcc_242_81 (GC/1.3.2) and
    mwcc_247_107 (GC/2.6). Seven --target ids in total, up from five.
  • REL modules. A module's code is placed by the game's loader, so its ELF is relocatable and
    every section starts at zero. --module <name> reads that module's own .plf beside the base
    ELF, unions in the DOL's globals, and narrows the objdiff.json unit lookup — REL code repeats
    names across modules, and a function several units define is refused rather than guessed.
  • Gekko disassembly. The PowerPC reader wants -M gekko; without it objdump decodes the
    console's paired-single opcodes as POWER VSX, turning psq_l f30,120(r1),0,0 into
    lq r30,112(r1) — a different register file at a different offset.
  • C++. Free and non-virtual member functions with word-sized fields, CodeWarrior mangling
    included. Virtual dispatch, references and ctors/dtors decline.
  • Several .text sections per object. CodeWarrior gives one translation unit many sections,
    all named .text and all starting at address 0, so --name is required there and the function
    is read from the section its st_shndx names.

Per-unit compiler flags

A project's units rarely build with one flag set. Every benchmark row now carries the flags its
own build compiles it with, and the CLI takes them from the first of --cflags, the objdiff.json
unit that defines the function, or the words already in your compiler command. The run says which
it used:

asmlift: [flags] -mthumb-interwork -O2 -fhex-asm (compiler command)

Templates gained {{cflags}} and {{cc}}, so a dtk project writes the flags once and lets asmlift
find each function's unit. targetFor(toolchain, cflags) is the library route from a toolchain and
its flags to a codegen profile.

What the decompiler learned

Seven capability rounds, each measured against the corpus:

  • A MIPS high half is a link-time placeholder, not the address zero — a half-materialised
    address refuses instead of becoming a literal 0.
  • A likely branch nullifies its delay slot, so the slot is modelled by placement: its own block
    on the taken edge alone.
  • A PowerPC global is recovered from its relocation, with a per-kind policy that refuses the
    linker names C cannot spell (extern u32 __vt__6System; compiles, and would be wrong).
  • Outgoing stack arguments are consumed where the declaration and the code agree.
  • A void return; is an edge the assembly decides, emitted only where the epilogue's in-edges
    show the source wrote one.
  • A MIPS parameter is narrowed where the object says it was declared narrow — a conjunction of
    a dead home store and a widen-in-place, never one alone.
  • The objdump reader stops losing words it could not decode.

The honest numbers

Raw totals flatter this release, so here is the split.

0.7.0 0.8.0
corpus 1,064 rows 1,198
projects 6 9
asmlift matches 626 654
m2c matches 416 446
asmlift match rate 58.8% 54.6%
  • On the 1,064 rows shared with 0.7.0, asmlift moves 626 → 643; m2c moves 416 → 419.
    That +17 is what the capability rounds bought.
  • On the 134 new rows, asmlift matches 11 where m2c matches 27.

So the match rate falls, and it falls because the GameCube corpus is hard for asmlift and the
benchmark is not arranged to hide that. The remaining PowerPC walls are floats and stack-as-data;
on MIPS they are calls, unmodelled effect instructions, floating-point condition codes and
unrecovered back-edges.

Breaking

  • tools.asmlift.candidateCache is removed. A decomp.yaml that still declares it is silently
    ignored, and that project's candidate-object cache turns back on — unset means on. It only
    ever silenced a refusal asmlift already reaches on its own: a command that runs the compiler
    through docker, ssh, wine or another opaque runtime refuses the cache and says so. If you need it
    off, ASMLIFT_CANDCACHE=0.
  • CompileCommandOptions drops candidateCache and gains cflags and cc, required exactly
    when the command carries the matching placeholder.

Also

@asmlift/cli now documents the ten environment variables it reads, including ASMLIFT_CANDCACHE,
which is on when unset. The pinned m2c baseline moved. @asmlift/core's README no longer
claims the package is unpublished.

Install

npm install -g @asmlift/cli

Full changelog: v0.7.0...v0.8.0

v0.7.0

Choose a tag to compare

@macabeus macabeus released this 14 Sep 23:14
dd0c74e

asmlift 0.7.0 lifts one function's assembly to source that recompiles byte-identical. This release renames the
enumeration's vocabulary
, which is an npm API break with no aliases. It also re-sources the kleod benchmark by ROM
address, and adds the Fan Explorer, a benchmark page that defines, prices and shows every variation asmlift
enumerates.

Measured against 0.6.0

Both releases use the same pinned m2c (ad5c529a). The numbers come from each release's committed results.json.

0.6.0 0.7.0
benchmark rows 1,022 1,064 +42
asmlift byte-exact 581 626 +45
— synthetic rows 501 / 770 546 / 812 +45
— real project rows 80 / 252 80 / 252 ±0 (see kleod below)
m2c byte-exact 407 416 +9
matches lost, over the 986 rows present in both releases 0 (8 gained)

Real decompilation projects, byte-exact rows out of 42 each:

project 0.6.0 0.7.0
pokeemerald (GBA) 22 23
kleod (GBA) 21 19 ¹
sa3 (GBA) 15 15
marioparty3 (N64) 15 15
af (N64) 4 4
snowboardkids2 (N64) 3 4

¹ Not comparable. kleod's 42 rows are now drawn from testyourmine/kleod and keyed by ROM address (#198). They are a
different set of functions, not the same functions scoring worse. No row in the other five projects moved in that swap.

A second check ran the CLI the way a user does, from each project's root. Both releases decompiled 3,541 undecompiled
functions
drawn from the checkouts' own asm/ trees:

  • GBA: sa3, kleod, FireEmblem7J.
  • N64: marioparty3, af, snowboardkids2.

The result was 0 regressed and 1 newly lifted. 3,423 outputs are byte-identical; the other 118 carry the improvements
below. The packages as published to npm were then installed from the registry and run over the same 3,541 functions, and
every output and stderr was byte-identical to the build the release gates tested.

Breaking changes

The enumeration's words are now candidate, fan, winner, variation, dropped, withheld (#199). docs/vocabulary.md
defines them. axis, lever, variant, and label for a candidate's name are gone, and a candidate's name is now its
list of variations.

Package Old New
@asmlift/core/rank-axes the module @asmlift/core/rank-variations
@asmlift/core/rank-variations STRUCTURING_AXES / StructuringAxis (.probeGate, .variantGate) STRUCTURE_VARIATIONS / StructureVariation (.sharedGate, .perLiftGate)
SHAPE_PRODUCTS / SHAPE_SUBSETS / applyShapes STACKED_VARIATIONS / STACKED_SUBSETS / applyStacked
PRE_FAN_PRODUCTS PRE_RESPELL_VARIATIONS
LIVEBASE_ / BASEFOLD_ / UNFOLDED_ / ORDERBASE_ADMISSIONS; BaseAdmission *_HOISTS; BaseHoist
SIGN_CANDS ([i].label) SIGNEDNESS ([i].variation)
@asmlift/core/rank Candidate.label: string, DroppedCandidate.label, WithheldCandidate.label .variations: readonly string[]
Candidate.group Candidate.preference
RankedResult.best RankedResult.winner
composeLevers composeRespellVariations
EnumerateOptions.onLeverError(label, error) onEnumerationError(variations, error)
EnumerateOptions.onAxisGated(suffix) onVariationGated(variation): defsite, not /defsite
@asmlift/core/variation-tokens (new) — VARIATION_TOKENS, VARIATION_KINDS, variationToken, parseVariation, hasVariation, hasVariations, joinVariations, splitVariations
@asmlift/core/variation-definitions (new) — VARIATION_DEFINITIONS, VARIATION_KIND_DEFINITIONS, READER_WORDS
@asmlift/cli RankOptions.onLeverError onEnumerationError
CandidateReport.label (DecompileReport.candidates[]) CandidateReport.variations: readonly string[]
rankedSummaryLine({ best }) rankedSummaryLine({ winner })

Printed output.

  • asmlift: [lever] <name> threw … is now asmlift: [threw] <function> <variations> threw (no candidate from it): ….
  • [ranked] … best <name>: is now [ranked] … winner <name>:.

What is new

  • Fan Explorer (#200). A benchmark tab that catalogues every variation by kind: signedness, lift, structure, respell,
    and symbol map.
    • Every variation has a definition. Its example compiles to two different objects, and "offered when" is read from the
      gate that decides it.
    • It prices each variation by candidates per win, and plots cost against win rate.
    • The function detail shows the winning spelling, and the variations that were considered and lost.
    • Every benchmark row now publishes fanVariations, which variations its fan carried.
  • A closed variation registry (#199, #200). Core mints every variation name through the registry's types, so an
    unregistered name is a type error.
  • Decompilation:
    • Early returns recovered from a one-set-arm select diamond (#188).
    • A short-circuit's branch sense, read per site from all its facts (#163, #187).
    • A cross-jumped store tail, spelled once after the if (#191) or pushed back into every else-if arm (#169).
    • Comparison ladders spelled as ladders when the compiler's layout says so (#170).
    • Bitfield stores of zero (#168), and variable indices into a pointer global's array member (#167).
    • Narrowing casts fused with their scale (#190).
    • Pool addresses advanced in place (#196).
    • Loop accumulators and nested-loop carried values kept in their variables (#164, #185).
    • Unconsumed volatile reads emitted (#183).
    • Kleod's CountCollectedGems, taken from 290 to a byte-exact match over #156–#191.
  • Benchmark and tooling:
    • pnpm bench sweep, a corpus-wide compile-free differential re-lift (#195).
    • Each row's candidate fan and ranking cost in the artifact (#192).
    • An IR oracle for structure(), so a dropped call fails a test (#193).
    • One dated cost doc, with a staleness gate (#194).
    • Reproducing a row with its own script (#178).
    • bench fan (#180) and bench gates (#181).
    • Preflight checks before a run (#175), and denominators on every score (#174).
    • The unmatchable-quirks register (#197).

Known limits

  • N64 is honest but narrow. The wall is still branch-likely instructions, then hardware float. These are loud
    declines, never wrong C.
  • GameCube / PowerPC still has synthetic rows only, and no real-project coverage. Real rows from three GameCube
    decomps are planned next.
  • @asmlift/core ships raw TypeScript. The asmlift binary is bundled and runs on stock Node, but importing the library
    needs a TypeScript loader such as tsx.

Install

npm i -g @asmlift/cli     # the binary
npm i @asmlift/core       # the library (needs a TS loader)

v0.6.0

Choose a tag to compare

@macabeus macabeus released this 07 Sep 02:38
0b30aeb

asmlift 0.6.0 lifts one function's assembly to source that recompiles byte-identical. This release
is a GBA release: every measured gain is on ARM/agbcc projects, and nothing regressed anywhere.

Measured against 0.5.0

Same 743 benchmark rows in both releases, same pinned m2c (ad5c529a), same vendored inputs:

0.5.0 0.6.0
byte-exact matches 363 411 +48
— synthetic rows 302 333 +31
— real project rows 61 78 +17
declined 195 186 −9
noncompile 6 4 −2
matches lost 0

Real decompilation projects, byte-exact rows out of 40 each:

project 0.5.0 0.6.0
kleod (GBA) 17 (43%) 21 (53%)
pokeemerald (GBA) 15 (38%) 20 (50%)
sa3 (GBA) 10 (25%) 15 (38%)
marioparty3 (N64) 12 (30%) 15 (38%)
af (N64) 4 (10%) 4 (10%)
snowboardkids2 (N64) 3 (8%) 3 (8%)

Rows that still do not match got closer: summed objdiff residual over the 131 real rows scored in
both releases fell 3,122 → 2,189, −30%.

A second check, on 2,864 functions sampled straight from the project checkouts' own asm/ trees —
functions no benchmark row was ever drawn from, both releases run over an identical file list:
+12 newly lifted, 0 regressed.

What is new

21 candidate axes, from 3 registered ranked axes to 24, none removed. The bulk of them are one
family: where a value lives. agbcc decides register allocation from how the source spells a
value, so recovering the spelling — not the semantics — is what closes a match.

  • Value and expression homes — /expr-home, /merge-home, /addr-home, /derived-home,
    /livebase, /basefold, /orderbase, /initfirst, /fresh-merge, /unmerge. A base address
    held live across a loop; the value two branch arms each re-derive hoisted to one definition; an
    init moved above or below a zero-trip guard.
  • Uninitialised locals — the IR gained a real undef, replacing a fabricated parameter.
  • Device registers — a volatile scalar value local, const-address pointer inlining, and
    polled reads (/pollread, /pollguard, /inplace).
  • Types and widths — narrow loop counters, parameters agbcc widens in its own prologue,
    member arrays, global shape recovery, an unsigned-divide L3 operator.
  • Control flow — switch fall-through with a shared return, jump-table dispatch shape, the
    pre-update loop family, short-circuit conditions, long-branch trampolines.
  • Thumb frontend — alignment padding read as data or .align, literal-pool handling,
    sub-word data tables.

Benchmark

The corpus grew from 743 to 1,022 rows — 288 synthetic function specs across four toolchains
(ARM/agbcc, MIPS/IDO 7.1, MIPS/gcc 2.7.2 + KMC, PowerPC/mwcc 2.4.2) and 252 rows from six real
decompilation projects. 11 new feature tags. On the full corpus asmlift is at 581 byte-exact
against m2c's 407.

Every row added since 0.5.0 was audited before this release for provisioning symmetry between the
two decompilers, tag accuracy, stale commentary and duplication (#160), and a new gate now checks
those properties mechanically.

Known limits

  • N64 is honest but narrow: two of the three MIPS projects did not move. The wall is one
    instruction family — 152 occurrences of branch-likely (bnezl, beqzl, beql, blezl,
    bgtzl, bnel, bltzl, bgezl, bc1fl, bc1tl) across the corpus, then hardware float.
    These are loud declines, never wrong C.
  • GameCube / PowerPC has 168 synthetic rows and no real-project coverage at all. Treat
    PPC support as unproven on real code.
  • @asmlift/core ships raw TypeScript. The asmlift binary is bundled and runs on stock Node,
    but importing the library needs a TypeScript loader such as tsx.

Install

npm i -g @asmlift/cli     # the binary
npm i @asmlift/core       # the library (needs a TS loader)

v0.5.0 — the Thumb stack frame, and gates that cost something

Choose a tag to compare

@macabeus macabeus released this 16 Aug 19:09
e957d45

@asmlift/core@0.5.0 and @asmlift/cli@0.5.0 are live on npm.

npm i @asmlift/cli   # pulls @asmlift/core ^0.5.0

Where the board stands

asmlift 363 matches vs m2c 342, over 743 rows — identical to 0.4.0. This release wins no new rows, and that is the honest headline. What moved is the column beside it: nine functions asmlift used to refuse now produce output, and two rows that scored one byte off a match now decline loudly, because that byte was a carry being silently dropped.

match nonmatch declined noncompile
asmlift 0.4.0 363 173 202 5
asmlift 0.5.0 363 179 195 6
m2c 342 202 42 157

17 of 743 rows moved: 8 out of declined into scored C, 6 nonmatch scores improved (one from 340 to 217), 2 from nonmatch:1 to an honest declined, and 1 from declined to noncompile, where the compiler now names the gap instead of asmlift refusing before reaching it. Explore it in the benchmark report.

The shape of the release follows from that table. Four rounds built capability against the largest decline family, and five more were spent on the thing the 0.4.0 notes ended on — the gates that hosted CI cannot run.

The Thumb stack frame

stack pointer used as data was the single biggest refusal in the Thumb frontend, and it was one message covering four unrelated situations. Taken apart across four PRs:

  • A frame adjustment is bookkeeping in both spellings (#28). agbcc writes add sp, sp, #N; disassemblers write add sp, #N. asmlift was transparent to the second and declined on the first — an inconsistency between two spellings of one instruction, not a decision about frames. The counts are now recorded next to the predicate, because the first version of that claim had the producers backwards and a reviewer caught it with a grep.
  • Incoming stack arguments (#29). AAPCS puts arguments 5+ on the stack, so a callee reads them at [sp, #N] at or above its own frame. Those are parameters, and they are separable from locals, which live below the frame top. Modelled by reading a virtual @sarg<k> key that the existing live-in path already turns into a parameter — no new opcode, pass or representation.
  • Local stack slots (#30). A spill whose address is never taken is transparent to dataflow: str rX,[sp,#k] … ldr rY,[sp,#k] moves a value. Now SSA variables keyed by offset, so a slot read-modify-written across a loop becomes one variable with a phi. Braun's phi construction does the hard part.
  • Five stacked capabilities (#31), which cleared lift, raise and recovery on the target function and left it declining at structure on a back-edge — a different tier, named precisely. Across 2,686 sa3 and Klonoa functions: 34 decline → lift, 0 lift → decline, 0 output changes.

The guard that remains is narrower and lives in one place. #28 ended by moving it from three decode arms onto the write itself, after a reviewer ran the lifter instead of reading it and found five arms that wrote sp and dropped it in silence — lsl sp, r4, #2, ldr sp, [r0, #4], and ldmia r0!, {sp} lifting as return a0 + 4. An enumeration of decode arms can only cover the arms someone thought of.

Jump tables, by instruction rather than by spelling

The recogniser matched its idiom by operand text, so it declined the shape whenever a disassembler wrote it equivalently but differently: lsls rejected, #0x02 rejected, adds rejected. In Thumb-1 the trailing s is a dialect marker, not a modifier — and it is the dominant dialect of the format asmlift advertises, since every pret-style split wraps its bodies in .syntax unified, where the non-suffixed spelling is a syntax error. Counted under one such tree: lsls 5,795 against lsl 0 (#34).

Separately, arms that share a body or fall through are now recovered (#32). SwitchCase.values and fallsThrough had always been renderable, and the L3 passes were already written around them — nothing had ever produced either. Both jump-table rows in the benchmark now score, and m2c declines both.

Correctness

A jump table recovered wrong is not a wrong expression, it is a wrong block — the C looks entirely ordinary and runs the wrong code. Fixing the spelling turned up three ways that could happen, all closed in #34.

An unmodelled instruction may not vanish because its destination is dead (#35). opaque.ts implements this project's first rule — an unmodelled instruction degrades to a loud failure, never a silent drop — and that contract held only while the fabricated destination stayed live. A dead one was reaped by DCE, and even when kept was skipped by structure's sideEffects, which had no opaque arm. What it cost, in this repo's own benchmark:

// long long lladd(long long a, long long b) { return a + b; }
s32 lladd(u32 a0, u32 a1, u32 a2, u32 a3) { return a0 + a2; }   // the adc carry, gone

Those are the two rows that went from nonmatch:1 to declined. In a 12,420-function corpus sweep the same hole emptied whole functions — void sqrtf(s32 a0) { return; }, thirteen float instructions.

The value-home fix (#33) landed as safety infrastructure first: one shared memory-disjointness query, where "could this write change what that read sees" had been answered at three different strengths with the weakest governing the most common case. The behaviour change ships as a differ-refereed axis (/reread-globals) — five rows better, none worse, no match flips.

Gates that cost something

benchmark.yml installed every toolchain the byte-level suites need and then never ran them (#36). Both missing gates were the difference between a claim and a guarantee: pnpm test:matching — 279 byte-exact tests — was invoked by no workflow at all, and a lost match was not a failure, so a match → nonmatch flip could be committed as a routine refresh. Both are now wired, and seven mechanisms were re-measured by ablation to establish which are actually guarded by a row.

coalesce.ts justified its loop-body exclusion as sound-critical and then admitted no such harness was committed. Now it is (#37) — a differential fuzzer that interprets the original and every candidate the pass emits, with control flow from a pre-drawn schedule so both trees take the same path by construction. Gates as shipped: 176 candidates, 0 changed reads. Loop flag suppressed: 376 candidates, 55 changed reads.

Then gates became data (#38). A gate is now {id, why, sound, guardedBy, rejects}, so sound: true costs something: ablation is a value, guardedBy is matched against real test titles (which caught a stale one immediately), and an earlier ~45-line ablation-by-input hack was deleted. The same PR found that the "three uncoordinated address-placement policies" are four, and that the unguarded edge — gvn hoisting to the entry block — is safe only because another module keys a rule on opcode === 'const'. Widen it and the function-top local gvn exists to delete comes back, with the entire offline suite passing.

That is the lesson worth carrying out of this release: a test can pass for the wrong reason, so every refusal test now needs a positive control.

Also

  • docs/ refreshed for what these rounds changed (#39)
  • 10 new test files and roughly 2,200 lines of tests against 2,600 lines of core source — the suite now stands at 868 offline core/cli tests (1,127 across the whole workspace) plus the 279 byte-exact matching tests that hosted CI still cannot run and that benchmark.yml finally does

Full changelog: v0.4.0...v0.5.0

v0.4.0 — Thumb fidelity, and a HUD counter that took four rounds

Choose a tag to compare

@macabeus macabeus released this 10 Aug 15:46
e0c1dce

@asmlift/core@0.4.0 and @asmlift/cli@0.4.0 are live on npm.

npm i @asmlift/cli   # pulls @asmlift/core ^0.4.0

Where the board stands

asmlift 363 matches vs m2c 342, over 743 rows — up from 343 vs 340 over 675 rows at 0.3.0. The corpus grew by 68 rows while the gap widened from +3 to +21.

match nonmatch declined noncompile
asmlift 363 173 202 5
m2c 342 202 42 157

Across five toolchains (agbcc, ido7.1, gcc2.7.2, gcc2.7.2kmc, mwcc_242_81). Explore it in the benchmark report.

The headline: one function, four rounds, six general capabilities

kleod:UpdateHUDCounterDisplay could not even be compiled at 0.3.0. Four rounds took it noncompile → diff:81 → diff:60 → diff:18 → MATCH. Each round was chosen by bisecting the gap — mutating the byte-exact reference one axis at a time and scoring each — before any code was written, so what shipped is general capability rather than a fix shaped like one row:

  • Control-flow short-circuits (#11) — if (a || b) X else Y produces no value at all, just two cond_br blocks sharing a target. Nothing recognised that, so the structurer tail-duplicated the shared block: correct C, but not the C the compiler compiled.
  • Scope-aware base hoist + address numbering (#12) — and it retired a wrong call from the round before, which had written off a += 0; in the reference as an unmatchable source quirk. A named base does the same thing to codegen, and it is an ordinary decomp idiom.
  • Common-tail merge, scoped base clusters, ranked live-range coalescing (#13) — which pair of locals the allocator coalesced is not derivable from the L3 tree, and first-fit gets it wrong, so every legal merge is emitted as a candidate and the differ referees.
  • Def-site anchored copies + bitfield members (#21) — SSA destruction places a merge copy where its CFG edge is, but the asm often materialised the constant earlier. The /defsite axis alone flipped five more synthetic rows carrying the v = 0; if (…) v = 1; idiom.

Several of these ship as differ-refereed candidate axes (/scopebase, /coalesce, /defsite): the alternative spelling is emitted alongside the default and the objdiff byte-diff picks the winner, so an axis can win a row and by construction cannot cost one. The passes that do run unconditionally — short-circuit recovery, common-tail merge — earn it structurally instead; mergeCommonTails moves a statement that ends every arm below the if, which runs it on the same paths in the same order, no liveness analysis required.

Thumb fidelity

  • Legacy mnemonic spellings (#25) — ldsh, ldsb, ldm, stm, ldmfd are not similar instructions to ldrsh/ldrsb/ldmia/stmia; they are the same instructions with two accepted spellings, and a disassembler emitting the legacy ones got a decline. Verifying that against ARM's own documentation turned up two semantic bugs, one of them a silent miscompile where ldm rN, {rD} deleted the load entirely. Measured over all 46 functions in the Klonoa: Empire of Dreams corpus containing an affected mnemonic: 13 gained, 0 lost.
  • && short-circuit returns (#26) — if (a && b) return X; return Y; had silently stopped matching byte-exact back in #11: branch fusion runs before return-sinking and destroyed the CFG signal it gates on. Neither hosted CI nor the benchmark covers that shape, so it survived twelve merged PRs. Found while cutting this release.

Ported from m2c

The pin was bumped to upstream ad5c529 (#22) and its commits worked in two passes — nice-to-haves (#23), then the four worth porting (#24). Net +3 matches, and three silent miscompiles found and fixed, two of them pre-existing and unrelated to the ports.

Two commits were verified as nothing to port and two were deliberately discarded, with the reason pinned as tests so the decision does not quietly rot. The one that paid most had been scanned ARM-only: seven MIPS rows carry xori …, 1, because MIPS has no set-on-greater-equal.

Also in #24: jump-table support turned out to exist but only recognised a spelling agbcc rarely emits — the m2c port was the smallest of three gaps standing between asmlift and six functions that dispatch through a table.

Correctness

Silent miscompiles are the worst failure a decompiler has, because the output looks fine. Fixed this release: the logical-shift miscompile (#9), ldm rN, {rD} deleting its load (#25), and three more found while porting from m2c (#23). Two noncompile rows also closed (#9).

Benchmark and webapp

  • 240 real rows, up from 192 (#15) — chosen to close a measured size/feature blind spot
  • A feature vocabulary that is machine-checked rather than hand-asserted (#16, #18, #19), and surfaced in the webapp so each tag explains itself (#20)

Also

  • docs/releasing.md (#27) now records the release procedure — including the two gates hosted CI cannot run, both of which caught real problems while cutting 0.4.0

Full changelog: v0.3.0...v0.4.0

v0.3.0 — the project's symbol map

Choose a tag to compare

@macabeus macabeus released this 02 Aug 13:26

@asmlift/core@0.3.0 and @asmlift/cli@0.3.0 are live on npm.

npm i @asmlift/cli   # pulls @asmlift/core ^0.3.0

What's new since 0.2.0

The project's symbol map (#8)

Point asmlift at your project's built ELF with one decomp.yaml key, and it spells the code the way your project does:

tools:
  asmlift:
    elf: rom.elf

Three channels feed the map — each one your ELF carries is used:

  • .symtab — names for globals and functions: gInputState instead of *(u16 *)0x03004668
  • DWARF (-g) — declaration shapes (gState.timer, gBlendModeTable[i], signedness, volatile/const) and callee signatures for every function the ELF compiled from C — arity drives call-argument recovery
  • .debug_macinfo (-g3) — address-cast macro names (#define gCounter (*(u16 *)0x03001234)), which no symbol table can carry

Every map fact is a ranked lever, never an override: the named spelling and its raw-address sibling are both scored, and --score-against's byte-diff picks the winner. Scored candidates declare themselves — no header wrangling, and the compile world is probed, not configured.

The CLI README covers how to produce a good ELF; the Klonoa decomp's asmlift-elf target is a worked example with all three channels.

Also

  • Playground: a Symbols pane ranks with your map in-browser, plus a byte-matching symbol-map example
  • Benchmark: all 192 real rows run symbol-fed with full provenance and reproduction scripts — asmlift 343 matches vs m2c 340 across 675 rows
  • Depends on the new @gba-kit/debug-info@0.4.0

Full changelog: v0.2.0...v0.3.0

v0.2.0

Choose a tag to compare

@macabeus macabeus released this 22 Jul 16:44
942917f

First tagged release of the published packages — @asmlift/core@0.2.0 and @asmlift/cli@0.2.0 are live on npm.

npx asmlift <file.s|file.o> --target <agbcc|ido7.1|gcc2.7.2kmc|gcc2.7.2|mwcc_242_81>
# or
npm i @asmlift/cli   # pulls @asmlift/core ^0.2.0

What's new since 0.1.0

N64 / MIPS support

  • Read Splat-disassembled MIPS .s — decompile functions from N64 projects that split assembly with Splat (#1)
  • Recover MIPS global-variable access — %hi/%lo relocation pairs are lifted back to named globals (gaddr) (#2), including recovery straight from an object file's objdump relocations when the input is .o (#4)
  • New gcc2.7.2 target (mainline GCC 2.7.2, -O1) alongside the existing gcc2.7.2kmc — surfaced as a CLI --target, and added to the benchmark with 32 Mario Party 3 functions (#5, #6)

Toolchain / API

  • decomp.me-standard target ids — target keys now match the compiler names decomp.me uses (agbcc, ido7.1, gcc2.7.2kmc, gcc2.7.2, mwcc_242_81) (#3)

Pull requests

  • #1 read Splat-disassembled MIPS .s (N64 projects)
  • #2 recover MIPS global-variable access (%hi/%lo → gaddr)
  • #3 rename target ids to decomp.me-standard compiler names
  • #4 recover MIPS globals from objdump relocations (reloc→gaddr bridge)
  • #5 add the gcc2.7.2 toolchain + 32 Mario Party 3 benchmark rows
  • #6 register gcc2.7.2 as a --target
  • #7 release bump to 0.2.0

Full changelog: https://github.com/macabeus/asmlift/commits/v0.2.0