Lowercase the AR, COFF and Universal2 backend names - #799
Conversation
Nineteen of the twenty-two names registered into ALL_BACKENDS are lowercase. Three are not, and _backend_resolver looks a name up exactly, so following the convention every other backend teaches raises CLEError: Invalid backend: ar. Rename the three to match their siblings, and resolve a backend name without regard to case so the old spellings keep working. That second half is not decoration: angr's .adb serializer writes the registered name into the database and feeds it straight back to Loader on reopen, so a database saved from an AR, COFF or Universal2 object holds the capitalized spelling on disk.
|
THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS Selecting each of the three backends by its lowercase name, before and after this the reproducer"""Select a backend by the lowercase name every other cle backend teaches."""
import os
import sys
import cle
from cle.backends import ALL_BACKENDS
binaries = sys.argv[1]
print("cle:", cle.__file__)
print("registered names:", sorted(ALL_BACKENDS))
print()
targets = [
(
"ar",
os.path.join(
binaries,
"tests_src/i2c_master_read-nucleol152re/mbed/TARGET_NUCLEO_L152RE/TOOLCHAIN_GCC_ARM/libmbed.a",
),
{"rebase_granularity": 0x1000},
),
("coff", os.path.join(binaries, "tests/x86_64/fauxware.obj"), {}),
("universal2", os.path.join(binaries, "tests/multi_arch/fauxware_macho_multiarch"), {}),
]
for name, path, opts in targets:
print(f'{name!r} in ALL_BACKENDS: {name in ALL_BACKENDS}')
try:
loader = cle.Loader(path, main_opts={"backend": name}, auto_load_libs=False, **opts)
except Exception as e: # noqa: BLE001
print(f' cle.Loader(..., main_opts={{"backend": "{name}"}}) -> {type(e).__name__}: {e}')
else:
print(f' cle.Loader(..., main_opts={{"backend": "{name}"}}) -> {type(loader.main_object).__name__}')
print()Before — the lowercase name every other backend teaches is not registered: cle at the merge baseAfter — all three resolve, and the registry is uniformly lowercase: cle with this changeThe capitalized spellings that an existing .adb round trip |
|
THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS Validation record for head Every run below used the workspace's Python with
Caveats: |
|
Corpus decompilation diffs can be found at angr/dec-snapshots@master...angr/cle_799 |
|
THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS How much of a corpus's failure surface this removes, measured rather than Sample. 12,000 objects drawn uniformly at random, from a seeded permutation, Method. Each object is loaded with its declared recipe passed through Before. 537 / 11,989 = 4.48% of the sample (CI 4.12–4.86) never reach a After. All 537 resolve, and 261 of them go all the way to CFG — 2.18% of Residual. The other 276 get past the name and stop further in, on defects Case folding also does not reach a separate group of 276 objects whose declared Control. 481 objects that already reached CFG on The corpus is not redistributable, so its objects are described by architecture, session: sharpen |
2 participants
THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS
Problem
Selecting the static-archive backend by name fails, on the lowercase spelling
that every other backend in the registry teaches:
{"backend": "coff"}onbinaries/tests/x86_64/fauxware.objand{"backend": "universal2"}onbinaries/tests/multi_arch/fauxware_macho_multiarchfail the same way, while
{"backend": "elf"}works. The spellings that do workfor these three are
AR,COFFandUniversal2, which appear nowhere outsidetheir own registration lines: not in cle's README, not in cle's docs, and not in
angr's table of backend names. So the only way to find them is to read the source
of each backend, and a caller who generalises from
elf,pe,mach-oorblobgets aCLEErrorinstead of a loader.Root cause
Loader._backend_resolverlooks a name up exactly:and the names it looks up are inconsistent. Nineteen of the twenty-two entries in
ALL_BACKENDSare lowercase;register_backend("AR", StaticArchive),register_backend("COFF", Coff)andregister_backend("Universal2", Universal2)are not. Each was spelled that way in the commit that added the backend, by three
different authors, and no commit message discusses the choice.
Fix
Rename those three registrations to match their siblings, so the convention the
registry already follows holds for every entry, and resolve a backend name
without regard to case.
The second half is what keeps the rename safe rather than being a convenience.
angr's
.adbserializer writes cle's registered name into the database(
LoaderSerializer.backend2name) and hands it straight back tocle.Loaderasmain_opts["backend"]when the database is reopened, so every database savedfrom an AR, COFF or Universal2 object has the capitalized spelling on disk. With
the rename alone, reopening one of those raises
AngrDBErrorfromCLEError: Invalid backend: COFF; with case-insensitive resolution it loads.Registering the lowercase names as extra aliases would do the same job and cost
more:
ALL_BACKENDSis iterated for format autodetection and is the listangr-management fills its backend dropdown from, so each alias would add a
duplicate probe and a duplicate menu entry, and it would make
backend2name, a dict inversion, pick its winner by insertion order.Testing
tests/test_backend_names.pyloads a real static archive, a real COFF object anda real universal binary by their lowercase names, checks that the capitalized
spellings an existing
.adbholds still resolve, checks that an unregistered nameis still rejected, and asserts that every registered name is lowercase, which is
the property the resolver now depends on. Its three lowercase cases and the
lowercase assertion fail at the merge base with
CLEError: Invalid backend: ar,: coffand: universal2. The archive isbinaries/tests_src/i2c_master_read-nucleol152re/mbed/TARGET_NUCLEO_L152RE/TOOLCHAIN_GCC_ARM/libmbed.a,the only committed
!<arch>fixture that loads today.Validation: #799 (comment)
session: sharpen