[AUTOMATED] feat(analysis): relocrebase - rebase load-time analysis facts for relocatable objects (DIV-79)

[mahaloz]

Fixes #289

Root cause

relocobjects (DIV-8/DIV-70) lays a relocatable object — an ELF ET_REL .o, a COFF .obj — out synthetically above RELOC_BASE, so every address the engine holds is a post-layout one. The load-time analysis passes never learned that: each re-parses the same file through its own object::File and therefore computes pre-link, section-relative addresses. Both spaces then landed in one inventory.

The visible half, on the in-repo decompiler/crates/kuna-analysis/tests/fixtures/ptx.o:

kuna functions     ptx.o  ->  95 entries: 26 real (0x400000+), 42 externs, and
                              27 PHANTOMS at 0x0, 0x20, 0x34, 0x5c, ...
kuna decompile-all ptx.o  ->  26 functions

— the two surfaces disagreed on a .o, which is exactly the invariant DIV-68 established. The phantoms decompose into two distinct mechanisms, and neither is a plain "off by a base address":

• 26 of them are .eh_frame FDE initial_location fields whose PC-relative relocation the linker has not applied. An unrelocated PC-relative field reads back as 0, so the FDE oracle computes section_addr(0) + field_offset + 0 — i.e. the FDE's own offset in .eh_frame — which happens to fall inside .text's [0, 0x2d9a) pre-link extent and survives every plausibility filter. 0x20/0x34/0x5c are the FDE bodies at .eh_frame+0x18/0x2c/0x54.
• 1 is DWARF. DW_AT_low_pc is likewise relocated, so every subprogram reads 0 — and so does every DW_FORM_strp, so the whole object's DWARF collapsed onto one function at address 0 named after whatever string sits at .debug_str+0 (output_format, which is not even a function).

The silent half: string literals and DWARF-named globals were keyed to pre-link addresses and never attached to the loaded image at all (ptx.o recognized 2 string literals out of 41).

#286 fixed the Listing/xref half by declining for such an object. Declining here would throw real information away — a -g .o carries full DWARF — so this rebases instead.

The fix: rebase the INPUT, not each output fact

A fact is a bare u64 by the time it reaches AnalysisOutput, and in a relocatable object every section sits at address 0 — .text+0x20 and .rodata+0x20 are the same number — so a post-hoc, per-fact rebase cannot tell which section's delta to apply. And the fields that matter are not offsets at all until their relocation is applied (see above).

So decompiler/crates/kuna-analysis/src/loader/kuna_relocrebase.rs re-presents the object to the analyzer tier before any pass reads it:

1. every laid-out section's contents are replaced by the loader's own relocated bytes (so .text/.eh_frame read exactly as the engine decodes them),
2. every unlaid section that carries relocations — the .debug_* tables — has them applied here: a target in a laid-out section resolves to that section's load VMA, a debug-to-debug target to its own section-relative offset (S = 0, which is what a single-object link leaves in place),
3. each laid-out section's address field (ELF sh_addr, COFF VirtualAddress) is set to its load VMA, and
4. each ELF symbol defined in a laid-out section has st_value shifted by its own section's delta — sections are laid out non-contiguously (alignment padding, the empty-section skip), so there is no single global offset. A COFF symbol needs no separate shift: object reports it as VirtualAddress + value, so step 3 already moved it.

Every pass then produces an already-rebased fact with no source change of its own. The section→VMA and extern-slot maps come from the layout the loader already builds (RelocLayout::section_vma / extern_addr, newly exposed).

Safety net. A field with no relocation still yields an address in no laid-out section (a hand-written .eh_frame, an SHN_COMMON symbol whose st_value is an alignment, a symbol in a discarded section). retain_in_image drops exactly those — the phantom class — rather than passing a pre-link address through. One documented exception: a NoReturnFact is kept with its address zeroed, because the commit resolves it by NAME when the address does not resolve (an undefined exit in a .o has always had address 0, and dropping the fact would lose a real no-return marking).

Per-pass coverage

Pass (option)	Address-keyed facts	How it is rebased
noreturn_known	noreturn	shifted st_value; an UND libc symbol keeps address 0 and still resolves by name
strings	strings	section scan reads the patched sh_addr/VirtualAddress
entry_disc	entries, entry_names, context_paints	.eh_frame FDE fields relocated by the byte splice; prologue scan + in_executable_section read patched section addresses; funcsym filter reads shifted st_value
eh_frame_full, fdeinterior	entries, fde_bodies	same .eh_frame splice
funcstart_patterns, cortexmvectors	entries, context_paints	patched section addresses
arm_markers	context_paints	shifted st_value ($t / STT_FUNC LSB) — this is what makes arm_thumb_le32.o decompile at all
mips_gp, mips_isa	tracked_regs, context_paints	shifted st_value
cppsig	cpp_sig.proven / .inferred (address-keyed)	shifted st_value
dwarf (+cppproto, typedepth)	symbols, data_objects, locals, cpp_dwarf.*	.debug_info/.debug_str/.debug_rnglists/… relocations applied
dwarf_lines	comments	.debug_line relocations applied
operand_refs (deferred, default-off)	strings, readonly	takes the same rebased view in run_operand_refs
libproto, libcsigs, callfixup	—	name-keyed; no address to rebase
itaniumrtti	—	driven by ELF dynamic relocations; structurally inert on an ET_REL
peimportcall	externref	needs a PE import directory; inert on a plain .obj
datasyms	loader-side data symbols	already rebased by the loader (not an analysis pass)
Listing tier + consumers (noreturn_disc, noreturn_propagate, fid, aif, ptrentry, poolentry, tailcallentry, fast_funcdisc)	—	#286's decline stands, unchanged (see follow-up)

Deliberately excluded

• MSVC CodeView in a .obj (.debug$S/.debug$T) — kuna's pdb pass is PE-only and reads an external, fingerprint-matched .pdb; it never parses a .obj's embedded CodeView, so there is no address-keyed fact to rebase. A .obj CodeView reader is a separate feature.
• Mach-O relocatable objects — ObjectFormat::relocatable_layout is false for Mach-O (DIV-70), so a Mach-O .o loads through the ordinary mapped-segment path and its analysis addresses already are the loaded ones. macho_dwarf.o/macho_min.o are byte-identical before and after (their sub-RELOC_BASE addresses are correct).
• Big-endian ELF ET_REL — the header patcher declines rather than emit a half-patched image (the sibling relocation engine in reloc_object is little-endian-only too), so such an object keeps today's behavior.

Before / after

kuna functions (entries below RELOC_BASE are the phantom class):

fixture	before	after
ptx.o	95 entries, 27 below RELOC_BASE	68 entries, 0 below
fid_lib_x86_64.o	6 entries, 3 below	3 entries, 0 below
arm_thumb_le32.o	2 entries, 2 below	2 entries, 0 below
coff_obj.obj	2, 0 below	2, 0 below
coff_comdat_i386.obj	3, 0 below	3, 0 below
msvc_mangled.obj / pe_min.obj / ftol_i386.obj	2–3, 0 below	unchanged
macho_dwarf.o / macho_min.o	unchanged (out of scope)	unchanged

The two surfaces now agree on ptx.o: functions reports 26 real functions + 42 extern stubs and nothing below RELOC_BASE; decompile-all still reports 26. The 26 real entries are byte-identical to before — the option only removes the phantom half.

DWARF now lands on the right addresses (readelf --debug-dump=info ptx.o shows main's DW_AT_low_pc relocated against .text.startup+0, which the layout places at 0x404280 — where main is):

// before                                   // after
unsigned int to_uchar(unsigned int a0)      char to_uchar(char ch)
{ return a0; }                              { return ch; }

Strings / named globals now attach to the loaded image — ptx.o recognized string literals 2 -> 41, and the DWARF/.symtab globals resolve:

// before                                   // after
if (dat_403520) { ... v8 = 0x403160;        if (ignore_case) { ... v9 = folded_chars;
error(1,0,dcgettext(0,0x40413c,5),v6);      error(1,0,dcgettext(0,"invalid line width: %s",5),v7);

Two more object fixtures moved, both strictly better:

• arm_thumb_le32.o: decompile-all went from [] (nothing) to both functions — the ARM TMode context paints from arm_markers now land at the rebased addresses, so the Thumb code decodes as Thumb.
• msvc_mangled.obj: int Bar::foo(unsigned long long a0, int a1) -> int Bar::foo(Bar *this, int a1) — cppsig's address-keyed prototypes now match.

Gating

Default-ON (DIV-79): the addresses it replaces are provably wrong — they name a different address space than the one the engine decodes in — so this is a correctness fix, not a judgement call.

The whole analyzer tier runs inside load file, upstream of every per-function option, and unlike the commit-gated passes (datasyms, itaniumrtti, …) this gate cannot be deferred to the commit: it changes the inputs every pass reads, so honouring it at commit would mean running the whole pass list twice. It therefore uses the i386_pie_plt/relocobjects env-var bridge (KUNA_RELOCREBASE), exported by kuna decompile onto the decomp_dbg subprocess and by decompile_all::apply_loadtime_env onto its own process before bootstrap_from_object. The Architecture::analysis_relocrebase bool exists for catalog visibility and the phase catalog live current field.

Proof both CLI paths honour it, both directions (ptx.o):

# in-process driver (functions / decompile-all)
kuna functions ptx.o                                 -> 68 entries, 0 below RELOC_BASE
kuna functions ptx.o --option relocrebase off        -> 95 entries, 27 below  (== origin/main)
kuna decompile-all ptx.o                             -> 26 functions
kuna decompile-all ptx.o --option relocrebase off    -> 26 functions

# subprocess driver (kuna decompile -> decomp_dbg)
kuna decompile ptx.o to_uchar                        -> char to_uchar(char ch)
kuna decompile ptx.o to_uchar --option relocrebase off -> unsigned int to_uchar(unsigned int a0)  (== origin/main)

# the raw env gate on decomp_dbg
KUNA_RELOCREBASE=on  decomp_dbg  ...                 -> char to_uchar(char ch)
KUNA_RELOCREBASE=off decomp_dbg  ...                 -> unsigned int to_uchar(unsigned int a0)

Off is byte-identical to origin/main (a fresh build of 40c829e): decompile-all --json compared over tests/bug-repro/{grep,sort,faillog,libselinux.so.1} **and all ten .o/.obj fixtures` — 14/14 identical.

Collateral

Linked images are structurally untouched (rebased_view returns None unless reloc_object::is_synthetically_laid_out), and it is measured, not just argued:

• decompile-all --json, on vs off, over tests/bug-repro/{grep,sort,faillog,libselinux.so.1} — identical, all four.
• functions --json, origin/main vs this branch, over the same four plus betaflight_STM32F405.elf — identical, all five. betaflight decompile-all — identical too (background run, BF_IDENTICAL).
• The only changed outputs anywhere are the three relocatable objects listed above; every changed function was reviewed and every change is the rebase landing (DWARF names/types, string literals, named globals, Thumb decode).

Speed

Interleaved min-of-11, origin/main build vs this branch, alternating run by run (the box was shared with three other agents' decompile sweeps, which is exactly why min-of-N interleaved):

target	origin/main	this branch	delta
decompile-all tests/bug-repro/grep --json (the no-op path)	15.02 s	14.40 s	-4.13 %
decompile-all ptx.o --json (the .o path)	1.52 s	1.54 s	+1.32 % (+20 ms)

The grep delta is contention noise in kuna's favour — that path is structurally unchanged (rebased_view returns None before doing any work on a linked image). The .o cost is real and is the whole budget of the feature: one extra layout_relocatable (the loader's own layout is not plumbed through to the analyzer tier), one bytes.to_vec(), and the relocation pass over the non-alloc .debug_* sections — +20 ms on a 120 KB, 1067-.debug_info-relocation object.

Tests

No stages XML is possible — that path never constructs an ObjectLoadImage (it uses LoadImageXml over <bytechunk>), so tests/stages/, decompiler/crates/kuna-base/src/xml.rs's corpus count and docs/baseline-stages.json are all untouched. The gate is the cargo suite instead:

• kuna-analysis unit tests (loader/kuna_relocrebase.rs) — ELF ET_REL sections/symbols/FDE starts all land in the loaded image, .debug_info really is relocated, both COFF .obj fixtures rebase, the gate declines when off, and a linked ELF is never touched.
• kuna-console/tests/verify_relocrebase.rs — the DIV-68 agreement invariant over ptx.o + both .objs + arm_thumb_le32.o (no inventory entry is a pre-link address), and the DWARF parameter name/type reaching the rebased function end-to-end.
• kuna-console/tests/verify_relocrebase_gate.rs — the on/off arms, in their own test binary because the gate is a process-global env var; the off arm asserts the phantoms return, so the on arm's clean inventory is a property of this option and not of the fixture.

Counters bumped

• phases.toml settable row (all fields incl. tier/symptoms), p0_knowledge/options.rs KUNA_OPTION_NAMES, kuna_console.rs kuna_live_value, Architecture::analysis_relocrebase (+ default-on), set_kuna_option arm.
• kuna_phases/tests.rs: settables 108 -> 109, tiers (24, 47, 37) -> (24, 47, 38), },\n count 107 -> 108, relocrebase added to the no-live_field allowlist.
• catalog_bytecompat.rs: 108 -> 109 (x3) + tests/fixtures/phase_catalog.json regenerated via the decomp_dbg openfile write capture.
• docs/options.md regenerated; kuna catalog --check green.
• docs/history.md DIV-79 row; docs/spec/01-program-prep.md (the rebasing contract, Anchors: chapter for kuna-analysis/src); docs/missing-ghidra-analyses.md (the pass contract's address-space clause). make check-spec green in strict mode too.

Every count was read off a green build, never derived by arithmetic.

Gates

All four run in the worktree at 9dc29292, on top of origin/main 40c829e9. Verbatim tails:

$ make test
datatests: 675/675 assertions passed
exit: 0

=== baseline parity ===
PARITY OK

$ make test-stages
datatests: 481/481 assertions passed
exit: 0

=== baseline parity ===
PARITY OK

$ make check-spec
python3 tools/check_spec.py
check-spec OK (lenient mode)

(python3 tools/check_spec.py --strict → check-spec OK (strict mode).)

$ make rust-test
   Doc-tests kuna_wasm

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

313 test result: ok lines, zero FAILED; options_md_matches_the_generator_byte_for_byte ... ok; kuna catalog --check → catalog OK: documents exactly the registered kuna options.

Follow-up (not bundled)

Now that a relocatable object's analysis facts are rebased, #286's Listing/xref decline is worth revisiting: listing_seeds would be correct against the rebased view, and the Listing already decodes through the (rebased) ObjectLoadImage, so the recursive-descent walk, aif and noreturn_propagate could run on a .o instead of being skipped. That is a behavior change of its own with its own measurement, so it is deliberately left out of this PR.

---

🤖 Generated with Claude Code

https://claude.ai/code/session_01VBn8vgwoHqPFnnV4ZMApfq