A stripped bzip2 build on Linux: what REA returned, checked against the build and the docs

[CassieFox23]

TL;DR: I asked REA which function prints one message in a stripped, optimized bzip2 build. REA's function, one caller and 24 callees all matched an unstripped build and the source. It also refused two bad requests with typed errors, and below I list what this does not show.

Here is what I saw on Linux. The steps are folded below so you can repeat them.

The test

The program is bzip2 1.0.8, under the bzip2 licence (a BSD-style licence). I built it with gcc -O2 and ran strip. The file has no function names, and the compiler has inlined some functions.

The question: which function prints "Can't guess original name for %s -- using %s", who calls it, and what does it call?

REA found the string at 0x413220. It traced the string to the function at 0x402ed0, listed one caller, and listed 24 callees. Each reply carried an Evidence id. A label on each id says "observed" (Ghidra saw it) or "derived" (REA worked it out from other records).

I then checked the answers against an unstripped build of the same source. REA never saw that build.

Item	REA said	Real build and source
Function	0x402ed0	uncompress at 0x402ed0
String	0x413220	Line 1383, inside uncompress
Callers	1, at 0x400600	main at 0x400600
Callees	24	The same 24 in objdump

Two refusals

• I asked for the function by its source name, uncompress. The stripped file has no such name. REA replied with the code invalid_request. The message said the name is unknown. A procedure_info call for main got the same code.
• I called a Mac-only tool, inspect_macho, on this Linux file. REA replied with the code capability_unavailable. The reason read: "Native macOS utilities require macOS."

Plain Ghidra

Plain Ghidra headless, run on the same file, printed its analyzer list and then "Import succeeded". It printed no function list, no string hit and no caller list.

What the docs say

Here is where REA's docs describe what this demo shows. I read them on today's main, f941968. The demo ran on 83a4445.

Evidence ids. From mcp-contracts.md, lines 11 to 13:

Evidence-producing tools return the complete canonical Evidence record in both
text and structured content. Read `structuredContent.normalized_result` for the
operation result and `structuredContent.evidence_id` for its identity.

Observed and derived. From tool-design.md, lines 57 to 59:

- Keep observed, derived, and inferred relationships distinct. Preserve
  unresolved edges and partial provider facts instead of requiring every
  provider to supply the same metadata.

The two refusals. From mcp-contracts.md, lines 24 to 26:

Tool execution failures return `isError: true` and the complete canonical
`{ "error": ... }` diagnostic as JSON in text `content`. Read that diagnostic
for the error code, target details, partial observations, and remediation.

Callee lists. From tool-design.md, lines 60 to 64:

- For caller/callee reads, `relationship_coverage` describes only the
  provider-observed direct relationship set. `complete` means that set was
  returned without a known read limit; it does not claim every possible runtime
  edge. Provider and execution caveats stay in `limitations` and do not imply
  incomplete relationship coverage. Missing coverage remains `unknown`.

So the 24 callees above are the direct calls Ghidra saw in the function. Here they matched objdump. The docs do not promise that for every program.

Not run

This demo does not show these things.

• It covers one program, one compiler setting and Linux x86-64 only.
• It is not a comparison. I tried no other Ghidra MCP server, so I make no claim that REA is better.
• It does not say how long a Ghidra script would take, because I wrote none.
• I did not check which of the two panic calls in uncompress matches which source string. The count of two matches.
• I did not test that the -O0 and -O2 builds behave the same.
• I did not verify the bzip2 tarball signature. I recorded its sha256 instead.
• The first call after opening the file took about 9 seconds, on a box shared with another job. That time includes import and auto-analysis, so treat it as rough.
• I did not look up an SPDX id for the bzip2 licence.

Two more things I saw

• I built bzip2 again with -O0 and compared one function, BZ2_bzCompress, in both builds. REA reported five changed parts, one unchanged and two unknown. It would not call them the same function, because the names were only addresses.
• REA wrote the evidence from the -O2 session to one file of 440,392 bytes. The file came out byte-identical on every repeat run. It holds the path of the binary, so the hash changes if the path changes (see the "Exact MCP calls" fold).

Build commands

I computed the tarball sha256 below on my box. The script checks it before it unpacks.

curl -fsSL -o bzip2-1.0.8.tar.gz https://sourceware.org/pub/bzip2/bzip2-1.0.8.tar.gz
echo "ab5a03176ee106d3f0fa90e381da478ddae405918153cca248e682cd0c4a2269  bzip2-1.0.8.tar.gz" | sha256sum -c -
tar -xzf bzip2-1.0.8.tar.gz
S=bzip2-1.0.8
FILES="$S/bzip2.c $S/blocksort.c $S/compress.c $S/crctable.c $S/decompress.c $S/huffman.c $S/randtable.c $S/bzlib.c"
gcc -O2 -D_FILE_OFFSET_BITS=64 -I$S $FILES -o bzip2-O2-unstripped
cp bzip2-O2-unstripped bzip2-O2
strip --strip-all bzip2-O2
gcc -O0 -D_FILE_OFFSET_BITS=64 -I$S $FILES -o bzip2-O0-unstripped
cp bzip2-O0-unstripped bzip2-O0
strip --strip-all bzip2-O0

My builds came out as follows. The -O2 file is an x86-64 ELF, dynamically linked and stripped.

d59241a21fd4deb5975c338b84ba6c0eeca0b80d67c36793556ab0688fc194d7  bzip2-O2
c6a6b4a77a109adf335aae7a89463092fe7822f7e85f65a4b552b3fe5657c166  bzip2-O0

The .text bytes are identical in the stripped and unstripped files (checked with objcopy -O binary --only-section=.text). So addresses from nm on the unstripped file apply to the stripped one.

Exact MCP calls and replies (trimmed)

A small Node script of mine (not included here) started node scripts/rea.mjs mcp over stdio and made these calls in order. The server reported rea 6.4.0 with 173 tools. Evidence ids are shortened.

Question and answer:

open_binary            {"path":"<absolute path>/bzip2-O2"}   ok
search_strings         {"pattern":"guess original name"}
  one hit at 0x413220          ev_54f4c642...2fe42738   observed (Ghidra)
trace_feature          {"query":"guess original name"}
  string used at 0x403100, inside the function at 0x402ed0
                               ev_a6833b81...ec0a99cd  derived (REA workflow)
procedure_info         {"procedure":"0x402ed0"}
  98 basic blocks, length 2008 ev_598720bf...acd3dcc6  observed (Ghidra)
procedure_callers      {"procedure":"0x402ed0"}
  ["0x400600"], complete       ev_8972bf86...00845dbc    observed (Ghidra)
procedure_callees      {"procedure":"0x402ed0"}
  24 callees, complete         ev_b4230a5d...dd04a7d0    observed (Ghidra)
procedure_pseudo_code  {"procedure":"0x402ed0"}
  decompiler text, has the fprintf call to stderr
                               ev_6edc76cc...b0257901    observed (Ghidra)

Of the 24 callees, 5 are inside the program. With the unstripped build, I named them as follows.

0x4014c0 panic
0x401500 ioError
0x4015f0 copyFileName
0x401670 pad.constprop.0
0x402310 uncompressStream

The other 19 are import-table names that Ghidra gave itself, such as fprintf, fopen64, strerror and utime.

The body of the function at 0x402ed0 has 19 separate ranges. The helpers mapSuffix, containsDubiousChars and fileExists have no function of their own in the unstripped -O2 build. The compiler inlined them.

Each reply also carries limitations. For callers and callees, the coverage note says Ghidra returns only calls it resolved. For procedure_info, two notes say that Ghidra marks two calls to 0x4014c0 (at 0x403630 and 0x4036f6) as no-return calls.

Refusals, with the reply fields trimmed:

{"error":{"code":"invalid_request","category":"invalid_input","retryable":true,
 "details":{"operation":"analyze_function","issues":[{"path":[],"reason":"invalid_value",
 "message":"Unknown Ghidra procedure name or address: uncompress"}]}}}

{"error":{"code":"capability_unavailable","category":"unsupported_provider","retryable":false,
 "details":{"provider_id":"ghidra","operation":"inspect_macho",
 "reason":"Native macOS utilities require macOS."}}}

Three other calls showed limits.

• xrefs with no arguments gave invalid_request, with the issue {"path":["address"],"reason":"missing_argument","expected":"string"}.
• A search_strings call for text that is not in the file gave 0 matches. The note said an empty search does not show that the text is absent.
• open_binary with a relative path failed input validation, and the message said the path must be absolute.

Other calls:

• trace_call_path {"start":"0x400600","goal":"0x402310"} gave goal_status: reached with traversal_path ["0x400600","0x402ed0","0x402310"]. That is main, uncompress, uncompressStream by nm. It covered 55 nodes and 101 edges, with no failures and no unresolved items.
• export_evidence_bundle wrote 440,392 bytes. The file holds the path of the binary, so its sha256 depends on that path. Two runs at the first path gave the first hash below. Two reruns at a second path gave the second hash.

78e2a077f977de455082385acb9da271609100bde5150befdf7fca247ce74e01
4c0ab0fd0c43f51b268e460ec9247487c3f78142d671aa9f9497c5aad80577bf

The -O2 versus -O0 compare output

I ran analyze_function on BZ2_bzCompress in each build (0x40e460 in -O2, 0x414031 in -O0, both from nm). Then I passed both full replies to compare_functions.

REA knew them only by address. The result was function_match.status: mismatched, confidence derived, authority analyst-inference, overall status unknown. The summary was 1 unchanged, 5 changed and 2 unknown. The counts are the numbers REA printed. I did not check their units.

dimension      status     -O2   -O0
identity       unknown
pseudocode     changed    2002  2335
assembly       changed    3122  4912
comments       unchanged  0     0
calls          unknown    4     5
references     changed    45    95
strings_names  changed    34    48
cfg            changed    38    46

REA gave these limits:

identity    Address-derived function names are not stable cross-version identity.
assembly    Assembly is opaque provider text; relocation normalization is unavailable.
calls       Address-derived callee or caller names cannot be matched safely.
references  Reference endpoints differ independently of provider reference-kind metadata.

Then find_changed_behavior took that comparison. It gave behavior_status: unknown, with 0 observed changes, 5 static candidates, 0 contradictions and 2 unresolved.

Commands to repeat it

1. Build the two bzip2 binaries with the commands in the "Build commands" fold.

2. Put Node 24 first on your PATH (I used 24.18.0). Build REA at the commit below. I first built from a git archive of that commit. I then ran this clone and checkout as written. It gave the same commit, and the tracked files matched the git archive files.

   git clone https://github.com/morluto/rea.git
   git -C rea checkout 83a4445f440fab978ad9376b4260acae3f5fb189
   npm ci --prefix rea
   npm run build --prefix rea
   npm run test:prepare --prefix rea

3. Set these environment variables for the REA process. My first run also set REA_PWNTOOLS_PYTHON to a Python with pwntools 4.15.0. My rerun without it gave the same results, so this demo does not need it.

   export GHIDRA_INSTALL_DIR=/path/to/ghidra_12.1.4_PUBLIC
   export JAVA_HOME=/path/to/jdk-21
   export REA_ANALYSIS_PROVIDER=ghidra
   export REA_GHIDRA_STARTUP_TIMEOUT_MS=600000

4. Ask the main question with the REA command line. This is the same call as trace_feature. It gave the same Evidence id (ev_a6833b81...ec0a99cd) as the MCP call, even from a new build at a new path.

   node rea/scripts/rea.mjs trace /absolute/path/to/bzip2-O2 "guess original name" --provider ghidra --json

5. For the rest, connect any MCP client to node rea/scripts/rea.mjs mcp and make the calls listed in the "Exact MCP calls" fold.

6. Check the answers with the unstripped build:

   nm -n bzip2-O2-unstripped | grep -E ' (uncompress|main|uncompressStream)$'
   objdump -d --no-show-raw-insn --disassemble=uncompress bzip2-O2-unstripped
   grep -n "guess original name" bzip2-1.0.8/bzip2.c

7. Run plain Ghidra on the same file. Make an empty project folder with no dot-folder in its path. Ghidra does not make it for you. One try failed with "Path element starting with '.' is not permitted" because the folder was under .cache. Another failed with "Directory not found" because the folder did not exist.

   mkdir /tmp/empty-folder
   /path/to/ghidra_12.1.4_PUBLIC/support/analyzeHeadless /tmp/empty-folder plain -import /absolute/path/to/bzip2-O2 -readOnly -deleteProject

I ran the whole demo twice at first. Then I rebuilt everything in a new folder and ran it twice more, with a copy of my script that pointed at the new folder. Every Evidence id in the saved replies matched the first runs and matched between the two new runs. The ids do not depend on the path. The console log matched the first run line for line, apart from call times and the bundle hash.

Test setup

• Box: Nobara 44 (Fedora 44 base), Linux 7.2.9
• CPU: AMD Ryzen 9 5900X, 12 cores / 24 threads
• RAM: 32 GB DDR4-3600
• Node: 24.18.0
• Head: 83a4445 (REA main, package 6.4.0)
• Other: Ghidra 12.1.4 (Apache-2.0), OpenJDK 21.0.12.1, gcc 16.2.1, bzip2 1.0.8 (bzip2 licence). The box ran another test job at the same time, so timings are rough.

🤖 Generated with Claude Code (Sonnet 5.5, verified by Opus 5.5) 🦊