Skip to content

[AUTOMATED] fix(p6): insert snip-reads trim COPYs after the op causing an INDIRECT effect (GH-181) - #307

Merged
mahaloz merged 1 commit into
mainfrom
fix/i181-snipreads-indirect
Aug 16, 2026
Merged

[AUTOMATED] fix(p6): insert snip-reads trim COPYs after the op causing an INDIRECT effect (GH-181)#307
mahaloz merged 1 commit into
mainfrom
fix/i181-snipreads-indirect

Conversation

@mahaloz

@mahaloz mahaloz commented Aug 16, 2026

Copy link
Copy Markdown
Contributor

Fixes #181

Root cause

Two stacked defects put the copy of a call's out-parameter textually above the call that fills it, so freecon(v1) consumed the pre-call NULL:

  1. Missing upstream carve-out in Merge::snipReads's insertion point (substrate/funcdata.rs (do_snip_reads_insert_point)). Upstream merge.cc:461-464 ("snip must come after OP CAUSING EFFECT") inserts the cover-trim COPY of an INDIRECT-defined Varnode after the op iop-encoded in the INDIRECT's input 1 (the call), never at the INDIRECT marker itself, which sits before the call in block order. kuna's port unconditionally returned the defining op, so the COPY feeding freecon landed before the lgetfilecon_raw CALL in the p-code. Ported faithfully via the existing op_iop_decode (the PcodeOp::getOpFromConst equivalent); the isInput() arm already matched upstream.
  2. A reordering hazard in kuna's own foldcallret (default-on, DIV-14, p6_variables/kuna_callretfold.rs). With the p-code fixed, the fold's order-safety predicate still sank the call expression into the if condition across the trim COPY that reads the call's own INDIRECT output, re-hoisting the copy at the text level. An op strictly between the call and its use that reads a value the call indirectly writes (an input defined by a CPUI_INDIRECT whose iop input names the call) is now a fold barrier. Marker ops are skipped: a later call's own INDIRECTs chain the earlier call's versions as inputs but have no textual evaluation point (without the skip, nearly every back-to-back call pair stops folding).

Both are strict correctness fixes: no new option, no DIV row.

Audit of the other upstream carve-out sites

  • trimOpOutput (merge.cc:663) — kuna do_trim_op_output already had the INDIRECT arm.
  • eliminateIntersect (merge.cc:554) / snipOutputInterference (merge.cc:814) — already ported via indirect_effect_op.
  • mergeIndirect's snip (merge.cc:877) inserts before the INDIRECT deliberately (addr-forced convention) — kuna matches.
  • trimOpInput inserts before the reading op / end of pred block — kuna matches.

Witness (tests/bug-repro/libselinux.so.1 @ 0x17370)

Before:

    v4 = NULL;
    v1 = v4;                                              // copy BEFORE the call
    if ((0 <= (int)lgetfilecon_raw(a0,&v4)) && (!strcmp(a1,v4)))
    ...
    freecon(v1);                                          // frees the pre-call NULL

After (the Ghidra/Binja shape; re-verified on current main, where #306's stackguard entry-store removal also drops the canary lines from this function):

    v5 = NULL;
    v3 = lgetfilecon_raw(a0,&v5);
    v1 = v5;                                              // copy AFTER the call
    if ((0 <= v3) && (v3 = strcmp(a1,v5), !v3))
    ...
    freecon(v1);                                          // frees the post-call value

The issue's second witness (fsetfilecon_raw @ 0xf650 in the non-vendored -O2 -fno-inline build) was not re-verified as such, but the same-idiom fsetfilecon_raw @ 0xf670 in the vendored O2 build showed the identical hoisted shape and is fixed the same way, as is setfilecon_raw @ 0x1f680.

Whole-binary collateral audit

decompile-all --json base vs fixed on tests/bug-repro/{grep,sort,faillog,libselinux.so.1}: 110 changed functions of 1629 (grep 21/449, sort 33/490, faillog 6/69, libselinux 50/621). Every diff was reviewed (normalized vN -> v# to separate renumbering); each is one of:

  • Trim COPY materialized at its correct place: new v = global; / v = slot; statements after the effect call where the base printed the copy hoisted, or printed degenerate self-assignments — the bogus dat_2b684 = dat_2b684; / dat_1e368 = dat_1e368; artifacts (the wrongly-placed COPY rendering as x = x) are gone from grep/sort/cvs outputs.
  • Fold refusals where a statement between a call and its use reads the call's INDIRECT effect: the call gets the explicit Ghidra-style v = call(...); spill (e.g. the many error(...,dcgettext(...)) sites, return fcntl(...) -> v = fcntl(...); return v;). Semantically strictly safer; slightly more verbose.
  • Downstream fallout of the above: local renumbering, a fold newly enabled where the copy no longer sits between, condition-collapse changes where a correctly-placed copy now occupies a branch arm.

No unexplained diffs; no lost statements or changed values found.

Stage tests

Gates

  • make testPARITY OK (675/675, no upstream datatest changed)
  • make test-stagesPARITY OK (473/473 after re-record on the rebased tree)
  • make rust-test — green
  • make check-spec — green (docs/spec/06-variables-and-merge.md gains prose for the snip-reads insertion rule and the fold barrier)

Speed

Interleaved min-of-11, kuna decompile-all tests/bug-repro/grep --json > /dev/null: base min 14.330s vs fixed min 14.325s = -0.04% (mins over 22 interleaved pairs; an initial round under heavy sibling-agent load read +4.5% and was discarded as contention -- the quieter second round's paired mins agree at noise level).

🤖 Generated with Claude Code

https://claude.ai/code/session_01VBn8vgwoHqPFnnV4ZMApfq

…T effect (GH-181)

[AUTOMATED] Two stacked correctness fixes, no new option:

1. Port the upstream Merge::snipReads carve-out (merge.cc:461, "snip must
   come after OP CAUSING EFFECT") that kuna's do_snip_reads_insert_point
   dropped: a cover-trim COPY of an INDIRECT-defined Varnode is inserted
   after the op iop-encoded in the INDIRECT's input 1 (the call), not at
   the INDIRECT marker, which precedes the call in block order.  Without
   it, libselinux lsetfilecon_raw's out-parameter copy printed above the
   lgetfilecon_raw call that fills the slot, so freecon() consumed the
   pre-call NULL.  Audited the other upstream sites: trimOpOutput already
   had the arm; eliminateIntersect/snipOutputInterference already used
   indirect_effect_op; mergeIndirect/trimOpInput match upstream.

2. Close the matching reordering hazard in kuna's own foldcallret
   (default-on, DIV-14): an op strictly between a call and its lone use
   that reads a value the call indirectly writes (an input defined by a
   CPUI_INDIRECT whose iop input names the call) is now a fold barrier,
   so the call expression is not sunk past the very copy fix 1 places
   after it.  Marker ops are skipped (a later call's INDIRECTs chain the
   earlier call's versions without a textual evaluation point).

New two-pass stage test gh181-snipreads-indirect.xml (default proves both
fixes; option foldcallret off isolates fix 1) plus a README row; corpus
count recounted to 206 on the rebased tree; baseline-stages re-recorded
(473/473 on post-#306 main, including #304's eight GH-180 assertions;
this branch's stage patterns are number-agnostic, so #306's canary
renumbering required no re-pinning).  Three stage tests pinned
renderings the fixes legitimately changed and were updated with comments:
ghangr-x8664-cvs-863633 (slot 0x58 now merge-coalesced even with
dedupvardecls off), ghangr-morton-my-message-callback (renumbering; the
bogus self-assign artifact is gone), ghdec-symbol-keyed-local-decls (the
checksum call is now sequenced before the CONCAT13 reading the slot it
may write).  Spec prose added to docs/spec/06-variables-and-merge.md.

Gates: make test PARITY OK (675/675), make test-stages PARITY OK,
make rust-test green, make check-spec green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VBn8vgwoHqPFnnV4ZMApfq
@mahaloz
mahaloz force-pushed the fix/i181-snipreads-indirect branch from b23fdf0 to 8595030 Compare August 16, 2026 20:19
@mahaloz
mahaloz merged commit 95ea590 into main Aug 16, 2026
9 checks passed
@mahaloz
mahaloz deleted the fix/i181-snipreads-indirect branch August 16, 2026 20:21
mahaloz added a commit that referenced this pull request Aug 17, 2026
The branch was cut from #302 while main advanced to #316; the engine
PRs in between (spillargtrial #315, stackguard #306, snip-reads #307,
loadguardrange #308, ...) legitimately shift the ghidra-path output, so
CI's merge-ref run measured register leaks 108 vs the pinned 106. Rebase
onto origin/main and re-pin every value to the rebased tree:

  registers 106/64/60 -> 108/58/60, unique 32/2/8 -> 34/4/8,
  mangled 21/13/7 -> 21/6/7, c_lines 241/174/89 -> 243/128/89,
  diff ratios 0.643/0.898/0.811 -> 0.646/0.867/0.811 (band unchanged);
  placeholders/resolvable/traffic unchanged (49/25/17, 24/18/14,
  1477/1003).

The sub_3320 shrink (c_lines -26%, FS_OFFSET gone from its register
set, mangled 13 -> 6) is the stackguard/snip-reads work stripping the
canary sequence -- explainable, not suspicious. Determinism verified:
three consecutive runs of the pins test are measurement-identical.

Gates on the rebased tree: harness release + --include-ignored green,
make test 675/675 PARITY OK, make test-stages PARITY OK, check-spec OK,
kuna-ghidra dev-profile spot check green. The full workspace suite ran
green on this branch pre-rebase and every added commit is already-CI-
green main history, so it is covered by the branch + main CI runs.

[AUTOMATED]

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AfGYKwUWvPYYj1Aw7tcLhC
mahaloz added a commit that referenced this pull request Aug 17, 2026
…the GUI-path quality gap (#317)

* feat(ghidra): ghidra-sim differential test harness — pin the GUI-path quality gap

Extract the decompile_at_e2e MockJava loopback into a shared module
(tests/ghidra_sim/mod.rs — AnswerSource-pluggable pump, wire builders,
session tracer, dual-<function> doc parser, a markup->C flattener that
replicates Java's getC() token cleaning (IllegalCharCppTransformer),
badness scanners, line-diff metric) and build ghidra-sim v1 on top
(tests/ghidra_sim/oracle.rs): a mock-Java answer source backed by kuna's
own analysis of real vendored ELFs — bootstrap_from_object for
bytes/labels, the real Sleigh re-encoded as wire <inst> docs for
getPcode, and a tspec GENERATED from the loaded Sleigh's
AddrSpaceManager so packed space indices agree by construction.
getMappedSymbols/getExternalRef answer EMPTY at a marked PHASE-3 SEAM.

tests/ghidra_sim_e2e.rs drives the full wire lifecycle (registerProgram
-> setAction -> decompileAt x3 -> flushNative -> repeat -> deregister)
over tests/bug-repro/faillog (sort/grep as an ignored breadth test),
asserts the response-document schema (name/entry echo, markup
opref/varref subset-of ast, 19-query legality + query-legal placement),
and PINS today's Phase-2 reality: per-function raw-register leaks
(106/64/60), Unique tokens (32/2/8), placeholders (49/25/17) of which
the loader already knows names for 24/18/14 (Phase 3 drives to 0),
getC()-mangled tokens (21/13/7, PR-C drives to 0), ghidra-vs-CLI line
diff ratios (0.64/0.90/0.81 floors), getPcode traffic 1477/1003, and
getMappedSymbols == 0 (Phase 3 flips to >=1). Pins move only with the
provider/emitter change that earns them.

[AUTOMATED]

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AfGYKwUWvPYYj1Aw7tcLhC

* ci(ghidra): run the ghidra-sim harness in the gates job + make test-ghidra

Add a `gates`-job step running `cargo test -p kuna-ghidra --release --
--include-ignored` (with the standard specs-skip canary grep) after the
catalog checks: the workspace suite is skipped on internal PRs, and a
GUI-path regression is exactly what the ghidra-sim pins exist to catch
pre-merge. Cost: ~1-2 min of compile over the release deps `make
binaries` already built, ~2 s of test runtime. `make test-ghidra` is the
same run locally.

[AUTOMATED]

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AfGYKwUWvPYYj1Aw7tcLhC

* docs(ghidra): live-smoke rig + testing-strategy section for the harness

integrations/ghidra/live-smoke/: a manual/dev pyghidra rig that swaps
DecompileProcessFactory.exepath to kuna_ghidra inside a real Ghidra,
decompiles the same functions with both cores, and writes a side-by-side
report with the same badness-scanner counts the in-tree harness pins
(README covers the offline-pyghidra setup and the getC()-vs-GUI-panel
rendering distinction). docs/ghidra-integration.md §11 rewritten around
the shipped harness: what it covers, how to run it, where the pins live.

[AUTOMATED]

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AfGYKwUWvPYYj1Aw7tcLhC

* test(ghidra): harden the ghidra-sim pins per adversarial review

- Diff-ratio pins are now a BAND: ceilings [0.70,0.95,0.90] join the
  floors (a markup regression that makes the GUI text worse now fails
  instead of saturating), the vacuous `<= 1.0` assert is gone, and the
  flattened-C normalized line count is pinned per target (241/174/89) --
  the assert that actually catches a <break>-token collapse. The pin
  comment now decomposes the ratio (Phase-3 symbol gap + option-preset
  skew until setOptions is wired) so nobody chases 0 with symbol work.
- decompile_cli now routes through the SHARED per-function step
  (kuna_console::decompile_step::decompile_one, DIV-66) with the
  error-noreturn CALL_RETURN flow overrides built exactly as
  kuna_console::project does -- the previous direct drive call silently
  dropped them (latent: faillog has no error() sites, and every faillog
  pin re-measured IDENTICAL; sort/grep-class fixtures would have
  diverged from the real CLI).
- Assert strength: warnings frames pinned trim-empty on registerProgram
  and decompileAt (no substring blocklists); markup oprefs AND varrefs
  pinned non-empty per class; ast varnode refs collected from the
  <varnodes> child only (all Java's buildVarnodeRefs keys -- an op
  operand ref undeclared there must not launder the subset assert);
  the name echo compares against the sim's code_label (which consults
  label_overrides) rather than the raw program lookup.
- CI: the gates-job harness step tees its output to a log before the
  canary grep, so a FAILING pin still prints its diagnostics (the old
  command substitution aborted under bash -e before any echo).
- live-smoke docs/script: kuna_ghidra is not built by `make binaries`;
  point at `cargo build --release -p kuna-ghidra`.

Tests/CI/docs-only diff: the parity gates are untouched by construction.
`cargo test -p kuna-ghidra --release -- --include-ignored` fully green;
`make check-spec` green. No pinned value moved.

[AUTOMATED]

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AfGYKwUWvPYYj1Aw7tcLhC

* test(ghidra): re-measure the faillog pins on main @ 813ee13 (rebase)

The branch was cut from #302 while main advanced to #316; the engine
PRs in between (spillargtrial #315, stackguard #306, snip-reads #307,
loadguardrange #308, ...) legitimately shift the ghidra-path output, so
CI's merge-ref run measured register leaks 108 vs the pinned 106. Rebase
onto origin/main and re-pin every value to the rebased tree:

  registers 106/64/60 -> 108/58/60, unique 32/2/8 -> 34/4/8,
  mangled 21/13/7 -> 21/6/7, c_lines 241/174/89 -> 243/128/89,
  diff ratios 0.643/0.898/0.811 -> 0.646/0.867/0.811 (band unchanged);
  placeholders/resolvable/traffic unchanged (49/25/17, 24/18/14,
  1477/1003).

The sub_3320 shrink (c_lines -26%, FS_OFFSET gone from its register
set, mangled 13 -> 6) is the stackguard/snip-reads work stripping the
canary sequence -- explainable, not suspicious. Determinism verified:
three consecutive runs of the pins test are measurement-identical.

Gates on the rebased tree: harness release + --include-ignored green,
make test 675/675 PARITY OK, make test-stages PARITY OK, check-spec OK,
kuna-ghidra dev-profile spot check green. The full workspace suite ran
green on this branch pre-rebase and every added commit is already-CI-
green main history, so it is covered by the branch + main CI runs.

[AUTOMATED]

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AfGYKwUWvPYYj1Aw7tcLhC

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Copy of a call's out-parameter is hoisted above the call: freecon() gets the pre-call value

1 participant