[AUTOMATED] fix(p6): insert snip-reads trim COPYs after the op causing an INDIRECT effect (GH-181) - #307
Merged
Merged
Conversation
…T effect (GH-181) [AUTOMATED] Two stacked correctness fixes, no new option: 1. Port the upstream Merge::snipReads carve-out (merge.cc:461, "snip must come after OP CAUSING EFFECT") that kuna's do_snip_reads_insert_point dropped: a cover-trim COPY of an INDIRECT-defined Varnode is inserted after the op iop-encoded in the INDIRECT's input 1 (the call), not at the INDIRECT marker, which precedes the call in block order. Without it, libselinux lsetfilecon_raw's out-parameter copy printed above the lgetfilecon_raw call that fills the slot, so freecon() consumed the pre-call NULL. Audited the other upstream sites: trimOpOutput already had the arm; eliminateIntersect/snipOutputInterference already used indirect_effect_op; mergeIndirect/trimOpInput match upstream. 2. Close the matching reordering hazard in kuna's own foldcallret (default-on, DIV-14): an op strictly between a call and its lone use that reads a value the call indirectly writes (an input defined by a CPUI_INDIRECT whose iop input names the call) is now a fold barrier, so the call expression is not sunk past the very copy fix 1 places after it. Marker ops are skipped (a later call's INDIRECTs chain the earlier call's versions without a textual evaluation point). New two-pass stage test gh181-snipreads-indirect.xml (default proves both fixes; option foldcallret off isolates fix 1) plus a README row; corpus count recounted to 206 on the rebased tree; baseline-stages re-recorded (473/473 on post-#306 main, including #304's eight GH-180 assertions; this branch's stage patterns are number-agnostic, so #306's canary renumbering required no re-pinning). Three stage tests pinned renderings the fixes legitimately changed and were updated with comments: ghangr-x8664-cvs-863633 (slot 0x58 now merge-coalesced even with dedupvardecls off), ghangr-morton-my-message-callback (renumbering; the bogus self-assign artifact is gone), ghdec-symbol-keyed-local-decls (the checksum call is now sequenced before the CONCAT13 reading the slot it may write). Spec prose added to docs/spec/06-variables-and-merge.md. Gates: make test PARITY OK (675/675), make test-stages PARITY OK, make rust-test green, make check-spec green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VBn8vgwoHqPFnnV4ZMApfq
mahaloz
force-pushed
the
fix/i181-snipreads-indirect
branch
from
August 16, 2026 20:19
b23fdf0 to
8595030
Compare
mahaloz
added a commit
that referenced
this pull request
Aug 17, 2026
The branch was cut from #302 while main advanced to #316; the engine PRs in between (spillargtrial #315, stackguard #306, snip-reads #307, loadguardrange #308, ...) legitimately shift the ghidra-path output, so CI's merge-ref run measured register leaks 108 vs the pinned 106. Rebase onto origin/main and re-pin every value to the rebased tree: registers 106/64/60 -> 108/58/60, unique 32/2/8 -> 34/4/8, mangled 21/13/7 -> 21/6/7, c_lines 241/174/89 -> 243/128/89, diff ratios 0.643/0.898/0.811 -> 0.646/0.867/0.811 (band unchanged); placeholders/resolvable/traffic unchanged (49/25/17, 24/18/14, 1477/1003). The sub_3320 shrink (c_lines -26%, FS_OFFSET gone from its register set, mangled 13 -> 6) is the stackguard/snip-reads work stripping the canary sequence -- explainable, not suspicious. Determinism verified: three consecutive runs of the pins test are measurement-identical. Gates on the rebased tree: harness release + --include-ignored green, make test 675/675 PARITY OK, make test-stages PARITY OK, check-spec OK, kuna-ghidra dev-profile spot check green. The full workspace suite ran green on this branch pre-rebase and every added commit is already-CI- green main history, so it is covered by the branch + main CI runs. [AUTOMATED] Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AfGYKwUWvPYYj1Aw7tcLhC
mahaloz
added a commit
that referenced
this pull request
Aug 17, 2026
…the GUI-path quality gap (#317) * feat(ghidra): ghidra-sim differential test harness — pin the GUI-path quality gap Extract the decompile_at_e2e MockJava loopback into a shared module (tests/ghidra_sim/mod.rs — AnswerSource-pluggable pump, wire builders, session tracer, dual-<function> doc parser, a markup->C flattener that replicates Java's getC() token cleaning (IllegalCharCppTransformer), badness scanners, line-diff metric) and build ghidra-sim v1 on top (tests/ghidra_sim/oracle.rs): a mock-Java answer source backed by kuna's own analysis of real vendored ELFs — bootstrap_from_object for bytes/labels, the real Sleigh re-encoded as wire <inst> docs for getPcode, and a tspec GENERATED from the loaded Sleigh's AddrSpaceManager so packed space indices agree by construction. getMappedSymbols/getExternalRef answer EMPTY at a marked PHASE-3 SEAM. tests/ghidra_sim_e2e.rs drives the full wire lifecycle (registerProgram -> setAction -> decompileAt x3 -> flushNative -> repeat -> deregister) over tests/bug-repro/faillog (sort/grep as an ignored breadth test), asserts the response-document schema (name/entry echo, markup opref/varref subset-of ast, 19-query legality + query-legal placement), and PINS today's Phase-2 reality: per-function raw-register leaks (106/64/60), Unique tokens (32/2/8), placeholders (49/25/17) of which the loader already knows names for 24/18/14 (Phase 3 drives to 0), getC()-mangled tokens (21/13/7, PR-C drives to 0), ghidra-vs-CLI line diff ratios (0.64/0.90/0.81 floors), getPcode traffic 1477/1003, and getMappedSymbols == 0 (Phase 3 flips to >=1). Pins move only with the provider/emitter change that earns them. [AUTOMATED] Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AfGYKwUWvPYYj1Aw7tcLhC * ci(ghidra): run the ghidra-sim harness in the gates job + make test-ghidra Add a `gates`-job step running `cargo test -p kuna-ghidra --release -- --include-ignored` (with the standard specs-skip canary grep) after the catalog checks: the workspace suite is skipped on internal PRs, and a GUI-path regression is exactly what the ghidra-sim pins exist to catch pre-merge. Cost: ~1-2 min of compile over the release deps `make binaries` already built, ~2 s of test runtime. `make test-ghidra` is the same run locally. [AUTOMATED] Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AfGYKwUWvPYYj1Aw7tcLhC * docs(ghidra): live-smoke rig + testing-strategy section for the harness integrations/ghidra/live-smoke/: a manual/dev pyghidra rig that swaps DecompileProcessFactory.exepath to kuna_ghidra inside a real Ghidra, decompiles the same functions with both cores, and writes a side-by-side report with the same badness-scanner counts the in-tree harness pins (README covers the offline-pyghidra setup and the getC()-vs-GUI-panel rendering distinction). docs/ghidra-integration.md §11 rewritten around the shipped harness: what it covers, how to run it, where the pins live. [AUTOMATED] Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AfGYKwUWvPYYj1Aw7tcLhC * test(ghidra): harden the ghidra-sim pins per adversarial review - Diff-ratio pins are now a BAND: ceilings [0.70,0.95,0.90] join the floors (a markup regression that makes the GUI text worse now fails instead of saturating), the vacuous `<= 1.0` assert is gone, and the flattened-C normalized line count is pinned per target (241/174/89) -- the assert that actually catches a <break>-token collapse. The pin comment now decomposes the ratio (Phase-3 symbol gap + option-preset skew until setOptions is wired) so nobody chases 0 with symbol work. - decompile_cli now routes through the SHARED per-function step (kuna_console::decompile_step::decompile_one, DIV-66) with the error-noreturn CALL_RETURN flow overrides built exactly as kuna_console::project does -- the previous direct drive call silently dropped them (latent: faillog has no error() sites, and every faillog pin re-measured IDENTICAL; sort/grep-class fixtures would have diverged from the real CLI). - Assert strength: warnings frames pinned trim-empty on registerProgram and decompileAt (no substring blocklists); markup oprefs AND varrefs pinned non-empty per class; ast varnode refs collected from the <varnodes> child only (all Java's buildVarnodeRefs keys -- an op operand ref undeclared there must not launder the subset assert); the name echo compares against the sim's code_label (which consults label_overrides) rather than the raw program lookup. - CI: the gates-job harness step tees its output to a log before the canary grep, so a FAILING pin still prints its diagnostics (the old command substitution aborted under bash -e before any echo). - live-smoke docs/script: kuna_ghidra is not built by `make binaries`; point at `cargo build --release -p kuna-ghidra`. Tests/CI/docs-only diff: the parity gates are untouched by construction. `cargo test -p kuna-ghidra --release -- --include-ignored` fully green; `make check-spec` green. No pinned value moved. [AUTOMATED] Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AfGYKwUWvPYYj1Aw7tcLhC * test(ghidra): re-measure the faillog pins on main @ 813ee13 (rebase) The branch was cut from #302 while main advanced to #316; the engine PRs in between (spillargtrial #315, stackguard #306, snip-reads #307, loadguardrange #308, ...) legitimately shift the ghidra-path output, so CI's merge-ref run measured register leaks 108 vs the pinned 106. Rebase onto origin/main and re-pin every value to the rebased tree: registers 106/64/60 -> 108/58/60, unique 32/2/8 -> 34/4/8, mangled 21/13/7 -> 21/6/7, c_lines 241/174/89 -> 243/128/89, diff ratios 0.643/0.898/0.811 -> 0.646/0.867/0.811 (band unchanged); placeholders/resolvable/traffic unchanged (49/25/17, 24/18/14, 1477/1003). The sub_3320 shrink (c_lines -26%, FS_OFFSET gone from its register set, mangled 13 -> 6) is the stackguard/snip-reads work stripping the canary sequence -- explainable, not suspicious. Determinism verified: three consecutive runs of the pins test are measurement-identical. Gates on the rebased tree: harness release + --include-ignored green, make test 675/675 PARITY OK, make test-stages PARITY OK, check-spec OK, kuna-ghidra dev-profile spot check green. The full workspace suite ran green on this branch pre-rebase and every added commit is already-CI- green main history, so it is covered by the branch + main CI runs. [AUTOMATED] Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AfGYKwUWvPYYj1Aw7tcLhC --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #181
Root cause
Two stacked defects put the copy of a call's out-parameter textually above the call that fills it, so
freecon(v1)consumed the pre-callNULL:Merge::snipReads's insertion point (substrate/funcdata.rs (do_snip_reads_insert_point)). Upstreammerge.cc:461-464("snip must come after OP CAUSING EFFECT") inserts the cover-trim COPY of an INDIRECT-defined Varnode after the op iop-encoded in the INDIRECT's input 1 (the call), never at the INDIRECT marker itself, which sits before the call in block order. kuna's port unconditionally returned the defining op, so the COPY feedingfreeconlanded before thelgetfilecon_rawCALL in the p-code. Ported faithfully via the existingop_iop_decode(thePcodeOp::getOpFromConstequivalent); theisInput()arm already matched upstream.foldcallret(default-on, DIV-14,p6_variables/kuna_callretfold.rs). With the p-code fixed, the fold's order-safety predicate still sank the call expression into theifcondition across the trim COPY that reads the call's own INDIRECT output, re-hoisting the copy at the text level. An op strictly between the call and its use that reads a value the call indirectly writes (an input defined by aCPUI_INDIRECTwhose iop input names the call) is now a fold barrier. Marker ops are skipped: a later call's own INDIRECTs chain the earlier call's versions as inputs but have no textual evaluation point (without the skip, nearly every back-to-back call pair stops folding).Both are strict correctness fixes: no new option, no DIV row.
Audit of the other upstream carve-out sites
trimOpOutput(merge.cc:663) — kunado_trim_op_outputalready had the INDIRECT arm.eliminateIntersect(merge.cc:554) /snipOutputInterference(merge.cc:814) — already ported viaindirect_effect_op.mergeIndirect's snip (merge.cc:877) inserts before the INDIRECT deliberately (addr-forced convention) — kuna matches.trimOpInputinserts before the reading op / end of pred block — kuna matches.Witness (tests/bug-repro/libselinux.so.1 @ 0x17370)
Before:
After (the Ghidra/Binja shape; re-verified on current main, where #306's stackguard entry-store removal also drops the canary lines from this function):
The issue's second witness (
fsetfilecon_raw@ 0xf650 in the non-vendored-O2 -fno-inlinebuild) was not re-verified as such, but the same-idiomfsetfilecon_raw@ 0xf670 in the vendored O2 build showed the identical hoisted shape and is fixed the same way, as issetfilecon_raw@ 0x1f680.Whole-binary collateral audit
decompile-all --jsonbase vs fixed ontests/bug-repro/{grep,sort,faillog,libselinux.so.1}: 110 changed functions of 1629 (grep 21/449, sort 33/490, faillog 6/69, libselinux 50/621). Every diff was reviewed (normalizedvN -> v#to separate renumbering); each is one of:v = global;/v = slot;statements after the effect call where the base printed the copy hoisted, or printed degenerate self-assignments — the bogusdat_2b684 = dat_2b684;/dat_1e368 = dat_1e368;artifacts (the wrongly-placed COPY rendering asx = x) are gone from grep/sort/cvs outputs.v = call(...);spill (e.g. the manyerror(...,dcgettext(...))sites,return fcntl(...)->v = fcntl(...); return v;). Semantically strictly safer; slightly more verbose.No unexplained diffs; no lost statements or changed values found.
Stage tests
tests/stages/gh181-snipreads-indirect.xml(two passes: default proves both fixes;option foldcallret offisolates the p-code-level snipReads fix), with atests/stages/README.mdrow. Fails 2/3 assertions on the base engine, passes 3/3 fixed. Corpus count inkuna-base/src/xml.rsbumped to 206 (recounted on the rebased tree: 83 datatests + 123 stages, including [AUTOMATED] fix(p6): close GH-180 - overflow guards read the pre-add operands (verified fixed by DIV-47; regression pinned) #304's gh180 testcase);docs/baseline-stages.jsonre-recorded on the rebased tree (473/473 on top of [AUTOMATED] fix(p6): close GH-180 - overflow guards read the pre-add operands (verified fixed by DIV-47; regression pinned) #304/[AUTOMATED] feat(analysis): datasyms - import ELF OBJECT symbols so libc globals render by name (DIV-76) #305/[AUTOMATED] fix(p7): stackguard - remove the entry-side canary init (and its TLS residue) with the stripped check #306's main -- [AUTOMATED] fix(p6): close GH-180 - overflow guards read the pre-add operands (verified fixed by DIV-47; regression pinned) #304's eight Overflow guards read the post-add value:v3 = v3 + v1; if (SCARRY8(v3,v1))#180 assertions and [AUTOMATED] fix(p7): stackguard - remove the entry-side canary init (and its TLS residue) with the stripped check #306's re-pinned canary tests all pass together with this change; this branch's stage-XML patterns are variable-number-agnostic, so [AUTOMATED] fix(p7): stackguard - remove the entry-side canary init (and its TLS residue) with the stripped check #306's renumbering required no re-pinning here).ghangr-x8664-cvs-863633.xml— slot 0x58 is now merge-coalesced to one declaration even withdedupvardecls off(the duplicate-high source was the mis-placed snip COPY), mirroring the file's own precedent for slot 0x3c.ghangr-morton-my-message-callback-bfd2fa.xml— pure renumbering (free(v6)->free(v7); a new explicit trim-copy local); pattern loosened tofree\(v[0-9]+\). This test's passes run withfoldcallret off, so it also witnesses that the removed self-assign artifact came from the snipReads fix alone.ghdec-symbol-keyed-local-decls.xml— the checksum call is now explicitly sequenced before theCONCAT13that reads the slot it may write (the base formv1 = CONCAT13(slot_checksum(&v1,4),v1)had the read and the call unsequenced in one expression); the declaration-collapse behavior under test is unchanged.Gates
make test— PARITY OK (675/675, no upstream datatest changed)make test-stages— PARITY OK (473/473 after re-record on the rebased tree)make rust-test— greenmake check-spec— green (docs/spec/06-variables-and-merge.mdgains prose for the snip-reads insertion rule and the fold barrier)Speed
Interleaved min-of-11,
kuna decompile-all tests/bug-repro/grep --json > /dev/null: base min 14.330s vs fixed min 14.325s = -0.04% (mins over 22 interleaved pairs; an initial round under heavy sibling-agent load read +4.5% and was discarded as contention -- the quieter second round's paired mins agree at noise level).🤖 Generated with Claude Code
https://claude.ai/code/session_01VBn8vgwoHqPFnnV4ZMApfq