Skip to content

bundle: the head magic must survive the optimizer (#882 follow-up) - #903

Merged
InauguralPhysicist merged 1 commit into
mainfrom
fix/882-macos-false-positive
Aug 6, 2026
Merged

bundle: the head magic must survive the optimizer (#882 follow-up)#903
InauguralPhysicist merged 1 commit into
mainfrom
fix/882-macos-false-positive

Conversation

@InauguralPhysicist

Copy link
Copy Markdown
Collaborator

#882 stores the archive head magic XOR-obfuscated so its plaintext never lands in the runtime image — otherwise every plain eigenscript start finds the constant inside itself and refuses to run.

That obfuscation is only as good as the compiler's willingness to leave it alone. gcc -O2 did not fold it. clang did — it constant-folded the XOR loop and materialized the plaintext in rodata. So on macOS the interpreter detected itself as a damaged bundle and exited 3:

FAIL: CLI09 REPL banner (rc=3 out='bundle: archive header found but the trailer is
      missing or unreadable — the file is truncated or damaged — refusing to run')
FAIL: 16 REPL check(s) failed
FAIL: plain interpreter still starts the REPL (rc=3 …)

Exactly the failure mode the obfuscation exists to prevent, on the platform I can't build for locally. CI's macos-latest job caught it.

Changes

  1. The XOR key is volatile — a volatile read cannot be folded away, so the plaintext cannot be precomputed into rodata by any optimizer.
  2. A scan hit is only believed when a well-formed entry header follows it[u32 path_len][printable path][u64 size] that fits inside the file. A coincidental magic match anywhere in future rodata therefore cannot make the interpreter refuse itself, independent of the obfuscation holding.

And a gate that doesn't depend on trusting either of those

tests/test_bundle.sh now greps the built interpreter for the plaintext magic and fails if it's present:

PASS: head magic plaintext is NOT in the interpreter image (fold guard)

That catches the fold at test time on any toolchain — present or future — instead of waiting for a platform job to go red. Validated both directions: it passes on the real binary and fires on a file that does contain the magic.

The lesson worth keeping: I verified the no-false-positive property on the compiler I had, and shipped a property that was compiler-dependent. The assertion is now on the outcome (is the string in the binary?) rather than on the mechanism.

  • Suite: 3805/3805
  • Bundle section: 13 → 14 checks

🤖 Generated with Claude Code

The head magic is stored XOR-obfuscated so its plaintext never lands in
the runtime image — otherwise every plain `eigenscript` start finds the
constant inside itself and refuses to run. That obfuscation is only as
good as the compiler's willingness to leave it alone.

gcc -O2 did not fold it. clang did: it constant-folded the XOR loop and
materialized the plaintext in rodata, so on macOS the interpreter
detected itself as a damaged bundle and exited 3. CI's macos-latest job
caught it — 16 REPL checks, 6 CLI checks and the bundle suite's own
"plain interpreter still starts the REPL" all red. Exactly the failure
mode the obfuscation exists to prevent, on the platform I cannot build
for locally.

Two changes:

  - the XOR key is `volatile`, so the loop cannot be folded away;
  - a scan hit is only believed when a well-formed entry header follows
    it ([u32 path_len][printable path][u64 size] that fits the file), so
    a coincidental magic match anywhere in future rodata cannot make the
    interpreter refuse itself.

And a gate that does not depend on trusting any of that:
tests/test_bundle.sh now greps the BUILT interpreter for the plaintext
magic and fails if it is there. That catches the fold at test time on
any toolchain, present or future, rather than waiting for a platform job
to go red. Validated both ways — the check passes on the real binary and
fires on a file containing the magic.

Suite 3805/3805; bundle section 13 -> 14 checks.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings August 6, 2026 00:33

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens --bundle self-exec detection against compiler optimizations that could accidentally materialize the bundle head magic plaintext in the interpreter image (notably under clang), and adds a test gate to detect such folds at build/test time.

Changes:

  • Makes the bundle head-magic XOR key volatile to prevent constant-folding into rodata.
  • Tightens head-magic scan validation by requiring a plausible archive entry header after a match (to reduce false positives).
  • Adds a test that scans the built interpreter image to ensure the plaintext head magic does not appear, and updates suite/changelog messaging accordingly.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
src/bundle.c Prevents optimizer folding of head-magic deobfuscation; adds plausibility validation after scan hits.
tests/test_bundle.sh Adds a regression gate that inspects the built interpreter for the plaintext head magic.
tests/run_all_tests.sh Updates the displayed bundle check count.
CHANGELOG.md Documents the new fold-guard and strengthened scan validation behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/bundle.c
Comment on lines +130 to +133
uint64_t size;
if (fread(&size, 8, 1, f) != 1) return 0;
if (size > (uint64_t)fsz) return 0;
return 1;
Comment thread tests/test_bundle.sh
Comment on lines +167 to +178
if command -v python3 >/dev/null 2>&1; then
if python3 -c "
import sys
plain = b'\\x7fEIGS-BUNDLE-ARCHIVE-v2\\x00'
sys.exit(0 if plain not in open('$EIGS','rb').read() else 1)
"; then
ok "head magic plaintext is NOT in the interpreter image (fold guard)"
else
fail "head magic plaintext is NOT in the interpreter image (fold guard)" \
"the obfuscation was constant-folded — every plain start will refuse itself"
fi
fi
@InauguralPhysicist
InauguralPhysicist merged commit 1f5d5ae into main Aug 6, 2026
19 checks passed
@InauguralPhysicist
InauguralPhysicist deleted the fix/882-macos-false-positive branch August 6, 2026 02:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants