Skip to content

llvm-inprocess diff has deterministic object-byte mismatch after behavior parity #8052

Description

@proggeramlug

Failure

The llvm-inprocess scheduled job now reaches the real end-to-end comparison and proves behavior parity for the spike, but native in-process emission is not byte-identical to the text-path object:

spike text exec/native behavior parity okay
PERRY_LLVM_INPROCESS=diff:
perry: [ir-diff] MISMATCH — object bytes differ (text 164063 vs native 180230)

Exact evidence: run 31737338464 / native-backend job 94574950972, head 3381e1b89048592cfc91f3d4a43349c6e7c16015.

This is independent of the earlier vacuous changes job and missing-diagnostic defects in #7971: the real macOS job ran, the source compiled through both paths, and the compiler's own diff verdict is visible. Behavior parity is useful evidence, but it does not satisfy the gate's object-emission equivalence oracle.

Required investigation

  • Persist and compare the two object files/sections from one =diff invocation.
  • Determine whether the delta is deterministic metadata/debug/object-writer output or a semantic code/data/relocation difference.
  • If exact whole-file bytes are intentionally not an invariant, replace that oracle only with a section-/relocation-aware invariant that is demonstrated to catch a real native-reader divergence; do not simply ignore the mismatch.

Acceptance

  • The current spike and batch-kernel end-to-end arms produce an explained, non-vacuous equivalence verdict.
  • Any accepted nonsemantic writer difference has a narrowly scoped, tested normalization; semantic sections and relocations remain covered.
  • A deliberately changed native lowering makes the comparison fail.

Refs #7971 and #7966.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions