Skip to content

[E1] tether verify never runs the export artifact; both arms trace to native weights **[flagship gate hole]** #267

Description

@DsThakurRawat

Labels: bug, parity, verification, priority:high
File: src/tether/verify.py:395 (gather_paired_samples), src/tether/runtime/fast_inference/libero_adapter.py:53-89 (from_policy rebuilds from native float32), TODO admission at verify.py:387

Description. The "optimized" arm is built via TritonLIBEROAdapter.from_policy(policy) where policy is the native PyTorch checkpoint, not the export under test. The export dir is only read for the expert_takes_state flag; its .onnx/.trt/.data files are never loaded.

Why it matters. A customer exports a numerically-broken ONNX (bad opset, dropped Cast, weight-fusion bug), runs tether verify ./export, and both arms share the same native weights → parity is trivially high → PASS. The exact regression verify exists to catch is invisible. This directly undermines the product's "deployment confidence" pitch.

Tasks.

  • Load the actual export (dispatch on tether_config.json["export_kind"] → ORT / decomposed / Triton-from-engine) for the optimized arm.
  • Fail loudly if the export artifact can't be loaded, instead of silently falling back to from_policy.

Dedup. #104 is the round-trip "mirror bug" blindspot; #30/#31 add cosine/pi0.5 to round-trip validation; #198 (PR) is the action-parity metric engine. None make verify load the exported graph. Distinct.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions