Skip to content

Publish the paper benchmark reproducibility contract - #5

Merged
JinPLu merged 1 commit into
mainfrom
agent/issue-4-reproducibility
Jul 11, 2026
Merged

Publish the paper benchmark reproducibility contract#5
JinPLu merged 1 commit into
mainfrom
agent/issue-4-reproducibility

Conversation

@JinPLu

@JinPLu JinPLu commented Jul 11, 2026

Copy link
Copy Markdown
Owner

Summary

Publishes the immutable paper_main_20260608 reproducibility contract for the frozen 23-model / 9,600-output WRBench paper table.

  • adds the exact 400-row local generation catalog and the distinct 400-row API source catalog;
  • adds the audited 100-row Wan2.7 static-camera TV2V conditioning-source manifest and resumable preparation flow;
  • freezes the local dual-angle/static/API camera scopes and the HyDRA 77+77 evaluation policy;
  • makes source-video routing task-specific and records source provenance in runner output;
  • corrects public documentation and model input taxonomy while preserving the rolling 11,100-video surface.

The reported prompt discrepancy was a public provenance-labeling defect. Frozen scoring rows remain aligned by variant_id and world_state_prompt; this change does not modify benchmark scores or generated video bytes.

Protected boundaries

  • scores_changed=false
  • video_bytes_changed=false
  • no MP4, first-frame, annotation, or active compatibility prompt replacement
  • frozen HyDRA CamPrec is retained and explicitly labeled as legacy full-concatenation pose provenance; the maintained CamAlign path is generated-only

Verification

  • 663 tests passed
  • bash scripts/oss_verify.sh: passed in an isolated install
  • final wheel 0.1.1 installed and validated with all release package data
  • task/hash/catalog/camera mutation tests reject mismatched provenance
  • six source-video wrappers passed runner dry-runs against a fixed audited asset
  • independent OSS, paper-alignment, and HF-staging reviews: ACCEPT

Addresses #4. The issue remains open until the cross-surface GitHub/Hugging Face release and public reproduction path are verified.

@JinPLu
JinPLu marked this pull request as ready for review July 11, 2026 19:03
@JinPLu
JinPLu merged commit 6c47fc6 into main Jul 11, 2026
3 checks passed
@JinPLu
JinPLu deleted the agent/issue-4-reproducibility branch July 12, 2026 15:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant