test(resolver): enforce variant-index parity in the rez differential#81
Merged
Conversation
The 188-case differential previously compared rer's resolution against rez's recorded one as a `(name, version)` set, ignoring the variant index — `normalize_rez` split rez's `"name/version/package.py[idx]"` strings on `/` and threw away the `[idx]` suffix. A future refactor could silently make rer pick a different variant of the same version (with the variant's own `requires` being different) and the gate wouldn't catch it. This is the gap the README itself called out under "Validated 1:1" — a real correctness commitment 1.0 should make. Strengthen the test: - `normalize_rez` now parses the suffix and returns `(name, version, Option<usize>)` triples. `package.py[]` (rez's no-variant form) maps to `None`; `package.py[N]` maps to `Some(N)`. - `solve()` likewise returns triples (now reading `v.index()` on every resolved variant). - The exact-match assertion compares triples — divergence on the variant index alone is now a test failure. - On divergence, the test also logs the *content* of the diff (`rer-only: [...]` vs `rez-only: [...]`) per case, so future failures point at the specific entries that drifted without re-running the suite to investigate. Result: 188 / 188 still pass under the strict check. The solver was already producing rez-faithful variant choices; we just weren't enforcing it. With this commit the test will catch any regression that changes which variant gets picked. Docs: - README's "Validated 1:1" status bullet now states that variant index is enforced (was: "currently does not enforce"). - `rez-integration.md` drops the "variant-index parity is rez-faithful by construction but not enforced" caveat — it's enforced now. Next steps toward 1.0 (Option A path): - Decide on `intersection_priority` (#63): implement or close as documented limitation. - Write CHANGELOG.md (rc.1 → 1.0). - Add a "What 1.0 promises" page. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Qodo reviews are paused for this user.Troubleshooting steps vary by plan Learn more → On a Teams plan? Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center? |
This was referenced May 15, 2026
doubleailes
added a commit
that referenced
this pull request
May 15, 2026
Last two items on the Option A 1.0 checklist: - A CHANGELOG.md following Keep a Changelog conventions, covering every rc release back to rc.1, plus a 1.0.0 entry capturing the variant-index parity (#81) and `intersection_priority` (#82) work. Includes compare-links pointing at the conventional tag names. - A new docs page `engineering/stability.md` that articulates exactly what 1.0 commits us to: - Versioning: workspace + wheel bumped in lock-step; semver as of 1.0; the rc.x series carried no contract. - Stable API: explicit list of the Python and Rust surface that's covered by semver, and an equally explicit list of what isn't (the `solver_micro` benches, the rez_solver submodule layout, `Reduction` / failure detail wording, internal performance characteristics). - Supported Pythons: 3.9–3.13 today (one abi3-py39 wheel per platform / architecture covers all of them), 3.14 when pyo3 catches up. - Supported rez: doesn't import rez; the `PackageData.from_rez` helper is exercised against rez 3.3.0 (vendored submodule) and via the 188-case differential. - Correctness contract: same `(name, version)` set as rez, same variant index, same status — divergence on any is a release blocker. - What's modelled vs not: a single table mapping each rez feature to "supported" or "not supported, fall back to rez." - Performance is explicitly NOT in the contract — the `solver_micro` baselines catch regressions on internal hot paths, not absolute numbers. - Breaking-change policy: RFC issue → addition with deprecation → one MINOR cycle → removal at MAJOR. - Regression reporting flow. The CHANGELOG entry for the `Unreleased` / `1.0.0` is intentionally specific about which PRs land in 1.0 so a reader of the file can trace each line back to a concrete change. Linked from: - README's new "Release notes & stability" section near the bottom. - The introduction page's "Next steps" list. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The 188-case rez differential previously compared rer's resolution against rez's recorded one as a `(name, version)` set, ignoring the variant index. A future refactor could silently make rer pick a different variant of the same version (with the variant's own `requires` being different) and the gate wouldn't catch it. The README's "Validated 1:1" status bullet itself called this out as a gap.
This PR closes it — the differential now enforces all three of (status, name+version set, variant index). Concretely the right correctness commitment for 1.0 (Option A path).
Changes
Result
188 / 188 still pass under the strict check. The solver was already producing rez-faithful variant choices on the benchmark; we just weren't enforcing it. The gate will now catch any regression that changes which variant gets picked.
```
running 1 test
test test_rez_benchmark_correctness ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 17.93s
```
Docs
What's next toward 1.0 (Option A path)
VariantSelectMode::intersection_priority#63): implement or close as a documented 1.0 limitation.🤖 Generated with Claude Code