test(rewards): extract reward-agnostic contract tests and scope fixtures - #323
Conversation
|
Warning Review limit reached
Next review available in: 54 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughChangesReward fixtures and input construction are scoped to Reward test organization
Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
k-chrispens
left a comment
There was a problem hiding this comment.
Generally looks good to me, I think we should have it somewhere on the roadmap to add some better tests here, but not quite sure where that fits in these days with all the loss function talk we had back in June!
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the placeholder 1.0 absolute bar with a value grounded in the 1VME case: the true structure scores ~0.0018 and a 0.5 A perturbation ~0.034, so 0.01 sits ~5x above the truth (robust to device/precision variance) yet ~3x below the perturbed loss and meaningfully fails a wrong structure. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The extract-contract-tests refactor removed this import (its users moved out), but main's #328 added TestStructureToRewardInput which still needs it. The 3-way merge kept the new class and dropped the import, leaving a NameError. Restore the import. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
57ed301 to
f399ffe
Compare
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…res (#323) The older PR #272 is broken into 3 smaller PRs for (1) improve synthetic data generation, (2) refactor reward tests, and (3) add structure factor reward. This PR addresses (2). ## Changes relative to main branch 1. Extract reward-agnostic contract tests into a shared `test_reward_function_contract.py` from `test_real_space_density_reward.py`. 2. Move reward fixtures from `tests/conftest.py` into a session scoped `tests/rewards/conftest.py`. 3. Add `tests/rewards/reward_input_helpers.py` for shared reward-input construction. ## Changes relative to the previous PR #272 No changes were requested or comments made for these files in the previous PR, except for the suggestion on reward fixture scope. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Tests** * Added GPU-marked, reward-agnostic contract tests covering protocol conformance, determinism, output correctness, ranking behavior, gradient flow, batching, loss optimization improvement, and numerical stability. * Refocused real-space density reward tests to edge cases and input-to-reward-input validation, removing substantial prior coverage. * Improved reward test infrastructure with suite-scoped fixtures for parsing the CIF, building the reward function, and extracting coordinates, plus shared helpers for scattering indices and reward inputs. * Removed overlapping session-scoped fixtures from the broader test setup. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Doris Mai <dorismai@unknown32bcc963f521.attlocal.net> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Doris Mai <dorismai@mac.lan>
The older PR #272 is broken into 3 smaller PRs for (1) improve synthetic data generation, (2) refactor reward tests, and (3) add structure factor reward. This PR addresses (2).
Changes relative to main branch
test_reward_function_contract.pyfromtest_real_space_density_reward.py.tests/conftest.pyinto a session scopedtests/rewards/conftest.py.tests/rewards/reward_input_helpers.pyfor shared reward-input construction.Changes relative to the previous PR #272
No changes were requested or comments made for these files in the previous PR, except for the suggestion on reward fixture scope.
Summary by CodeRabbit