fix FP16 tests precision overflow [MOD-13429]#884
Conversation
There was a problem hiding this comment.
Pull request overview
This PR addresses FP16 precision overflow issues in vector similarity search tests by introducing scale factors to prevent distance calculations from exceeding FP16's maximum value (~65504). The changes ensure test vectors remain within safe numerical ranges while preserving test correctness.
Key changes:
- Added scale factor (0.1f) to SVS tiered index batch iterator tests to prevent overflow when n=1000
- Added scale factor (0.01f) to FP16 override and batch iterator tests to prevent overflow when n=250
- Upgraded ARM CI infrastructure from t4g.medium to r8g.xlarge for better performance
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| tests/unit/test_svs_fp16.cpp | Introduced 0.1f scale factor to prevent FP16 overflow in tiered batch iterator tests with proper documentation |
| tests/unit/test_fp16.cpp | Added 0.01f scale factor to test_override and test_batch_iterator_basic, plus tolerance-based comparison for FP16 precision loss |
| .github/workflows/arm.yml | Upgraded EC2 instance type to r8g.xlarge for improved CI performance |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #884 +/- ##
=======================================
Coverage 97.13% 97.13%
=======================================
Files 129 129
Lines 7614 7615 +1
=======================================
+ Hits 7396 7397 +1
Misses 218 218 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
Are the tests in the description relevant? |
|
@meiravgri Sorry it was an old description |
Describe the changes in the pull request
This PR fixes FP16 SVS tests' precision and assertion issues:
Tests updated:
'test_batch_iterator'
test_overridetest_batch_iterator_basicWhich issues this PR fixes
Main objects this PR modified
Mark if applicable