Skip to content

fix FP16 tests precision overflow [MOD-13429]#884

Merged
dor-forer merged 5 commits intomainfrom
dorer-fix-fp16-tests-precistion
Jan 11, 2026
Merged

fix FP16 tests precision overflow [MOD-13429]#884
dor-forer merged 5 commits intomainfrom
dorer-fix-fp16-tests-precistion

Conversation

@dor-forer
Copy link
Collaborator

@dor-forer dor-forer commented Jan 8, 2026

Describe the changes in the pull request

This PR fixes FP16 SVS tests' precision and assertion issues:

  • Fix FP16 overflow in batch iterator tests: Scale vector indices by 0.1 to avoid FP16 overflow in L2 distance calculations.
  • Changed the ARM machine include the SVE optimizations

Tests updated:
'test_batch_iterator'
test_override
test_batch_iterator_basic

Which issues this PR fixes

  1. MOD-13429

Main objects this PR modified

  1. ...
  2. ...

Mark if applicable

  • This PR introduces API changes
  • This PR introduces serialization changes

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses FP16 precision overflow issues in vector similarity search tests by introducing scale factors to prevent distance calculations from exceeding FP16's maximum value (~65504). The changes ensure test vectors remain within safe numerical ranges while preserving test correctness.

Key changes:

  • Added scale factor (0.1f) to SVS tiered index batch iterator tests to prevent overflow when n=1000
  • Added scale factor (0.01f) to FP16 override and batch iterator tests to prevent overflow when n=250
  • Upgraded ARM CI infrastructure from t4g.medium to r8g.xlarge for better performance

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
tests/unit/test_svs_fp16.cpp Introduced 0.1f scale factor to prevent FP16 overflow in tiered batch iterator tests with proper documentation
tests/unit/test_fp16.cpp Added 0.01f scale factor to test_override and test_batch_iterator_basic, plus tolerance-based comparison for FP16 precision loss
.github/workflows/arm.yml Upgraded EC2 instance type to r8g.xlarge for improved CI performance

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@codecov
Copy link

codecov bot commented Jan 8, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.13%. Comparing base (f6df960) to head (39ce274).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #884   +/-   ##
=======================================
  Coverage   97.13%   97.13%           
=======================================
  Files         129      129           
  Lines        7614     7615    +1     
=======================================
+ Hits         7396     7397    +1     
  Misses        218      218           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@meiravgri
Copy link
Collaborator

Are the tests in the description relevant?

@dor-forer
Copy link
Collaborator Author

@meiravgri Sorry it was an old description

@dor-forer dor-forer added this pull request to the merge queue Jan 11, 2026
Merged via the queue into main with commit f5e69ac Jan 11, 2026
17 checks passed
@dor-forer dor-forer deleted the dorer-fix-fp16-tests-precistion branch January 11, 2026 10:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants