[MOD-14955] Add DistanceCalculatorWithNorm

[dor-forer]

Describe the changes in the pull request

This PR is stacked on #1000. Review only the incremental diff against dor-forer-MOD-14952-support-quant-normalization.

Cherry-picks the two commits from ARM-software/VectorSimilarity-for-Arm#3: 493ee7a8 and c27401d6.

This PR adds DistanceCalculatorWithNorm, which invokes the selected SQ8 kernel and applies the mean-normalization correction terms needed to recover the distance between the original vectors. The follow-up commit makes metadata reads safe for unaligned storage.

The calculator integrates with the construction-time dispatch introduced in #999:

• HNSW caches a non-virtual callback and a minimal immutable context containing only the stored/query kernels and mean_sum_squares.
• The candidate loop does not call IndexCalculatorInterface::calcDistance or calcDistanceForQuery, so it does not restore the per-distance vtable lookup removed by MOD-14916 Devirtualize HNSW / brute-force search hot path #937.
• Existing calculators retain the stateless raw-kernel path. Stored-to-stored normalized L2 also uses that path because its mean terms cancel.

Tests cover FP32/FP16, L2/IP, symmetric/asymmetric, zero-mean, randomized-vector, and stateless/stateful dispatch behavior.

Which issues this PR fixes

1. MOD-14955

Main objects this PR modified

1. DistanceCalculatorWithNorm
2. Normalized-distance component unit tests

Mark if applicable

• This PR introduces API changes
• This PR introduces serialization changes

Validation

• make format
• make check-format
• Release build of test_components and test_hnsw
• test_components: 40/40 passed
• test_hnsw excluding the two external serialization fixtures: 169/169 passed
• git diff --check dor-forer-MOD-14952-support-quant-normalization...HEAD
• Focused stateless-dispatch microbenchmark: 36 alternating-order rounds, 20M calls per path per round; cached raw kernel averaged 2.75 ns/call and DistanceDispatch averaged 2.69 ns/call (no measurable regression within run-to-run noise)

---

Note

Medium Risk
Changes core VecSim distance math and blob layouts for normalized SQ8 indices; mistakes would skew ANN search, but behavior is heavily covered by new unit tests including numerical edge cases.

Overview
Adds DistanceCalculatorWithNorm, which runs the existing SQ8 distance kernels on mean-centered quantized storage and applies IP-only correction terms (x_mean_ip, y_mean_ip, mean_sum_squares) so reported distances match the original (unshifted) vectors. Normalized L2 skips runtime correction: queries are centered at preprocess time (y - mean), so stored/query kernels already compute the true L2 distance; L2 paths expose stateless DistanceDispatch (raw kernels), while IP uses stateful dispatch for the correction callbacks.

QuantPreprocessor (WithNorm) is aligned with that split: L2 query blobs store centered values and metadata sums over centered data; IP keeps raw query values and records y_mean_ip from the original query. sq8 metadata layout only adds the extra mean_ip slot when WithNorm + IP (not for normalized L2).

Unit tests cover FP32/FP16, L2/IP, symmetric/asymmetric paths, zero mean, large-offset L2 stability, random vectors, odd dimensions (unaligned metadata reads via load_unaligned), and dispatch stateless vs stateful behavior.

^{Reviewed by Cursor Bugbot for commit 385741f. Bugbot is set up for automated code reviews on this repo. Configure here.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.15%. Comparing base (de00dd5) to head (385741f).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1002      +/-   ##
==========================================
- Coverage   97.16%   97.15%   -0.02%     
==========================================
  Files         141      141              
  Lines        8293     8328      +35     
==========================================
+ Hits         8058     8091      +33     
- Misses        235      237       +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[ofiryanai]

P3: Centered L2 no longer consumes x_mean_ip, y_mean_ip, or mean_sum_squares, but WithNorm still computes both dot products and stores an extra float in every L2 storage/query blob. Make correction metadata conditional on Metric == VecSimMetric_IP to avoid dead O(dim) preprocessing work and storage.

[dor-forer]

Fixed in e0a0a20: gated the extra mean_ip metadata slot/computation to WithNorm && Metric == VecSimMetric_IP (in sq8.h and preprocessors.h) instead of WithNorm alone. L2 no longer computes or stores x_mean_ip/y_mean_ip since the centered-L2 correction never reads them. Also had to fix a pre-existing test in QuantPreprocessorWithNormMetricTest (1b39f3f) that was asserting on mean_ip_index<Metric>() unconditionally for L2 too — caught by Bugbot after the metadata layout changed size.

[ofiryanai]

P4: The underlying odd-dimension SQ8 kernels are covered by #1001, but the promised odd-dimension DistanceCalculatorWithNorm integration case is still missing. Add one to exercise the complete normalized metadata layout and correction path.

[dor-forer]

Added DistanceCalculatorWithNormTest.OddDimension_MatchesBruteForce (dim=13) covering IP and L2 symmetric + asymmetric paths through the unaligned storage-blob metadata offset.

[ofiryanai]

Blocking correctness and alignment concerns are resolved. The remaining P3/P4 comments are non-blocking.

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit e0a0a20. Configure here.}