perf: reuse IVF coarse-quantizer routing across index segments

[majin1102]

Summary

• add a content-derived IVF coarse-quantizer fingerprint to vector index segment metadata
• rank shared IVF centroids once and fan the selected partitions out to compatible index segments
• fail closed to existing per-segment routing for missing, malformed, or different fingerprints
• add query-plan metrics and a LAION 100M A/B benchmark harness

Discussion: #8965

Format change

Adds the following optional field to VectorIndexDetails:

optional bytes coarse_quantizer_fingerprint = 10;

The value is a 32-byte SHA-256 identity derived from the routing metric and ordered IVF centroids. It is an optional performance hint: old readers ignore it, and new readers fall back to per-segment routing when it is absent or invalid.

Benchmark plan

Compare two six-segment IVF_RQ indexes built from the same LAION 100M dataset version, fragment groups, IVF centroids, and RaBitQ model. The only A/B variable is fingerprint presence. Pre-warm both indexes, verify the routing path with query-plan counters, measure QPS and P50/P95/P99 latency using _rowid/_distance projections, and calculate Recall@k outside the timed interval from the actual ANN results.

Detailed benchmark results will be posted in the Discussion.

Validation

• cargo check -p lance --tests
• cargo clippy --all --tests --benches -- -D warnings
• cd python && uv run make lint
• pre-commit hooks: Ruff, rustfmt, typos, and Cargo lockfile sync