perf(vector): migrate to InternalId = u32 throughout vector index + search

[mosuka]

Cross-cutting data-structure rewrite tracked under #537.

Current state

Doc IDs flow through the vector index as u64:

• HnswGraph::id_to_index: AHashMap<u64, usize> and nodes: Vec<Vec<Vec<u64>>> (vector/index/hnsw/graph.rs:21,27).
• ConcurrentHnswGraph::nodes: HashMap<u64, Vec<RwLock<Vec<u64>>>> (vector/index/hnsw/writer.rs:40-41).
• Candidate / ResultCandidate carry id: u64.
• IVF inverted lists hold Vec<(u64, String, Vector)>.

A single vector segment is capped at u32 entries elsewhere (IVF / Flat headers use vector_count: u32); the u64 is unnecessary inside the index.

Proposed direction

• Define InternalId(u32) with TryFrom<u64> at the segment boundary.
• HNSW neighbour lists: Vec<u32> (50% saving vs Vec<u64>).
• IVF inverted list ids: Vec<u32>.
• Pair with the CSR HNSW migration (perf(vector/index): Round-3 roadmap — Qdrant/LanceDB parity #535 children) so the neighbour arena is a single contiguous Vec<u32>.
• Keep external u64 doc id at the searcher result boundary.

Acceptance

• HNSW graph RAM drops by ~50% on the neighbour arrays.
• Candidate heap packed format (X-04) becomes natural.

References

• Lucene uses int ordinals throughout vector index.
• Qdrant uses u32 PointOffset.

[mosuka]

Splitting into sub-issues per CLAUDE.md sizing rules

This umbrella covers a cross-cutting type change touching HNSW (graph + writer), IVF, and the on-disk format — well over the "800 lines / multi-layer" threshold for a single PR. Split into four sub-issues so each lands with a focused review:

Sub-issue	Scope	Status
#709	HnswGraph in-memory Vec<u32> neighbour lists with serde compat shim (wire format unchanged)	OPEN — starting now
(b)	ConcurrentHnswGraph writer-side u32 narrowing	(file when (a) lands)
(c)	IVF inverted lists Vec<(u64, String, Vector)> u32 narrowing	(file when (b) lands)
(d)	On-disk format bump (drop the serde compat shim, full u32 wire)	(file when (b)/(c) land — needs format version bump)

Each sub-issue includes its own Phase 0 bench gate. (b)/(c)/(d) are deliberately not opened upfront — they will be filed when their predecessors land and the actual implementation plan can be written against current code rather than against a guess. This umbrella stays OPEN as the tracker.