feat: pluggable index cache via CacheBackend trait

[wjones127]

Previously the Session's index cache was hardcoded to Moka. This adds a
CacheBackend trait so users can provide their own cache implementation,
with MokaCacheBackend as the default.

Changes:

• Split cache.rs into cache/{mod,backend,keys,moka}.rs
• Add CacheBackend trait with get, insert, get_or_insert,
  invalidate_prefix, and stats methods
• Rename CacheKey::type_id() to type_name() (returns &'static str)
• Accept custom CacheBackend in DatasetBuilder
• Document why get_or_insert uses Pin<Box<dyn Future>> (object safety)

Serialization and zero-IO reconstruction are in the follow-up PR #6223.

[github-actions]

PR Review: feat: pluggable index cache via CacheBackend trait

P0: TOCTOU race in get_or_insert — dedup is broken

The check-then-register pattern in get_or_insert has a race between the two separately-locked critical sections:

// Block 1: check (drops lock at end of block)
{
    let map = self.in_flight.lock().await;
    if let Some(rx) = map.get(&cache_key) { ... }
}
// <-- gap: another task can also see an empty map here

// Block 2: register as leader (re-acquires lock)
let (tx, rx) = tokio::sync::watch::channel(None);
{
    let mut map = self.in_flight.lock().await;
    map.insert(cache_key.clone(), rx);
}

Between dropping the lock in block 1 and re-acquiring it in block 2, N tasks can all see an empty in-flight map and all decide they are the "leader." The last one to insert overwrites the earlier receivers, so:

1. The loader runs N times (dedup completely defeated).
2. Tasks that cloned an earlier receiver get RecvError when that sender drops without the value propagating through the map.

Fix: Merge the check and register into a single critical section:

let mut map = self.in_flight.lock().await;
if let Some(rx) = map.get(&cache_key) {
    let mut rx = rx.clone();
    drop(map);
    // ... wait for leader ...
} else {
    let (tx, rx) = tokio::sync::watch::channel(None);
    map.insert(cache_key.clone(), rx);
    drop(map);
    // ... run loader ...
}

The test_get_or_insert_dedup test passes by luck because tokio::task::yield_now() with a broadcast barrier doesn't reliably interleave into this gap. A test with a longer sleep or explicit synchronization at the gap point would expose this.

P1: type_tag relies on unspecified pointer identity

fn type_tag<T: 'static>() -> [u8; 8] {
    (std::any::type_name::<T>().as_ptr() as u64).to_le_bytes()
}

The Rust spec does not guarantee that type_name::<T>().as_ptr() returns the same address across calls, nor that distinct types produce distinct addresses (string deduplication/interning is compiler-implementation-defined). The old code used TypeId as a key component, which has correct identity semantics.

If two different types happen to collide (same pointer or same string literal address), the cache will silently return a wrong type and the downcast::<T>().unwrap() will panic at runtime.

Consider using TypeId bytes instead. TypeId implements Hash and Eq, so you could hash it to get a stable discriminator, or use transmute to extract its bytes (it's currently 128 bits).

P1: invalidate_prefix is fire-and-forget

pub fn invalidate_prefix(&self, prefix: &str) {
    let prefix_bytes = self.make_prefix(prefix);
    let cache = self.cache.clone();
    tokio::spawn(async move {
        cache.invalidate_prefix(&prefix_bytes).await;
    });
}

This means invalidation is not guaranteed to complete before subsequent cache reads. A caller that invalidates then immediately reads could get stale data. The old code was synchronous. If this must remain sync (non-async signature), at minimum document this caveat prominently, or consider changing the signature to async fn.

Minor notes

• DeepSizeOf for LanceCache now returns 0, which is a silent behavioral regression for any code that relies on deep_size_of() for memory accounting. Consider at least delegating to approx_size_bytes().
• WeakLanceCache::get_or_insert lost all dedup behavior — concurrent loads for the same key will all run the loader independently. The old code used moka's optionally_get_with which handled this.

[codecov]

Codecov Report

❌ Patch coverage is 83.43465% with 109 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
rust/lance-core/src/cache/mod.rs	87.89%	53 Missing and 5 partials ⚠️
rust/lance/src/session.rs	30.43%	16 Missing ⚠️
rust/lance-core/src/cache/moka.rs	76.08%	11 Missing ⚠️
rust/lance/src/dataset/builder.rs	46.66%	8 Missing ⚠️
rust/lance-core/src/cache/keys.rs	56.25%	7 Missing ⚠️
rust/lance-core/src/cache/backend.rs	0.00%	6 Missing ⚠️
rust/lance/src/session/index_caches.rs	66.66%	3 Missing ⚠️

📢 Thoughts on this report? Let us know!