fix: tolerate concurrent manifest cleanup

[lance-gatefixer]

Summary

• Ignore a typed NotFound only when a manifest listed by cleanup is older than the cleanup snapshot and has been removed concurrently.
• Keep failures for the current snapshot manifest, newer manifests, and all other I/O errors fatal.
• Add regression coverage for a manifest deleted between listing and reading.

Root cause

V2 manifest filenames are intentionally encoded as u64::MAX - version, so 18446744073709548523.manifest represents logical version 3092 rather than an underflowed version. Two concurrent cleanup calls can race after one lists that old manifest: the other removes it before the first reads it, and the first propagated the resulting NotFound.

Validation

• cargo test -p lance cleanup_ignores_old_manifest_removed_after_listing
• cargo test -p lance dataset::cleanup::tests
• cargo fmt --all -- --check
• cargo clippy --all --tests --benches -- -D warnings

Fixes #8212

[lance-gatekeeper]

⚠️ Gate recommendation: approve with a non-blocking risk.

The patch addresses the reported list/read race at the right boundary: it uses parsed logical versions, keeps current/newer manifests and non-NotFound failures fatal, and covers deletion between the manifest and index reads.

The production report does not timestamp-confirm the competing deletion, and the test exercises the stale location directly rather than two full concurrent cleanups. An end-to-end concurrency regression would reduce that remaining uncertainty, but the implementation and focused coverage support this fix.

[Xuanwo]

Thank you!