feat: object store decides scheduler type#6373
Conversation
The lite scheduler was designed for io_uring (lazy polling, no dedicated I/O loop thread), so a file+uring:// object store should automatically use it. This changes SchedulerConfig::use_lite_scheduler from bool to Option<bool> and resolves the final value in ScanScheduler::new() using the object store's preference as the default, with the env var serving as an override. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ReviewClean, focused PR. Two items worth addressing: P1: use_lite_scheduler: if std::env::var("LANCE_USE_LITE_SCHEDULER").is_ok() {
Some(true)
} else {
None
},
P1: No test coverage for the new fallback logic The tri-state resolution ( |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
… test Address review feedback: - Use str_is_truthy to parse the env var value so that "0"/"false" map to Some(false) (force-disable) instead of always enabling on presence. - Add test_object_store_selects_scheduler covering all four cases: None with memory (standard), None with file+uring (lite), Some(false) override, and Some(true) override. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
file+uring:// is a local filesystem backend (io_uring), but is_local() only matched "file" and is_cloud() excluded only "file" and "memory", so file+uring was incorrectly classified as cloud. This affected heuristics like scanner byte-width thresholds and skipped local-only code paths (copy, remove_dir_all). Define is_cloud() in terms of is_local() to keep the logic in one place. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Summary
ObjectStore::prefers_lite_scheduler()that returnstrueforfile+uring://stores, so the lite scheduler is used automatically without needing the env varSchedulerConfig::use_lite_schedulerfrombooltoOption<bool>—Some(true/false)overrides,Nonedefers to the object store's preferenceLANCE_USE_LITE_SCHEDULERenv var still works as an override when explicitly setTest plan
cargo check -p lance-io --tests --benchescompiles cleanlycargo test -p lance-io— all 148 tests passcargo clippy -p lance-io --tests --benches -- -D warnings— no warnings🤖 Generated with Claude Code