Conversation
Introduce reverse scanning and async variants across the store and RocksDB backend to support efficient "latest-version" lookups and avoid blocking the async runtime (scan_reverse, get_latest_by_index_prefix, insert_batch_async, etc.). Add LIKE/ILIKE expression support to live filter evaluation using regex-based pattern translation. Refactor SharedTableProvider to cache per-column vector hot stores and make vector staging/upsert/delete operations asynchronous and batch-capable. Add helper loader methods in DmlExecutor to centralize loading rows for file cleanup. Misc: small formatting and test updates, GitHub release workflow added a warning step if Docker Hub README update fails, and removed rustc-wrapper from Cargo config.
Bump DataFusion to 52.3.0 (Arrow 57.3.0) and update related Rust dependencies in Cargo.toml/Cargo.lock. Several crates were bumped (clap, cc, once_cell, sysinfo, tempfile, zip, miniz_oxide, assert_cmd, tracing-subscriber, etc.), typed-path was added, and obsolete arbitrary/derive_arbitrary entries were removed from the lockfile. Regenerated web WASM artifacts and TypeScript defs to expose new wasmtimestampformatter APIs and updated closure symbol names, and replaced prebuilt Dart native libraries for Android (arm64/x86_64) and iOS. These updates bring dependency compatibility with the newer DataFusion/Arrow versions and include regenerated SDK binaries and bindings.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request includes several dependency upgrades, minor code cleanups, and some targeted refactoring to improve maintainability and error handling. The most significant changes are grouped below.
Dependency and Tooling Upgrades
Cargo.tomlfor bug fixes and improved compatibility, includingdatafusion,arrow,clap,sysinfo,tempfile,assert_cmd,once_cell,zip,miniz_oxide, and others. [1] [2] [3] [4] [5]rustc-wrapper = "sccache"from.cargo/config.toml, which may affect build caching behavior.Refactoring and Code Quality
DmlExecutorto use dedicated helper methods (load_user_row_for_cleanup,load_shared_row_for_cleanup), reducing code duplication and improving error handling. [1] [2] [3] [4]Configuration and Workflow Improvements
kalamdb-configsfor clarity and to ensure correct public API exposure. [1] [2]Documentation
next_storage_key_bytesto warn against using it for prefix scans, preventing subtle bugs.These changes collectively improve project maintainability, keep dependencies up to date, and enhance developer experience.