fix(timeseries): correct ILP shard routing and scan column output#1
Closed
emanzx wants to merge 1 commit intoNodeDB-Lab:mainfrom
Closed
fix(timeseries): correct ILP shard routing and scan column output#1emanzx wants to merge 1 commit intoNodeDB-Lab:mainfrom
emanzx wants to merge 1 commit intoNodeDB-Lab:mainfrom
Conversation
ILP ingest used VShardId::from_key() (FxHash) for per-series core routing while the SQL scan planner used VShardId::from_collection() (DJB hash). Different hash functions on the same collection name routed data and queries to different Data Plane cores, making ingested timeseries data invisible to scans. Additionally, the timeseries scan handler only emitted hardcoded "timestamp" and "value" columns, ignoring all other columns (tags, additional fields) stored in the columnar memtable. Changes: - ILP listener now routes via VShardId::from_collection() to match the scan planner. Per-series sharding deferred until scan supports fan-out across cores. - Scan handler iterates all memtable schema columns and resolves Symbol (tag) columns through the SymbolDictionary. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
7 tasks
farhan-syah
added a commit
that referenced
this pull request
Apr 5, 2026
Fix #1: TriggerBatchRow now decodes msgpack directly to HashMap<String, nodedb_types::Value> via rmpv, skipping the serde_json::Value intermediate entirely. The full chain: raw bytes → rmpv::Value → nodedb_types::Value (direct) Previously: raw bytes → serde_json::Value → nodedb_types::Value - TriggerBatchRow cache type: serde_json::Map → HashMap<String, Value> - Added rmpv_to_value() converter (msgpack → nodedb_types::Value) - fire_after/before/instead/common: params changed from &serde_json::Map to &HashMap<String, Value> - Removed map_to_hashmap() — no longer needed - dml_hook.rs: deserialize_value_to_fields returns Value map directly - RetryEntry: fields changed to HashMap<String, Value> - dispatcher: converts at WriteEvent boundary (one remaining point) Fix #2: dispatch/other.rs tuple responses (Vec<(u64,i64,f64)>, Option<(i64,f64)>) now use encode() directly instead of encode_serde() since zerompk implements ToMessagePack for tuples natively.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
VShardId::from_key()(FxHash) while SQL scan usedVShardId::from_collection()(DJB hash) — same collection name routed to different Data Plane cores, making ingested data invisible to queries. Now both usefrom_collection().timestampandvaluecolumns. Now iterates all memtable schema columns and resolves Symbol (tag) columns through the SymbolDictionary.Test plan
SELECT * FROM metrics LIMIT 5returns all columns (host, region, cpu_usage, mem_usage, timestamp)cargo clippyclean,cargo fmt --allapplied🤖 Generated with Claude Code