chore: release v0.3.2#4
Merged
Merged
Conversation
|
|
polaz
added a commit
that referenced
this pull request
May 21, 2026
…imitives Closes the trigger surface for G103 sub-system #4 (background overflow compactor). After this slice the catalog can: (a) discover every bucket with stale overflow, and (b) compact all of them in one driver tick. New modality API on `TimeSeriesStore`: - `list_overflow_buckets() -> Vec<(u32, NodeId)>` — single prefix scan over `ts_overflow:` folded to unique `(label_id, bucket_id)` pairs. Memory bounded by the number of stale buckets, not the overflow volume. Malformed overflow keys (< 12-byte suffix) skip with `tracing::warn!` instead of erroring. New catalog API on `BucketCatalog`: - `compact_all_pending() -> CatalogResult<usize>` — discover via `list_overflow_buckets`, run `compact_if_needed` against each candidate, return count of buckets that actually compacted (i.e. were above `OVERFLOW_COMPACT_THRESHOLD`). Production wires this into a periodic `tokio::time::interval` task per shard above the catalog — the catalog itself spawns no threads. 4 new tests: - `list_overflow_buckets_returns_unique_pairs_across_multiple_entries` — 4 overflow entries across 3 unique `(label, bucket)` pairs; list returns exactly 3 in sorted order. Same `bucket_id` under different labels stays distinct (the pair, not the bucket_id alone, identifies a target). - `list_overflow_buckets_empty_when_no_overflow` — empty path. - `compact_all_pending_discovers_and_compacts_every_stale_bucket` — 3 buckets: A and B with 51 overflow each (above threshold), C with 10 (below). Driver compacts A and B (returns 2), leaves C's overflow intact. - `compact_all_pending_no_op_when_overflow_set_empty` — empty baseline returns 0. Sub-system #4 status update in lib.rs comments (next commit on GAPS.md will mirror): the trigger primitive landed in Slice C; the discover-and-compact primitive lands here. Remaining for the full sub-system: spawning the periodic driver above the catalog (the catalog deliberately spawns no threads — driver lives in a service layer; out of scope for this crate). Verified: cargo clippy --workspace --all-targets clean (-D warnings), 183/183 modality + timeseries tests pass.
polaz
added a commit
that referenced
this pull request
May 21, 2026
…with engine-backed restart monotonicity User pointed out: lsm-tree IS the durable layer. The prior commit (03f182c) documented restart-monotonicity as a follow-up needing "a persistence layer"; that layer was already on hand via the StorageEngine. This commit closes the gap end-to-end. ## New: PersistentMonotonicHlcClock Wraps the same CAS-monotonic logic as MonotonicHlcClock but adds: - **Engine-backed checkpoint** under `Partition::Schema` key `schema:meta:ts_clock:<shard_id_u16_be>` (8-byte big-endian i64). - **Seed-on-load** from `max(persisted_last + 1, wall_clock_now)`. First stamp after open is strictly > both the persisted value AND the current wall clock, so a backward NTP jump can't violate monotonicity. - **Periodic persistence** every 64 calls to `next()` on the hot path. Best-effort: a transient engine error logs `tracing::warn` and the next checkpoint catches up. Worst-case data loss on crash is ≤63 stamps; combined with the seed-on-load rule, the only failure mode is a backward wall-clock jump that exceeds the lost-stamp window. - **Explicit `checkpoint()`** for the catalog flush path — forces a write regardless of cadence so crash recovery sees a stamp strictly bounding every flushed measurement. ## New: MonotonicHlcClock public surface - `with_prior(prior: i64)` — explicit seed (used by PersistentMonotonicHlcClock and future Raft-leader-stamped variants to resume from a replicated last stamp). - `current() -> i64` — read last-issued stamp without burning a number on the read (checkpoint reads, observability hooks). ## Free functions (also publicly exported) - `load_last_stamp(engine, shard_id) -> Result<Option<i64>>` — reads the checkpoint key; `None` for first-ever open; tolerates corrupt body (8-byte length check) by returning `None` with a warning (not an error — a corrupt checkpoint falls back to wall clock, which is safe). - `persist_last_stamp(engine, shard_id, stamp) -> Result<()>` — overwrites the checkpoint key with 8-byte BE stamp. ## Storage key choice `Partition::Schema` with reserved key `schema:meta:ts_clock:<shard_id_u16_be>` — mirrors the convention used by `schema:current_revision:<kind>:<name>` etc. (engine- reserved metadata keys live under the schema partition). ## Tests (5 new in clock::tests) - `persistent_clock_seeds_from_wall_when_no_checkpoint` — first open, no persisted value, first stamp ≥ wall clock at open time, strict monotonic from there. - `persistent_clock_resumes_above_persisted_on_reopen` — burn 64 stamps, force checkpoint, drop clock, reopen. First stamp from the new instance must be strictly > the persisted value regardless of wall clock state (simulates the backward NTP jump). - `persistent_clock_explicit_checkpoint_writes_current_stamp` — explicit `checkpoint()` after a single `next()` persists exactly that stamp. - `persistent_clock_shards_dont_interfere` — two shards on the same engine persist to disjoint keys; writes on shard 1 leave shard 2's checkpoint absent. - `persistent_clock_corrupt_checkpoint_falls_back_to_wall_clock` — plant garbage at the checkpoint key, `load_last_stamp` returns `None` (with warn), clock opens normally seeded from wall. ## Updated documentation - `MonotonicHlcClock` docstring no longer claims persistence is a "follow-up" — it now points readers to PersistentMonotonicHlcClock for production use and explains when the bare variant is appropriate (tests, future Raft-stamped variants). ## Verified - `cargo clippy --workspace --all-targets -- -D warnings` clean. - `cargo nextest run -p coordinode-timeseries` 49/49 pass (5 new persistence tests + prior 44). ## Out of scope (genuinely separate work) - **Catalog wiring**: BucketCatalog construction sites in production code (not yet wired — the v0.4-alpha gate work that hooks catalog into coordinode-query/server is where the PersistentMonotonicHlcClock will replace MonotonicHlcClock). - **Raft-leader-stamped variant** for multi-replica CE: needs consensus log integration; will land alongside the Raft-aware bucket reopen path.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤖 New release
coordinode-core: 0.3.1 -> 0.3.2coordinode-storage: 0.3.1 -> 0.3.2coordinode-search: 0.3.1 -> 0.3.2coordinode-vector: 0.3.1 -> 0.3.2coordinode-query: 0.3.1 -> 0.3.2coordinode-raft: 0.3.1 -> 0.3.2coordinode-embed: 0.3.1 -> 0.3.2coordinode-auth: 0.3.1 -> 0.3.2coordinode-server: 0.3.1 -> 0.3.2coordinode-s3: 0.3.1 -> 0.3.2Changelog
coordinode-serverThis PR was generated with release-plz.