try perf things#1
Merged
Merged
Conversation
jaredLunde
pushed a commit
that referenced
this pull request
Feb 4, 2026
Update README.md with link to SlateDB
jaredLunde
added a commit
that referenced
this pull request
May 16, 2026
Encodes the handoff state machine as a typed enum (Idle/Warming/Freezing/ Cutover) instead of a single AtomicBool. Preserves exact "any non-Idle" semantics in the two read sites (flush_packs gate, checkpoint gate) via HandoffPhase::is_active(), so behavior is unchanged. Sets up future per-phase behavior (#1 atomic flush refactor, future PIOD work) without forcing it now. Validated by handoff_sequential_50_crh (50/50 clean, 574s). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
jaredLunde
added a commit
that referenced
this pull request
May 16, 2026
…suite The 217-test tests/integration suite (which my local --tests run missed) caught four real issues from refactor #1 (c3e560b atomic flush_packs): 1. flush_to_s3 must always push the manifest, not just when packs were uploaded. Cold readers depend on the S3 manifest's presence to bootstrap; an all-zero-write export that cross-dedups to zero packs was leaving S3 empty (prop_zero_block_roundtrip). 2. flush_dirty_inner skipped flushes when a flushing file was on disk even after every claimed block had been promoted back to DIRTY (via guest write) — a stale rotation the cache couldn't recover from on its own. Now detected via `has_any_syncing()`: if `flushing_active` is set but no block is SYNCING, clean up the orphan and proceed (state_transition_table_completeness). 3,4. test_c1_…_causes_data_loss and test_manifest_failure_in_drain_… asserted the OLD non-atomic ordering's failure modes (blocks evicted to NP, data lost across manifest failure + crash). Atomic flush eliminates those windows: manifest failure returns Err BEFORE eviction; outer recovery re-dirties via the flushing file; crash recovery preserves data. Updated assertions to verify the new (stronger) invariants. Also: handoff_durability test fixture now bumps RLIMIT_NOFILE to 65536 so foyer's SSD cache (16 GB, many segment fds) doesn't hit EMFILE on CI runners with the default 1024 soft limit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.