feat(ram): restore write-back RAM tier as default, keep write-around opt-in - #269
Merged
Merged
Conversation
…opt-in PR#246 (ed4d72b) switched PUT to write-around (direct-to-disk + GET read-promotion) to remove the per-PUT 1MiB arena memcpy (44% serve CPU, PUT 3.42->4.54 GB/s). That traded away the RAM tier's burst absorption and GET-cache role: bursts hit the disk-inflight gate (kCacheFull) and cold GETs served from disk until promotion. Restore write-back as the default: CacheDirectForKey admits PUT into the RAM arena first (async flusher drains to disk); only genuine arena backpressure falls through to direct disk write. GET serves zero-copy from the arena. Write-around stays available via --ram-write-mode=writearound (env DFKV_RAM_WRITE_MODE) for throughput-first deployments. Verified on 0064 (B200, RDMA v2, 8GB RAM tier, 32GB cap, 64KB slab granularity, credits=256, t32/b8, 30000 x 64KB): - write-back: PUT 0 fails (burst absorbed), GET 0 fails with ram_hit=30000/30000 (100% arena), flushed=30033, bypass=0, evict=0 - writearound: PUT 0 fails, GET 0 fails (disk reads), arena untouched (note: read-promotion only runs on the URING/coalescer path; a UREAD=OFF server's sync-pread GET path never fills the arena — the promotion gap is pre-existing from ed4d72b and out of scope here)
yuhaijun999
pushed a commit
to yuhaijun999/DingoCache
that referenced
this pull request
Aug 7, 2026
…, mode typo guard Review of v2.6.2 (dingodb#269/dingodb#270) found: 1. SteadyUs() returned steady_clock::count() raw ticks, which are NANOSECONDS on Linux. kEvictIdleMinUs=2,000,000 was therefore 2 ms, not 2 s: under normal traffic nearly every pooled connection counts as idle, so a full segment could evict recently-active connections and trigger reconnection storms. Now duration_cast<microseconds>. 2. Eviction woke a live_eps_ pointer whose owning Serve thread then exited and destroyed the stack endpoint — a concurrent evictor could Wake a freed endpoint (UAF). The evictor now erases the victim from live_eps_ under conn_mu_ before Wake (Serve's own erase becomes a no-op), so exactly one evictor ever touches an endpoint. 3. Total eviction wait is bounded to 5 s (was up to 32 s, exceeding the client's 10 s bootstrap). 4. last_active_us_ == 0 (inserted, not yet stamped by Serve) is never treated as idle. 5. DFKV_RAM_WRITE_MODE accepts only writeback|writearound; anything else warns and falls back to writeback (was silent writeback on typos). Verified on 0064 (64MB segment = 15 conns, t16/b1 rounds): A all-active refusal 26 fails (expected), B-F 1-2 fails each (eviction boundary), 75 evictions, zero crashes/segfaults.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
PR#246 (ed4d72b) switched PUT to write-around (direct-to-disk + GET read-promotion) to remove the per-PUT 1MiB arena memcpy (44% serve CPU, PUT 3.42→4.54 GB/s). That traded away the RAM tier's burst absorption and GET-cache role: bursts hit the disk-inflight gate (kCacheFull → PUT failure) and cold GETs served from disk until promotion.
Change
CacheDirectForKeyadmits PUT into the RAM arena first (ram_->PutCommitted), the async flusher drains to disk; only genuine arena backpressure (kCacheFull) falls through to direct disk write (existingput_busy_limit_gate preserved for the disk path).--ram-write-mode=writearound(envDFKV_RAM_WRITE_MODE; default writeback) — flag→env facade follows the existing pattern.Verification (0064 B200, RDMA v2, 8GB RAM tier, 32GB cap, 64KB slab granularity, credits=256, t32/b8, 30000 × 64KB)
Known limitation (pre-existing, out of scope)
Read-promotion (
PutDurable) runs on the URING/coalescer async paths only. A UREAD=OFF server's sync-pread GET path never fills the arena in write-around mode — this gap came from ed4d72b and is unchanged here; write-back (the default) fills the arena on every PUT regardless of build flags.