Skip to content

fix: prevent blocking operations from freezing shard threads - #62

Merged
kacy merged 5 commits into
mainfrom
feat/lazy-free-unlink
Feb 8, 2026
Merged

fix: prevent blocking operations from freezing shard threads#62
kacy merged 5 commits into
mainfrom
feat/lazy-free-unlink

Conversation

@kacy

@kacy kacy commented Feb 8, 2026

Copy link
Copy Markdown
Owner

summary

  • adds a dedicated background OS thread (ember-drop) for deferring expensive value deallocations off the shard hot path
  • introduces UNLINK command — semantically identical to DEL but always defers the destructor to the background thread
  • extends FLUSHDB to accept an optional ASYNC argument that swaps the entries map instantly and defers bulk deallocation
  • existing DEL, eviction (try_evict), and lazy expiration (remove_if_expired) now automatically defer large value drops when the drop handle is present (values with >64 collection elements)
  • adds a warning log when KEYS is called on keyspaces with >10k keys

what was tested

  • all 720+ existing tests pass across the workspace (cargo test --workspace)
  • new unit tests for dropper (defer small/large values, defer entries, empty entries skip)
  • new unit tests for is_large_value threshold behavior (strings, lists, hashes, sets)
  • new protocol tests for UNLINK (single, multi, no args) and FLUSHDB ASYNC (with/without, case insensitive)
  • cargo clippy --workspace — zero warnings
  • cargo fmt --check — clean

design considerations

  • OS thread, not tokio task: dropping large data structures is CPU-bound work that would starve the async executor. a plain std::thread avoids this entirely.
  • bounded channel with try_send fallback: the 4096-capacity SyncSender ensures the shard never blocks waiting for the drop thread. if backpressure hits, we just drop inline — graceful degradation, not a hard failure.
  • memory tracker updated immediately: when a key is unlinked, memory usage is decremented before the value is sent to the background thread. this is correct because the keyspace no longer references the data, and it prevents the memory limit from being stale.
  • UNLINK maps to AofRecord::Del: on replay, the deallocation strategy doesn't matter — only the fact that the key was removed. reusing the existing Del record avoids adding a new record type.
  • concurrent mode ignores async_mode: DashMap doesn't have the same blocking issue as shard channels, so both DEL and UNLINK use the same keyspace.del() path. FLUSHDB ASYNC calls clear() directly since there's no drop handle in concurrent mode.

kacy added 5 commits February 8, 2026 13:51
adds a dedicated OS thread that receives large values and drops them
off the hot path. this prevents expensive destructors (large lists,
hashes, sorted sets) from blocking shard threads.

- dropper.rs: bounded channel (4096 capacity) with try_send fallback
- memory.rs: is_large_value() helper with 64-element threshold
- strings always drop inline (Bytes is O(1) ref-counted)
integrates the background dropper into keyspace operations:

- unlink(): like del but always defers the destructor
- flush_async(): swaps entries map and returns old entries for deferred drop
- del/try_evict/remove_if_expired: defer large value drops when handle is set
- keys(): warn when scanning >10k keys (suggest SCAN instead)
- adds Unlink and FlushDbAsync shard request variants
- spawn_shard now accepts an optional DropHandle for lazy free
- engine creates a shared DropHandle and passes it to all shards
- FlushDbAsync handled in the main loop: swaps entries, defers drop
- Unlink maps to AofRecord::Del for replay (same semantics)
- Command::Unlink: parses like DEL, accepts one or more keys
- Command::FlushDb: now has async_mode field, accepts optional ASYNC arg
- parse_unlink mirrors parse_del exactly
- parse_flushdb updated to accept optional ASYNC argument
- sharded mode: UNLINK routes through multi_key_bool like DEL
- sharded mode: FLUSHDB broadcasts FlushDb or FlushDbAsync based on flag
- concurrent mode: UNLINK handled same as DEL (DashMap has no blocking issue)
- concurrent mode: FlushDb pattern updated for async_mode field
@kacy
kacy merged commit c78ce15 into main Feb 8, 2026
5 checks passed
@kacy
kacy deleted the feat/lazy-free-unlink branch February 8, 2026 18:58
kacy added a commit that referenced this pull request Feb 11, 2026
* feat: add background value dropper for lazy free

adds a dedicated OS thread that receives large values and drops them
off the hot path. this prevents expensive destructors (large lists,
hashes, sorted sets) from blocking shard threads.

- dropper.rs: bounded channel (4096 capacity) with try_send fallback
- memory.rs: is_large_value() helper with 64-element threshold
- strings always drop inline (Bytes is O(1) ref-counted)

* feat: add unlink and flush_async to keyspace with lazy free

integrates the background dropper into keyspace operations:

- unlink(): like del but always defers the destructor
- flush_async(): swaps entries map and returns old entries for deferred drop
- del/try_evict/remove_if_expired: defer large value drops when handle is set
- keys(): warn when scanning >10k keys (suggest SCAN instead)

* feat: wire up UNLINK and FLUSHDB ASYNC through shard and engine

- adds Unlink and FlushDbAsync shard request variants
- spawn_shard now accepts an optional DropHandle for lazy free
- engine creates a shared DropHandle and passes it to all shards
- FlushDbAsync handled in the main loop: swaps entries, defers drop
- Unlink maps to AofRecord::Del for replay (same semantics)

* feat: add UNLINK command parsing and FLUSHDB ASYNC support

- Command::Unlink: parses like DEL, accepts one or more keys
- Command::FlushDb: now has async_mode field, accepts optional ASYNC arg
- parse_unlink mirrors parse_del exactly
- parse_flushdb updated to accept optional ASYNC argument

* feat: handle UNLINK and FLUSHDB ASYNC in connection handlers

- sharded mode: UNLINK routes through multi_key_bool like DEL
- sharded mode: FLUSHDB broadcasts FlushDb or FlushDbAsync based on flag
- concurrent mode: UNLINK handled same as DEL (DashMap has no blocking issue)
- concurrent mode: FlushDb pattern updated for async_mode field
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant