Description
Currently ApexStore runs compaction in a single background thread (maybe_compact). Multiple column families compact sequentially. Competitive engines support concurrent compaction across column families.
Impact
- With 8+ column families, compaction becomes a bottleneck
- Write stalls while waiting for single-thread compaction to finish
- CPU cores are underutilized during compaction I/O
Proposed Implementation
- Change
compaction_thread from Mutex<Option<JoinHandle>> to Mutex<Vec<JoinHandle>>
- Allow up to
N concurrent compactions (configurable via compaction.max_concurrent)
- Assign one compaction per column family
- Use a semaphore to limit total concurrent compactions
- Ensure
close() waits for all compaction threads to finish
Labels
Description
Currently ApexStore runs compaction in a single background thread (
maybe_compact). Multiple column families compact sequentially. Competitive engines support concurrent compaction across column families.Impact
Proposed Implementation
compaction_threadfromMutex<Option<JoinHandle>>toMutex<Vec<JoinHandle>>Nconcurrent compactions (configurable viacompaction.max_concurrent)close()waits for all compaction threads to finishLabels