Skip to content

[PERF] Memory-mapped SSTable reads — zero-copy I/O via mmap for cold data #203

@ElioNeto

Description

@ElioNeto

Description

ApexStore currently reads SSTable data through SstableReader with explicit read() syscalls. MMAP allows the OS to manage disk-to-memory paging transparently, reducing syscall overhead and enabling better cache utilization.

Proposed Implementation

  1. Add mmap_read: bool to StorageConfig (default: false for compatibility)
  2. When enabled, open SSTable files with memmap2 crate
  3. Serve read requests directly from the mmap region (zero-copy)
  4. Use MADV_RANDOM or MADV_SEQUENTIAL advice depending on access pattern
  5. Fall back to pread() for blocks that fail to mmap (e.g., file too large on 32-bit)

Expected Impact

  • 20-40% reduction in cold read latency (fewer syscalls)
  • Reduced memory pressure (OS manages page cache vs application buffer)
  • Faster database restarts (no need to pre-warm cache)

Labels

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions