Skip to content

[RESILIENCE] Automatic WAL archiving and truncation — prevent WAL infinite growth #224

@ElioNeto

Description

@ElioNeto

Description

After per-CF WAL (#157), each column family has its own WAL. If a CF is written heavily but never flushed, its WAL grows unbounded, eventually filling the disk.

Implementation

  1. Add WAL size limit per CF (configurable, default: 512MB)
  2. When a WAL exceeds the limit:
    • Force flush the CF's memtable
    • Archive the WAL segment to <db>/wal-archive/{cf}-{seq}.wal
    • Create a fresh WAL for the CF
  3. Archive retention policy:
    • Keep last N archive segments
    • Auto-delete archives older than T days
  4. WAL archive serves as additional crash recovery layer

Configuration

resilience:
  wal:
    max_size_per_cf_mb: 512
    archive_retention_count: 10
    archive_retention_days: 7

Labels

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions