Skip to content

v2.0 – Snapshots (checkpointing) and WAL truncation

Latest

Choose a tag to compare

@Subham1100 Subham1100 released this 24 Jan 17:18
· 48 commits to main since this release

Cognex

Cognex is a single-node, persistent key–value database built incrementally to understand real database internals such as durability, crash recovery, and storage invariants.

The project is developed version by version, with each version freezing a clear set of guarantees before moving forward.
Cognex prioritizes correctness and clarity over performance.


Project Goals

  • Learn how real databases are built internally
  • Implement correctness before optimization
  • Evolve features incrementally with explicit guarantees
  • Keep the system simple, inspectable, and auditable

Version Overview

Version Description
v0 In-memory key–value store
v1 Write-Ahead Log (WAL) with crash recovery
v2 Snapshots (checkpointing) and WAL truncation
v3 (planned) Concurrent reads and writes

Cognex v2 — Snapshots & WAL Truncation

Cognex v2 extends the durability guarantees of v1 by introducing snapshots (checkpoints).
This makes recovery time and disk usage bounded, turning Cognex into a practical single-node storage engine.


Why v2 Exists

In v1, all writes were persisted using a Write-Ahead Log (WAL).
While correct, this approach had inherent limitations:

  • WAL size grew indefinitely
  • Startup time increased linearly with WAL length

These are expected constraints of a WAL-only system.

v2 solves this by introducing snapshotting.


Key Features in v2

Snapshots (Checkpointing)

Cognex periodically writes the full in-memory state to disk as a snapshot.

  • Represents a fully materialized database state
  • Independent of WAL history
  • Stored in a simple, human-readable format

Example snapshot: