Skip to content

Release v0.1.0

Choose a tag to compare

View all tags
@nickooan nickooan released this 11 Jul 11:46
v0.1.0
00be0a7

ntee-db v0.1.0

The first minor release: ntee-db grows from an embedded library into a small database system — a standalone server, online compaction, and blob-space reclamation — while keeping every hot-path benchmark unchanged.

🚀 Highlights

New: nteedb-server — run ntee-db as a standalone daemon. A redis/memcached-style TCP server over the core: memcached-inspired text protocol, every response a single line of pure JSON, pipelining, one goroutine per connection with truly parallel reads. Pure Go, cross-compiles to a ~3 MB static binary via the new root ./build.sh — scp and run.

Online compaction — reads never stop. Compact/Reindex used to hold the exclusive lock for their whole rewrite; reads now flow throughout, with writes pending only until the rebuild finishes. Reads arriving mid-compaction went from ~4.4 ms to 1.25 µs (~3,500× faster) on the contention benchmark.

Blob-space reclamation. Orphaned blobs (overwritten/deleted large values) used to accumulate forever. New core APIs split mechanism from policy: BlobsRelieve() rewrites blobs into a fresh generation file and drops orphans (crash-safe — the main-log rename stays the single atomic commit point), and BlobUsage() measures total/live/orphaned bytes so callers decide when to act.

Auto-compaction with user-tunable policy. The server watches dead space and reclaims it in the background — main log and blobs as independent triggers — fully configurable in schema.json ("autoCompact": true or an options object: intervalSeconds, mainRatio, mainMinBytes, blobsRelieve on/off, blobRatio, blobMinRelieveDataSize). Off by default.

Server features

  • Protocol: get/getm/has/scan, index queries (ix, ixh, ixp, ixr, ixrec), writes (put inline or length-prefixed binary-safe frames, putx with explicit index values, del, rml/rmg range deletes), plus stats, dropped, prospective.
  • Auth, modeled on redis/memcached: no-auth (loopback-only via protected mode), single password (NTEEDB_AUTH), or user:password:role file. Two roles — admin alone may run compact, reindex, relieve. Pre-auth allowlist matches redis (auth, hello, quit; ping requires auth; hello hides the index schema until authenticated).
  • Ops: stats reports liveBytes (dead-space ratio at a glance), autoCompacts, blobCompacts; graceful shutdown writes the boot hint; clear startup errors (locked store, protected mode).
  • Startup: -schema schema.json (same shape as the JS open options, with jsonPath-derived indexes), -addr, -auth/-auth-file, -idle.

Core improvements

  • Compaction gate: Compact/Reindex/BlobsRelieve exclude only writers during the long rebuild; the exclusive lock is held just for the atomic file swap.
  • Blob generations: each blob ref names its file generation (blobs.dat, blobs.N.dat) — self-describing, crash-consolidating, byte-compatible with existing stores.
  • New APIs: LiveBytes(), BlobUsage(), BlobsRelieve().
  • No regressions: put/get/scan/batch/boot benchmarks statistically unchanged vs 0.0.x; Open gains <1 ms (blob-generation discovery); compaction remains ~4 µs per live record (68 ms @ 10k records, measured M2 Pro).

Compatibility notes

  • Existing stores open unchanged; all on-disk formats are backward compatible.
  • After the first BlobsRelieve on a store, its blob file moves to a new generation — binaries older than v0.1.0 won't resolve those refs. Don't downgrade after relieving (or run stores side-by-side across versions).
  • The JS binding (ntee-db on npm) embeds the core directly and is unaffected API-wise; its prebuilt binaries will pick up the online-compaction engine in the next npm release.
    `
Assets 5
Loading