Skip to content

Release v0.2.0

Choose a tag to compare

@nickooan nickooan released this 12 Jul 04:00
e195a17

ntee-db v0.2.0

Counters land: ntee-db gains its first in-place write path — atomic int64 incr/decr that never grow the log — plus a stricter indexing model: secondary indexes now cover JSON-object values only; immediate values are primary-key-only.


🚀 Highlights

New: atomic int64 counters — Incr in core, incr/decr on the server. Redis-style semantics: a missing key initializes to 0 before the delta applies, the post-operation value comes back in the response, delta defaults to 1 (incr hits, decr hits 5, negative deltas work). Counters are a distinct value type — incr on any other value fails with ErrNotCounter, overflow past int64 fails with ErrCounterOverflow leaving the value untouched, and a put demotes a counter to a plain value. Int64 only, by design: no floats (store scaled integers — cents, not dollars).

Zero log growth for hot counters — the first in-place update in the engine. Counter values are stored fixed-width (explicit sign + 19 zero-padded digits — every int64 has exactly one such form), so an increment overwrites the digits in place at the same file offset through a new dedicated O_RDWR handle. No append per update, no compaction debt: in the benchmark run, 440,000 incr/decr ops left exactly one 48-byte record in main.jsonl. Torn-write safe by construction — a crash mid-patch can only mix old/new digit characters, never corrupt the line, so recovery is never tricked into truncating records after it.

Indexes now cover JSON-object values only. Immediate values — strings, numbers, booleans, arrays, binary — are plain key:value pairs addressed by primary key alone: Extract functions are never executed over them, and PutIndexed/putx reject explicit index values for one (an error; nothing written). One consistent model: an index entry always points at a structured record. This is also what makes counter patches unconditionally safe — nothing can derive index state from an immediate value, so increments take the in-place fast path even on schemas with jsonPath indexes.


Server features

  • New commands: incr <pk> [delta] and decr <pk> [delta] — write commands (serialized, auth like put), response is the new value as a JSON number: {"ok":true,"result":42}. incr <pk> 0 is the read idiom (plain get returns the raw 20-char fixed-width string).
  • Argument safety: non-integer deltas (1.5, abc) rejected with a usage error before reaching the store; decr of -9223372036854775808 rejected (negation overflow).
  • Measured (M2 Pro, loopback, single connection, schema with two jsonPath indexes): incr ~122k ops/s pipelined (8.2 µs/op) / ~32k ops/s sequential; get ~142k ops/s pipelined; put ~85k ops/s pipelined. Counters beat puts on the wire — smaller records, no index derivation.

Core improvements

  • New APIs: Incr(key, delta int64) (int64, error) (decrement = negative delta), errors ErrNotCounter / ErrCounterOverflow; new benchmarks BenchmarkIncr / BenchmarkIncrDurable.
  • Record format: new "c":true flag marks counter records explicitly — a value that merely looks like the fixed-width format is never treated as a counter.
  • Fail-safe fallback: if a counter record's on-disk layout doesn't match expectations byte-for-byte, the update falls back to a normal append — correctness over speed, always.
  • Handle hygiene: the three main.jsonl fds (append log / shared reader / counter patcher) are now opened and closed as one unit (openMainHandles), used identically by Open and the compaction swap; each handle keeps least privilege (the append fd physically cannot overwrite existing records, the reader cannot write at all).
  • No regressions: get untouched (1.15 µs); put unchanged (2.4 µs — object values pay one extra byte-check); core Incr 2.7 µs in-place; with SyncEveryWrite every write type shares the same ~ms fsync floor.

Compatibility notes

  • Behavior change: PutIndexed/putx/PutBatch with explicit index values on a non-object value now fail (previously allowed). Wrap such payloads in an object envelope if they must be indexed. Extract functions likewise no longer run over non-object values.
  • Enforcement is write-time only: existing stores open unchanged, legacy records indexed under the old rules still load and are preserved by Compact; they age out as records are rewritten.
  • Binaries older than v0.2.0 read counter records as plain 20-char strings (unknown "c" field ignored); an old binary's Compact would drop the counter flag — don't downgrade after using counters.
  • The JS binding (ntee-db on npm) is API-unaffected; exposing Incr through the FFI binding is planned as a follow-up.
    `