Skip to content

fippli/notedb

Repository files navigation

notedb

A Datomic- and SQLite-inspired embedded document database. Immutable, append-only, file-backed. Rust core, JavaScript API.

  • 🗃️ Document store — JSON documents in named collections, no schema.
  • 📜 Append-only log — every write is a fact appended to one file. The file is the history.
  • Time travel — read any note (or list a whole collection) as it was at a past point in time.
  • 🧱 Crash safe — a torn write from a crash is discarded on reopen. No corruption, no recovery step.
  • 🦀 Native speed — Rust core via napi-rs, exposed as a plain JS module.
  • 📦 Zero infrastructure — no server, no daemon. One file is one database; copy it anywhere.

It fills the gap SQLite leaves for document data: SQLite is relational-first, MongoDB needs a running server, and the JS-only embedded stores are abandoned.

Install

npm install @fippli/notedb

Quick start

import notedb from "@fippli/notedb"

const db = notedb.use("/data/db.log")

const cars = db.collections.add("cars")

// remember(doc) inserts (id generated); remember(id, doc) updates
const note = cars.remember({ make: "Volvo", year: 1992 })
cars.remember(note.id, { make: "Volvo", year: 1993 })

cars.read(note.id)              // { id, make: "Volvo", year: 1993 }
cars.list()                     // [{ id, ... }, ...]
cars.find((d) => d.year > 1990) // filter with a predicate
cars.map((d) => d.year)         // map over notes

// time travel
cars.read(note.id, { at: 1 })   // the note as it was at logical time t=1

// full version history of a note
cars.history(note.id)
// [ { t, op: "remember", doc }, ... ]

cars.forget(note.id)            // tombstone (stays in history)

Concepts

  • The database is a value, not a mutable blob. Writes never overwrite; they append.
  • Two primitives: remember (insert/update) and forget (tombstone).
  • Logical time t is a monotonic counter stamped on every write — that's what { at: t } refers to.
  • Single writer. Designed for one process (e.g. one container, one backend). No concurrent-writer locking.

See SDK.md for the full API specification.

API

Operation Description
notedb.use(path) Open (or create) a database; replays the log into memory
db.collections.add(name) Create a collection, return its handle
db.collections.use(name) Get a collection handle
db.collections.drop(name) Drop a collection (history preserved)
db.collections.list() List collection names
col.remember(doc) Insert a new note (id generated)
col.remember(id, doc) Update an existing note
col.forget(id) Tombstone a note
col.read(id, [opts]) Read a note (opts.at for time travel)
col.list([opts]) List notes (opts.at for time travel)
col.find(fn, [opts]) Filter notes with a predicate
col.map(fn) Map a function over notes
col.history(id) Full chronological version history of a note
db.compact([opts]) Compact the log to a snapshot (opts.archive to keep the old log)

Log format

The database is newline-delimited JSON — one fact per line, human readable:

{ "t": 1, "op": "remember", "collection": "cars", "id": "abc", "doc": { "make": "Volvo", "year": 1992 } }
{ "t": 2, "op": "remember", "collection": "cars", "id": "abc", "doc": { "make": "Volvo", "year": 1993 } }
{ "t": 3, "op": "forget",   "collection": "cars", "id": "abc" }

(Collection lifecycle uses two additional structural ops, mkcol/dropcol, so empty collections and drops survive a reopen.)

Compaction

The log grows with every write and the full history is held in memory. When it gets large, compact it to a snapshot of current state:

db.compact()                                  // discard prior history
db.compact({ archive: "/data/db.archive.log" }) // keep the old log

Building from source

npm install
npm run build   # compiles the Rust core to a native .node addon
npm test

Status & limitations

notedb is young. Known constraints to be aware of:

  • Single process / single writer — opening the same file from two processes can corrupt it.
  • Synchronous, fsync-per-write — durable, but writes block; best for moderate write rates.
  • In-memory history — memory and the log grow until you compact().

License

MIT © Filip Johansson

About

A Datomic and SQLite inspired embedded document database. Immutable, append-only, file-backed.

Resources

License

Stars

0 stars

Watchers

0 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors