v1.2.0 — Redis Layer-2: Backfill + Write Shadowing

NEDB v1.2.0 — Redis Layer-2: Backfill + Write Shadowing

NEDB can now wrap any existing Redis connection and retroactively absorb all historical data plus auto-chain every future write into its hash-verified, time-traveling log — in three lines.

---

What's new

CollectionMapping — teach NEDB your key structure

Map Redis key glob patterns to NEDB collections with a custom id extractor and value parser.

register() — chainable collection registration

(r.nedb
 .register("driver:*", collection="driver", value_parser=json.loads)
 .register("trip:*",   collection="trip",   value_type="hash")
)

backfill() — one-time import of all existing Redis data

Scans Alice's existing Redis keys via SCAN and imports them into NEDB's hash chain in one pass. Returns the number of keys imported. Evidence is tagged "backfill" on every imported record.

imported = r.nedb.backfill()   # → 6  (scanned and imported)

shadow_writes = True — all future surface-1 writes auto-chain

Once enabled, every r.set(), r.hset(), r.setex(), etc. is silently mirrored into NEDB alongside the normal Redis write. Alice's app code changes zero lines.

r.nedb.shadow_writes = True

r.set("driver:d1", json.dumps({"name": "Bob", "status": "active"}))
# goes to Redis AND into NEDB's hash chain automatically

---

Full three-step migration

import redis, json
from nedb import wrap_redis

# ONE LINE — Alice's app doesn't change
r = wrap_redis(redis.Redis("localhost", 6379), db_name="rideshare")

# Step 1 — register key patterns as NEDB collections
(r.nedb
 .register("driver:*", collection="driver", value_parser=json.loads)
 .register("trip:*",   collection="trip",   value_type="hash")
)

# Step 2 — backfill all existing Redis data into NEDB (one-time)
imported = r.nedb.backfill()
print(f"Imported {imported} existing keys")

# Step 3 — shadow all future surface-1 writes
r.nedb.shadow_writes = True

# Alice's existing app — zero changes
r.set("driver:d1", json.dumps({"name": "Bob", "status": "active", "lat": 37.77}))
r.hset("trip:t1", mapping={"status": "en_route", "driver_id": "d1"})

# New NEDB features on the same connection
active = r.nedb.query('FROM driver WHERE status = "active" ORDER BY lat ASC')

# Time-travel
snap = r.nedb.seq
r.set("driver:d1", json.dumps({"name": "Bob", "status": "offline"}))
past = r.nedb.get_as_of("driver", "d1", snap)   # {"status": "active", ...}

# Causal provenance
r.nedb.put("dispatch", "e1",
    {"trip_id": "t1", "driver_id": "d1", "algo": "nearest"},
    caused_by=[r.nedb.seq - 1],
    evidence="inference",
    confidence=0.97)

r.nedb.query('FROM dispatch WHERE _id = "e1" TRACE caused_by')

# Tamper evidence
r.nedb.verify()   # True
r.nedb.head()     # 64-char BLAKE2b commitment hash

---

Isolation guarantee

NEDB never writes to Alice's namespace. It owns only:

Key	Type	Purpose
nedb:{db_name}:oplog	Redis Stream	append-only op log
nedb:{db_name}:snapshot	Redis Hash	checkpoint
nedb:{db_name}:meta	Redis Hash	index config

---

Local demo (no Redis server needed)

pip install nedb-engine fakeredis
git clone https://github.com/Eth-Interchained/nedb
cd nedb
python3 examples/fakeredis_demo.py
# 29/29 checks passed

---

Test coverage

• 74/74 tests passing (python3 tests/test_wrap_redis.py)
• 30 new tests covering: backfill, write shadowing, hset merge, time-travel through shadows, full pipeline + restart

---

Install

pip install nedb-engine==1.2.0
npm install nedb-engine@1.2.0

---

Changelog

Version	Highlight
1.2.0	wrap_redis() backfill + write shadowing — register(), backfill(), shadow_writes
1.1.0	wrap_redis() — NEDB as a Redis layer-2, dual surface
1.0.5	License → GPL-3.0-or-later, npm homepage fix
1.0.3	Rust NQL integer/float comparison fix
1.0.2	Rust AS OF index bypass fix
1.0.1	napi-rs binding rewrite

---

Built by INTERCHAINED LLC × Claude Sonnet 4.6
Docs: nedb.aiassist.net

nedb-engine v0.9.0 — Causal Write Provenance

nedb-engine v0.9.0 — Causal Write Provenance

The first embedded database with cryptographically-sealed causal chains.
Every write can now declare why it happened, what caused it, and how confident
the writer was — sealed inside the hash chain at write time, tamper-evident,
time-travelable, and queryable in both directions.

---

Install / Upgrade

pip install --upgrade nedb-engine

Supports Python ≥ 3.8. Native Rust-core wheels ship for Linux (x86_64 manylinux),
macOS (arm64), and Windows (x86_64). All platforms fall back to the pure-Python
reference engine automatically.

---

What's New in v0.9.0 — Causal Write Provenance

The feature

AI agents write data constantly. Until now, no embedded database tracked why a
write happened — which inputs triggered it, what it was inferred from, or how
certain the writer was. That meant auditing an agent's reasoning required
reconstructing causality from application logs, which are untrustworthy and
inevitably stale.

NEDB v0.9.0 makes causality a first-class storage primitive. Three optional
fields on every put() call:

Field	Type	Meaning
caused_by	List[int]	Seqs of the ops that caused this write
evidence	str	Source type: user_message · inference · tool_result · correction · external
confidence	float	Agent certainty, 0.0–1.0

These are sealed inside the BLAKE2b hash chain at write time — if you change
them after the fact, verify() fails. They are also mirrored into the document
as queryable _caused_by, _evidence, _confidence fields so they work with
any WHERE clause.

API

from nedb import NEDB

db = NEDB("./agent-memory")

# Raw inputs — uncaused roots
db.put("inputs", "turn_1", {"role": "user", "text": "I hate bright screens"})
db.put("inputs", "turn_2", {"role": "user", "text": "I have migraines"})
seq_1, seq_2 = db.seq - 1, db.seq

# Derived belief, sealed in the chain
db.put("beliefs", "dark_mode_pref",
    {"value": True, "summary": "User prefers dark mode"},
    caused_by=[seq_1, seq_2],
    evidence="user_message",
    confidence=0.95)

# Second-order inference
db.put("beliefs", "low_blue_light",
    {"value": True},
    caused_by=[db.seq],       # caused by dark_mode_pref
    evidence="inference",
    confidence=0.82)

# Provenance is queryable via normal WHERE
db.query('FROM beliefs WHERE _evidence = "user_message"')
db.query('FROM beliefs WHERE _confidence > 0.9')

NQL: TRACE operator

FROM beliefs WHERE _id = "dark_mode_pref" TRACE caused_by

Backward traversal — recursively follows caused_by seqs to their originating
documents. Answers: "Why does the agent believe this?"

FROM inputs WHERE _id = "turn_1" TRACE caused_by REVERSE

Forward traversal — uses the in-memory reverse index to find all documents that
declared this op as a cause. Answers: "What did this input cause downstream?"

Why this matters

• EU AI Act (full applicability: August 2026): Article 13 requires operators of
  high-risk AI systems to produce records of "the logic involved" in decisions.
  Causal provenance at the storage layer makes this verifiable, not self-reported.
• OWASP Agentic Top 10 (ASI06 — Memory & Context Poisoning): A tamper-evident
  causal chain means you can detect injected beliefs — they either lack provenance
  or break verify().
• Operational auditing: "Why does the agent recommend X?" is now a database
  query, not a forensic reconstruction exercise.

Engram and Operad build causal provenance at the application layer over PostgreSQL
and Neo4j. NEDB builds it at the storage layer — sealed in the same hash chain
that already proves tamper-evidence and time-travel.

Backward compatibility

Fully backward-compatible. Existing databases and AOF files verify without
modification. Old ops without provenance omit the fields from their hash body —
mixed chains (some ops with provenance, some without) verify correctly. No
migration required.

---

What Changed Since v0.7.4

v0.8.3 — Deploy fix (encrypted new databases)

Bug: Creating a new database while NEDB_TMK (encryption) was set failed with
FileNotFoundError: ./nedb-data/<name>/key.enc.tmp because load_or_create_dek()
ran before _open() created the directory.

Fix: os.makedirs(path, exist_ok=True) immediately before the DEK call.

This was the root cause of every "Deploy failed (502)" in the studio when the
daemon was running with at-rest encryption enabled.

v0.8.2 — Structured logging + deploy integration test

• NEDBD_DEBUG=1 / nedbd --log-level N (0=errors only, 1=requests, 2=deploy phases, 3=verbose)
• Full traceback always printed on unhandled exceptions (never swallowed silently)
• tests/test_deploy.py — integration test for the full scaffold deploy path
  (the test that would have caught the v0.8.3 bug before production)

v0.8.0 — Concurrent daemon (group-commit sequencer)

ThreadingHTTPServer was thread-unsafe: concurrent writes to one database raced
the hash chain, causing 500s surfaced as the studio's 502. The fix:

• Single-writer per database: writers enqueue intents, one committer thread
  owns all mutation — correct chain by construction, zero write locks.
• Group commit: the committer drains the whole queue, applies every op,
  then issues one fsync per batch. More concurrent writers → bigger batches →
  higher throughput. ~15,000 writes/s under load.
• Lock-free MVCC reads: reads run at the last committed seq (snapshot
  isolation); they never touch the write queue or take a lock.

v0.7.6 — Self-healing chains

The encryption backfill (plaintext DB opened with NEDB_TMK) appended a
checkpoint op that was never persisted — creating a permanent gap in the chain.
Every subsequent open returned verify() = False, showing the "tampered" pill
in the studio even though nothing was tampered.

• Backfill no longer checkpoints — it drops the stale snapshot and rewrites
  the AOF cleanly.
• _self_heal_if_needed() on open: if verify() fails but every op is
  internally consistent (only the linkage broke, content is intact), the chain
  is re-linked in place and the AOF rewritten. Genuine content tampering is left
  False with a warning — real attacks are never masked.

v0.7.5 — MongoDB compatibility adapter

nedb.mongo.MongoCompat (MongoClient alias) — the third compatibility layer
alongside SQL and Redis adapters. Full document/collection API:

from nedb import NEDB, MongoClient

db = NEDB()
users = MongoClient(db)["users"]
users.insert_many([{"name": "Alice", "age": 31}, {"name": "Bob", "age": 24}])
list(users.find({"age": {"$gt": 25}}).sort("age", -1))
users.update_one({"name": "Alice"}, {"$inc": {"logins": 1}})
users.aggregate([{"$group": {"_id": None, "avg": {"$avg": "$age"}}}])

Supports: find findOne count distinct aggregate insertOne insertMany
updateOne updateMany deleteOne deleteMany replaceOne · Query operators:
$eq $ne $gt $gte $lt $lte $in $nin $exists $regex $and $or $nor $not $size $all $mod $elemMatch · Update operators: $set $unset $inc $mul $min $max $rename $push $addToSet $pull $pop $setOnInsert · Aggregation: $match $group $sort $skip $limit $count $project

Also exposed over nedbd: POST /v1/databases/:name/mongo

---

Full Changelog (v0.7.4 → v0.9.0)

Version	Date	Summary
v0.7.4	2026-06-14	Fix maturin native wheel — stage Python source into crate so the built wheel contains the full package, not just _native.so
v0.7.5	2026-06-14	MongoDB compatibility adapter (MongoCompat/MongoClient)
v0.7.6	2026-06-14	Self-healing chains; encrypt-backfill gap fix
v0.8.0	2026-06-14	Concurrent daemon — single-writer group-commit sequencer
v0.8.1	2026-06-14	MongoDB nedbd endpoint (POST /v1/databases/:name/mongo)
v0.8.2	2026-06-14	Structured logging (--log-level); deploy integration test
v0.8.3	2026-06-14	Fix encrypted new-DB deploy (makedirs before DEK creation)
v0.9.0	2026-06-15	Causal Write Provenance — caused_by, evidence, confidence, TRACE NQL

---

Architecture Summary

nedb-engine
├── Hash-chained append-only OpLog     (tamper-evident, replay-protected)
│   └── Causal provenance v0.9.0      (caused_by / evidence / confidence)
├── MVCC store                         (time-travel AS OF seq)
├── Relations + adjacency index        (TRAVERSE, link/unlink)
├── Eq / Ordered / Search indexes
├── Concurrent Sequencer               (group-commit, lock-free reads)
├── AES-256-GCM at-rest encryption    (TMK / DEK double-envelope)
├── AOF durable persistence + snapshots
└── Compatibility adapters
    ├── SQL    (SELECT/INSERT/UPDATE/DELETE)
    ├── Redis  (GET/HSET/SADD/LPUSH/…)
    └── MongoDB (find/aggregate/update/…)

---

Links

• PyPI: https://pypi.org/project/nedb-engine/0.9.0/
• GitHub: https://github.com/Eth-Interchained/nedb
• Studio: https://studio.interchained.org
• INTERCHAINED: https://interchained.org · https://aiassist.net

---

Built by INTERCHAINED LLC × Claude Sonnet 4.6
Apache-2.0 (engine) · GPLv3 (studio)

nedb-engine v0.6.0 — RESP2 + Encryption + Snapshots

nedb-engine v0.6.0

The headline release. nedbd now speaks the Redis wire protocol natively — redis-cli, redis-benchmark, and every Redis client library connects without modification. Combined with AES-256-GCM at-rest encryption, Redis-style AOF persistence with snapshot checkpoints, and SQL/Redis/Python adapters, NEDB is a time-traveling, verifiable, encrypted Redis replacement.

pip install nedb-engine

# Start the server
NEDB_TMK=$(python3 -c "import os; print(os.urandom(32).hex())") \
NEDBD_RESP2_PORT=6379 nedbd

# Connect with redis-cli
redis-cli -p 6379 PING
redis-cli -p 6379 SELECT mydb
redis-cli -p 6379 HSET user:1 name Ada age 31
redis-cli -p 6379 EVAL "FROM users LIMIT 10" 0

---

What's new in v0.6.0

RESP2 Wire Protocol (NEDBD_RESP2_PORT)

Nedbd now runs a second TCP server speaking RESP2 (Redis Serialization Protocol v2). Every Redis client in every language connects natively.

• All major command groups: SET/GET/DEL/EXISTS/INCR, HSET/HGET/HGETALL, SADD/SMEMBERS/SCARD, LPUSH/RPUSH/LRANGE, KEYS/TYPE/DBSIZE/FLUSHDB
• SELECT <name> — switches to a named NEDB database (maps Redis's integer DBs to NEDB's named databases)
• EVAL "<NQL>" 0 — NQL pass-through: run any NQL query and get rows back as JSON strings
• Unsupported commands return clear -ERR with a roadmap note (EXPIRE/TTL, SUBSCRIBE/PUBLISH, MULTI/EXEC)

AES-256-GCM Encryption at Rest (NEDB_TMK)

Double-envelope key architecture: a random per-database DEK is wrapped by an external TMK.

• Encrypts the AOF, snapshot.json, and BlobStore chunks
• Auto backfill-encrypt: enabling NEDB_TMK on an existing unencrypted database rewrites the entire AOF encrypted in place (atomic) on first open
• Key rotation via db.rewrap_key() — re-wraps the DEK without touching data
• pip install nedb-engine now includes cryptography as a required dependency — encryption just works

---

Full changelog since v0.4.2

Version	Feature
v0.5.0	Snapshot checkpoints (db.checkpoint()), TTL/expiry (ttl_s=, expire(), sweep()), GROUP BY aggregations (COUNT/SUM/AVG/MIN/MAX)
v0.5.1	nedbd auto-checkpoints all databases on SIGTERM/SIGINT — synchronized restart
v0.5.2	BlobStore (Cascade files) persisted in snapshots — get_file(), Merkle proofs survive restart
v0.5.3	AES-256-GCM encryption at rest — AOF, snapshots, blob chunks; NEDB_TMK env
v0.5.4	nedbd Manager wires NEDB_TMK — all databases encrypted when env is set
v0.5.5	cryptography bundled as required dep — encryption just works
v0.5.6	Auto backfill-encrypt existing plaintext AOF on first encrypted open
v0.5.7	Eager-open all databases at startup — backfill visible in boot log
v0.6.0	RESP2 wire protocol — redis-cli, redis-benchmark, any Redis client

---

Full feature set

• Hash-chained append-only log — nonce-enforced replay protection, idempotency, MVCC time-travel (AS OF seq), verify() integrity
• NQL — the NEDB Query Language: FROM t [AS OF n] [WHERE ...] [SEARCH "text"] [ORDER BY f] [TRAVERSE rel] [LIMIT n] [GROUP BY f COUNT|SUM|AVG|MIN|MAX]
• Relations — first-class graph edges with O(1) traversal, time-travel aware
• Indexes — eq (hash), ordered (bisect), full-text inverted — maintained incrementally
• Cascade compression — content-defined chunking + dedup + zlib/LZMA tiers, Merkle proofs per file version
• SQL adapter — sql_exec(db, sql) translates SELECT/INSERT/UPDATE/DELETE to NQL/NEDB
• Redis adapter — RedisCompat(db).execute(cmd, *args) — 30+ commands
• Auto-indexing — AutoIndexDB(db, threshold=5) creates indexes from query patterns
• nedbd — HTTP/JSON server daemon + RESP2 wire protocol; multi-database, bearer auth, CORS
• Benchmark suite — bench/benchmarks.py with reproducible baseline in bench/RESULTS.md

Install

pip install nedb-engine          # v0.6.0, universal wheel, all platforms
pip install --upgrade nedb-engine  # upgrade from any earlier version

Requirements

• Python 3.8+
• cryptography>=41 (bundled — no separate install needed)

License

Apache-2.0 · © INTERCHAINED, LLC — interchained.org · powered by AiAssist