Persistence

Persistence

PolyGraph is backed by a pluggable PersistenceAdapter. Two are bundled, and the contract is open for custom backends.

Bundled adapters

• MemoryAdapter(maxNodes?) — stores serialized records in memory. When maxNodes is set, the least-recently-put node (plus its edges and vector) is evicted once the cap is reached; writing a node again (via putNode, bulkPutNodes, or applyChanges) bumps it back to most-recently-put.
• BinaryStoreAdapter({ storeDir, compactThreshold?, fileIO?, syncWrites?, mutationLogRetention? }) — persists as a MessagePack snapshot plus an append-only write-ahead log (WAL). Nodes, edges, and vectors are committed atomically per batch; the WAL is compacted into a snapshot once it passes an adaptive threshold and on close(). compactThreshold is the minimum WAL-entry count at which compaction is scheduled (default 10,000) — the effective threshold also grows with the store (max(threshold, records / 4)), so a 1M-node build doesn't rewrite the snapshot quadratically. Startup replays the WAL, then persists a snapshot before deleting it, so a crash between those steps loses nothing; a truncated WAL tail from a mid-append crash is also tolerated.
  • syncWrites: true fsyncs WAL appends and snapshot writes (including the containing directory) for crash durability at a throughput cost.
  • mutationLogRetention: { maxEntries?, maxAgeMs? } opts the durable mutation log (mutations.msgpack) into trimming on the same compaction cadence — unset by default, which keeps the full mutation history forever. When both bounds are set, a record must satisfy both to survive. trimMutationLog() also trims on demand, independent of WAL-driven compaction. Only enable this when nothing depends on getMutationsSince/getMutationLogPage reaching back further than the retained window (e.g. replicas re-snapshot instead of tailing arbitrarily far behind). The Rust core and Python bindings expose the same policy — see Database core.

BinaryStoreAdapter lives behind platform subpaths so the core entry point stays free of node: built-ins:

• @0xx0lostcause0xx0/polypack/persistence/node — NodeFileIO (filesystem).
• @0xx0lostcause0xx0/polypack/persistence/opfs — OPFSFileIO (browser File System Access API).
• @0xx0lostcause0xx0/polypack/persistence — MemoryFileIO and the FileIO type, for tests and custom storage. When fileIO is omitted, a platform default is created at first use.

Building a custom adapter

• FileIO — the storage contract for BinaryStoreAdapter: readFile, writeFile, appendFile, deleteFile, fileExists. Implement it to plug in any backing byte store.
• PersistenceAdapter — the contract for a fully custom adapter: node, edge, and vector single/bulk operations, plus clearAll() and close().
• PersistenceChanges — describes one logical node/edge/vector commit. Adapters may implement applyChanges(changes) to commit it atomically; PolyGraph prefers this hook and restores the complete dirty batch when it rejects.
• Adapter methods should reject on storage errors. Bulk methods should be atomic where the backing store permits it. MemoryAdapter applies changes through copy-on-commit maps; BinaryStoreAdapter appends one WAL batch covering all three record kinds. Existing custom adapters without applyChanges remain compatible but cannot guarantee cross-record atomicity through the fallback path.

Change feed & mutation log

Adapters that expose changeFeed provide the durable logical log through graph.mutationLogSince(sequence), graph.mutationLogPage(sequence, limit), and graph.latestMutationSequence(). Sequences are exclusive bigint cursors, so callers can resume replication or audit scans without rereading acknowledged records. This is the mechanism the sync layer and cross-language conformance fixtures build on.

Schema metadata

Adapters may implement getSchemaDefinitions()/setSchemaDefinitions() to persist structural node/edge schema metadata. The canonical shape is nodeTypes[{ nodeType, ... }] and edgeTypes[{ edgeType, ... }], shared with the Rust and Python bindings. Runtime validator callbacks are intentionally not serialized; applications must re-register them after opening a store.

Capability checks

Adapters declare AdapterCapabilities: atomicBatches, transactions, fsync, secondaryIndexes, snapshots, changeFeed, concurrentWriters, and vectorSearch ('none' | 'exact' | 'ann'). graph.adapterCapabilities reads them; graph.requireAdapterCapabilities({ name: expected, ... }) throws AdapterCapabilityError unless the attached adapter declares every listed capability at the expected value. See Database core for resource limits and transactions built on top of this.

---

Back to Home.