@git-stunts/git-warp

A multi-writer graph database that uses Git commits as its storage substrate. Graph state is stored as commits pointing to the empty tree (4b825dc...), making the data invisible to normal Git workflows while inheriting Git's content-addressing, cryptographic integrity, and distributed replication.

Writers collaborate without coordination using CRDTs (OR-Set for nodes/edges, LWW registers for properties). Every writer maintains an independent patch chain; materialization deterministically merges all writers into a single consistent view.

npm install @git-stunts/git-warp @git-stunts/plumbing

For a comprehensive walkthrough — from setup to advanced features — see the Guide.

Quick Start

import GitPlumbing from '@git-stunts/plumbing';
import WarpGraph, { GitGraphAdapter } from '@git-stunts/git-warp';

const plumbing = new GitPlumbing({ cwd: './my-repo' });
const persistence = new GitGraphAdapter({ plumbing });

const graph = await WarpGraph.open({
  persistence,
  graphName: 'demo',
  writerId: 'writer-1',
  autoMaterialize: true,  // auto-materialize on query
});

// Write data using the patch builder
await (await graph.createPatch())
  .addNode('user:alice')
  .setProperty('user:alice', 'name', 'Alice')
  .setProperty('user:alice', 'role', 'admin')
  .addNode('user:bob')
  .setProperty('user:bob', 'name', 'Bob')
  .addEdge('user:alice', 'user:bob', 'manages')
  .setEdgeProperty('user:alice', 'user:bob', 'manages', 'since', '2024')
  .commit();

// Query the graph
const result = await graph.query()
  .match('user:*')
  .outgoing('manages')
  .run();

How It Works

Each writer creates patches: atomic batches of graph operations (add/remove nodes, add/remove edges, set properties). Patches are serialized as CBOR-encoded Git commit messages pointing to the empty tree, forming a per-writer chain under refs/warp/<graphName>/writers/<writerId>.

Materialization replays all patches from all writers, applying CRDT merge semantics:

• Nodes and edges use an Observed-Remove Set (OR-Set). An add wins over a concurrent remove unless the remove has observed the specific add event.
• Properties use Last-Write-Wins (LWW) registers, ordered by Lamport timestamp, then writer ID, then patch SHA.
• Version vectors track causality across writers, ensuring deterministic convergence regardless of patch arrival order.

Checkpoints snapshot materialized state into a single commit for fast incremental recovery. Subsequent materializations only need to replay patches created after the checkpoint.

Multi-Writer Collaboration

Writers operate independently on the same Git repository. Sync happens through standard Git transport (push/pull) or the built-in HTTP sync protocol.

// Writer A (on machine A)
const graphA = await WarpGraph.open({
  persistence: persistenceA,
  graphName: 'shared',
  writerId: 'alice',
});

await (await graphA.createPatch())
  .addNode('doc:1')
  .setProperty('doc:1', 'title', 'Draft')
  .commit();

// Writer B (on machine B)
const graphB = await WarpGraph.open({
  persistence: persistenceB,
  graphName: 'shared',
  writerId: 'bob',
});

await (await graphB.createPatch())
  .addNode('doc:2')
  .setProperty('doc:2', 'title', 'Notes')
  .commit();

// After git push/pull, materialize merges both writers
const state = await graphA.materialize();
await graphA.hasNode('doc:1'); // true
await graphA.hasNode('doc:2'); // true

HTTP Sync

// Start a sync server
const server = await graphB.serve({ port: 3000 });

// Sync from another instance
await graphA.syncWith('http://localhost:3000/sync');

await server.close();

Direct Sync

// Sync two in-process instances directly
await graphA.syncWith(graphB);

Querying

Query methods require materialized state. Either call materialize() first, or pass autoMaterialize: true to WarpGraph.open() to handle this automatically.

Simple Queries

await graph.materialize();

await graph.getNodes();                              // ['user:alice', 'user:bob']
await graph.hasNode('user:alice');                   // true
await graph.getNodeProps('user:alice');              // Map { 'name' => 'Alice', 'role' => 'admin' }
await graph.neighbors('user:alice', 'outgoing');    // [{ nodeId: 'user:bob', label: 'manages', direction: 'outgoing' }]
await graph.getEdges();                              // [{ from: 'user:alice', to: 'user:bob', label: 'manages', props: {} }]
await graph.getEdgeProps('user:alice', 'user:bob', 'manages');  // { weight: 0.9 } or null

Fluent Query Builder

const result = await graph.query()
  .match('user:*')           // glob pattern matching
  .outgoing('manages')       // traverse outgoing edges with label
  .select(['id', 'props'])   // select fields
  .run();

Object shorthand in where()

Filter nodes by property equality using plain objects. Multiple properties = AND semantics.

// Object shorthand — strict equality on primitive values
const admins = await graph.query()
  .match('user:*')
  .where({ role: 'admin', active: true })
  .run();

// Chain object and function filters
const seniorAdmins = await graph.query()
  .match('user:*')
  .where({ role: 'admin' })
  .where(({ props }) => props.age >= 30)
  .run();

Multi-hop traversal

Traverse multiple hops in a single call with depth. Default is [1, 1] (single hop).

// Depth 2: return only hop-2 neighbors
const grandchildren = await graph.query()
  .match('org:root')
  .outgoing('child', { depth: 2 })
  .run();

// Range [1, 3]: return neighbors at hops 1, 2, and 3
const reachable = await graph.query()
  .match('node:a')
  .outgoing('next', { depth: [1, 3] })
  .run();

// Depth [0, 2]: include the start set (self) plus hops 1 and 2
const selfAndNeighbors = await graph.query()
  .match('node:a')
  .outgoing('next', { depth: [0, 2] })
  .run();

// Incoming edges work the same way
const ancestors = await graph.query()
  .match('node:leaf')
  .incoming('child', { depth: [1, 5] })
  .run();

Aggregation

Compute count, sum, avg, min, max over matched nodes. This is a terminal operation — select(), outgoing(), and incoming() cannot follow aggregate().

const stats = await graph.query()
  .match('order:*')
  .where({ status: 'paid' })
  .aggregate({ count: true, sum: 'props.total', avg: 'props.total' })
  .run();

// stats = { stateHash: '...', count: 12, sum: 1450, avg: 120.83 }

Non-numeric property values are silently skipped during aggregation.

Path Finding

const result = await graph.traverse.shortestPath('user:alice', 'user:bob', {
  dir: 'outgoing',
  labelFilter: 'manages',
  maxDepth: 10,
});

if (result.found) {
  console.log(result.path);   // ['user:alice', 'user:bob']
  console.log(result.length); // 1
}

Subscriptions & Reactivity

React to graph changes without polling. Handlers are called after materialize() when state has changed.

Subscribe to All Changes

const { unsubscribe } = graph.subscribe({
  onChange: (diff) => {
    console.log('Nodes added:', diff.nodes.added);
    console.log('Nodes removed:', diff.nodes.removed);
    console.log('Edges added:', diff.edges.added);
    console.log('Props changed:', diff.props.set);
  },
  onError: (err) => console.error('Handler error:', err),
  replay: true,  // immediately fire with current state
});

// Make changes and materialize to trigger handlers
await (await graph.createPatch()).addNode('user:charlie').commit();
await graph.materialize();  // onChange fires with the diff

unsubscribe();  // stop receiving updates

Watch with Pattern Filtering

Only receive changes for nodes matching a glob pattern:

const { unsubscribe } = graph.watch('user:*', {
  onChange: (diff) => {
    // Only includes user:* nodes, their edges, and their properties
    console.log('User changes:', diff);
  },
  poll: 5000,  // optional: check for remote changes every 5s
});

When poll is set, the watcher periodically calls hasFrontierChanged() and auto-materializes if remote changes are detected.

Patch Operations

The patch builder supports seven operations:

const sha = await (await graph.createPatch())
  .addNode('n1')                                    // create a node
  .removeNode('n1')                                 // tombstone a node
  .addEdge('n1', 'n2', 'label')                    // create a directed edge
  .removeEdge('n1', 'n2', 'label')                 // tombstone an edge
  .setProperty('n1', 'key', 'value')               // set a node property (LWW)
  .setEdgeProperty('n1', 'n2', 'label', 'weight', 0.8)  // set an edge property (LWW)
  .commit();                                        // commit as a single atomic patch

Each commit() creates one Git commit containing all the operations, advances the writer's Lamport clock, and updates the writer's ref via compare-and-swap.

Writer API

For repeated writes, the Writer API is more convenient:

const writer = await graph.writer();

await writer.commitPatch(p => {
  p.addNode('item:1');
  p.setProperty('item:1', 'status', 'active');
});

Checkpoints and Garbage Collection

// Checkpoint current state for fast future materialization
await graph.materialize();
await graph.createCheckpoint();

// GC removes tombstones when safe
const metrics = graph.getGCMetrics();
const { ran, result } = graph.maybeRunGC();

// Or configure automatic checkpointing
const graph = await WarpGraph.open({
  persistence,
  graphName: 'demo',
  writerId: 'writer-1',
  checkpointPolicy: { every: 500 },  // auto-checkpoint every 500 patches
});

Observability

// Operational health snapshot (does not trigger materialization)
const status = await graph.status();
// {
//   cachedState: 'fresh',          // 'fresh' | 'stale' | 'none'
//   patchesSinceCheckpoint: 12,
//   tombstoneRatio: 0.03,
//   writers: 2,
//   frontier: { alice: 'abc...', bob: 'def...' },
// }

// Tick receipts: see exactly what happened during materialization
const { state, receipts } = await graph.materialize({ receipts: true });
for (const receipt of receipts) {
  for (const op of receipt.ops) {
    if (op.result === 'superseded') {
      console.log(`${op.op} on ${op.target}: ${op.reason}`);
    }
  }
}

Core operations (materialize(), syncWith(), createCheckpoint(), runGC()) emit structured timing logs via LoggerPort when a logger is injected.

CLI

The CLI is available as warp-graph or as a Git subcommand git warp.

# Install the git subcommand
npm run install:git-warp

# List graphs in a repo
git warp info

# Query nodes by pattern
git warp query --match 'user:*' --outgoing manages --json

# Find shortest path between nodes
git warp path --from user:alice --to user:bob --dir out

# Show patch history for a writer
git warp history --writer alice

# Check graph health, status, and GC metrics
git warp check

All commands accept --repo <path> to target a specific Git repository and --json for machine-readable output.

Architecture

The codebase follows hexagonal architecture with ports and adapters:

Ports define abstract interfaces for infrastructure:

• GraphPersistencePort -- Git operations (read/write commits, refs)
• IndexStoragePort -- bitmap index storage
• LoggerPort -- structured logging
• ClockPort -- time measurement

Adapters implement the ports:

• GitGraphAdapter -- wraps @git-stunts/plumbing for Git operations
• ConsoleLogger / NoOpLogger -- logging implementations
• PerformanceClockAdapter / GlobalClockAdapter -- clock implementations
• CborCodec -- CBOR serialization for patches

Domain contains the core logic:

• WarpGraph -- public API facade
• Writer / PatchSession -- patch creation and commit
• JoinReducer -- CRDT-based state materialization
• QueryBuilder -- fluent query construction
• LogicalTraversal -- graph traversal over materialized state
• SyncProtocol -- multi-writer synchronization
• CheckpointService -- state snapshot creation and loading
• BitmapIndexBuilder / BitmapIndexReader -- roaring bitmap indexes
• VersionVector / ORSet / LWW -- CRDT primitives

Dependencies

Package	Purpose
@git-stunts/plumbing	Low-level Git operations
@git-stunts/alfred	Retry with exponential backoff
@git-stunts/trailer-codec	Git trailer encoding
cbor-x	CBOR binary serialization
roaring	Roaring bitmap indexes (native C++ bindings)
zod	Schema validation

Testing

npm test                # unit tests (vitest)
npm run lint            # eslint
npm run test:bench      # benchmarks
npm run test:bats       # CLI integration tests (Docker + BATS)

AIΩN Foundations Series

This package is the reference implementation of WARP (Worldline Algebra for Recursive Provenance) graphs as described in the AIΩN Foundations Series. The papers define WARP graphs as a minimal recursive state object (Paper I), equip them with deterministic tick-based operational semantics (Paper II), and develop computational holography, provenance payloads, and prefix forks (Paper III). This codebase implements the core data structures and multi-writer collaboration protocol described in those papers.

License

Apache-2.0

---

_{Built by FLYING ROBOTS}