Releases: ModernRelay/omnigraph
v0.7.1
Omnigraph v0.7.1
A patch release on top of v0.7.0: three correctness fixes (camelCase filters,
cluster-apply crash loops, branch-merge OOM on embedding tables), one CLI
catalog-metadata improvement, and a warm-read performance fix. No breaking
changes, no on-disk format change, and no migration — drop-in over v0.7.0.
Fixes
-
camelCase property filters now execute (#283). A query — or a chained
mutation — that filtered on a camelCase schema field (e.g.repoName) linted
and planned cleanly but failed at run time withNo field named reponame. Column names are case sensitive.The identifier's case was destroyed at two
engine→Lance boundaries: the read-filter pushdown built the column with a
case-normalizing constructor, and the pending-batch mutation scan re-parsed
the predicate through a normalizing SQL context. Both now preserve case (the
read path uses a case-preserving column reference; the pending scan disables
SQL identifier normalization), so camelCase fields work consistently in read
and write predicates and a camelCase@indexequality still routes to the
scalar index. The fix is correct-by-construction rather than a per-query
guard; a regression test pins index routing so a silent full-scan fallback
can't slip back in. -
cluster applyno longer crash-loops a booting server (#284). Applying a
schema change while a graph had non-main (agent/review) branches, or a
migration that needed a backfill, could throw a freshly-booting
omnigraph-server --clusterinto an unescapable crash loop. Neither input is
an engine bug — the engine rejects both cleanly and before moving any graph
state — butcluster applywrote a recovery sidecar before calling the
engine and left it in place on the clean rejection, and the server refuses to
boot while a sidecar is pending. The asymmetric-cleanup path is fixed so a
pre-movement rejection leaves no stale sidecar, breaking the loop. -
Branch-merge fast-forward no longer OOMs on embedding tables (#277). A
branch→main fast-forward merge of a forked, embedding-bearing table
re-derived the whole branch through a single Lancemerge_insert— a
full-outer hash join over the entire delta — which exhausted the DataFusion
memory pool on high-dimensional embeddings (e.g. 8k rows × 3072-dim) and hung
or failed the merge. New rows now stream throughstage_append(no hash
join), only genuinely-changed rows are upserted, embeddings are no longer
stringified to diff them, and index coverage defers to the reconciler, so a
fast-forward merge completes in bounded work. The three-way merge path is
unchanged.
Improvements
-
omnigraph queries listsurfaces stored-query@description/
@instruction(#280). The CLI now shows a stored query's catalog metadata —
what it does and how to invoke it — in both human and--jsonoutput,
matching whatGET /queriesalready returned. Previously both fields were
silently dropped on the CLI side. -
Warm reads no longer pay an O(history) metadata tax (#268). Warm reads
used to re-derive per-query metadata (coordinator re-open,__manifest+
commit-graph re-scans, per-table re-open, double schema validation) on a cost
that scaled with commit history and never warmed up. A warm same-branch read
now does one cheap version probe, one schema read, and zero table opens on a
warm repeat (warm coordinator reuse, open-by-location+version, validate-once,
heldDatasethandles + one shared LanceSessionper graph). This also
closes a commit-DAG fork where a same-branch write after an external commit
could append off a stale cached head.
Upgrade notes
Drop-in over v0.7.0 — no configuration, schema, or data changes. Upgrade the
server and CLI together as usual. Graphs created on v0.7.0 read and write
identically on v0.7.1.
Edge
Rolling prerelease from b38b36e48f8103d64de18b9039ded3ebcee76a82.
v0.7.0
Omnigraph v0.7.0
v0.7.0 is three large arcs in one release. Operations: the cluster control
plane moves to object storage and the configuration architecture collapses to two
single-owner surfaces — a cluster can live entirely on an S3-compatible bucket, a
server boots from it with no local files, and the legacy combined omnigraph.yaml
is removed. CLI: the command-line surface is unified and made honest —
embedded and remote runs are one execution path, load becomes the single
bulk-write command, every command declares the capability it needs (and
rejects flags that don't apply), and the server boots only from a cluster.
Engine & substrate: Lance moves to 7.x, traversal/index/recovery internals
get faster and self-healing, and text embedding becomes provider-independent.
Highlights
Clusters & storage on object storage
- Clusters on object storage (
storage:).cluster.yamlgains an optional
storage: s3://bucket/prefixroot. Every stored byte — state ledger, lock,
recovery sidecars, approval artifacts, catalog blobs, and the derived graph
roots (<storage>/graphs/<id>.omni) — flows through one storage layer, so
file://(the default, byte-compatible with existing clusters) ands3://
are a single code path. The ledger's compare-and-swap uses S3 conditional
writes (If-Match/If-None-Match), verified against AWS, RustFS, and other
S3-compatible stores; the state lock is genuinely cross-machine on object
storage. - Config-free serving:
--cluster s3://bucket/prefix. The server accepts a
bare storage-root URI and boots from the applied revision on the bucket — the
ledger and catalog are the whole deployment artifact. Policy bundles serve as
digest-verified content from the catalog (never re-read from disk). The
preferred container shape becomes bucket, no volume (see
docs/user/deployment.md). - Cluster-only server.
omnigraph-serverboots only from `--cluster ` and serves N graphs (N ≥ 1) under cluster routes (`/graphs/{id}/…`, plus a read-only `GET /graphs` enumeration). The old single-graph flat-route mode, positional-`` boot, and `omnigraph.yaml` `graphs:`-map boot are gone — add or remove graphs with `cluster apply` and restart. - Resilient cluster boot with strict opt-out. Graph-attributed startup
failures now quarantine that graph and let healthy graphs serve;/graphs
lists only ready graphs, and quarantined graph routes return 404. Cluster-
global failures still refuse boot, and--require-all-graphs(or
OMNIGRAPH_REQUIRE_ALL_GRAPHS=1) restores fail-fast all-or-nothing startup
for operators who prefer any degraded graph to abort the process. - One storage substrate + recovery liveness. The cluster storage backend and
the engine both go through oneStorageAdapter(versioned read, conditional
replace/CAS, prefix delete), exercised by a storage fault-injection matrix.
A long-lived server now heals a recoverable write on its next write rather
than only at restart.
Configuration: two single-owner surfaces
The legacy combined omnigraph.yaml is removed. Configuration now lives in
two surfaces with single owners, plus a zero-config tier:
- Cluster config (
cluster.yaml+ checkout, team-owned) declares what the
system is: graphs, schemas, stored queries, policies, storage. A server boots
from it via--cluster. - Per-operator config (
~/.omnigraph/config.yaml, person-owned) declares who
you are:operator.actor(the last hop of the--aschain), output
defaults, named servers + clusters, profiles, aliases, and a default scope.
$OMNIGRAPH_HOMErelocates it. - Credentials keyed by server name.
omnigraph login <server>stores a
bearer token in~/.omnigraph/credentials(created0600; over-permissive
files refused). Resolution for a request whose URL matches an operator-defined
server:OMNIGRAPH_TOKEN_<NAME>env → the credentials file → the default
OMNIGRAPH_BEARER_TOKEN. A token is only ever sent to the server it is keyed to. - Operator targeting and aliases.
--server <name>(with--graph <id>for
multi-graph servers) addresses operator-defined endpoints. Operator aliases are
pure, read-only bindings — personal name → (server, graph, stored-query
name, default params) — invoking catalog-owned stored queries; they carry no
query content and a binding to a stored mutation is rejected. - Default scopes.
defaults.server(served) ordefaults.store(a zero-flag
local default — mutually exclusive withserver) supply the no-flag scope,
with an optionaldefault_graph.--profile <name>/$OMNIGRAPH_PROFILE
selects a named scope bundle wholesale;omnigraph profile list/
profile show [<name>]inspect what's defined (read-only).
Unified, capability-aware CLI
- One bulk-write command:
omnigraph load.loadis now the single data-write
command and works against remote graphs (over HTTP with the same bearer/actor
resolution as every other remote command) — previously the only data command
forced to open storage directly.--mode overwrite|append|mergeis required
(overwrite is destructive, so there is no default);--from <base>opts into
fork-if-missing for--branch.omnigraph ingestbecomes a deprecated
alias (--from main --mode mergedefaults; one-line stderr warning). - No implicit branch forks. Loading into a branch that does not exist is an
error unless--from <base>is given — a typo'd branch name no longer
silently forksmainand lands your data there. Same rule on the server. - One execution path, embedded ≡ remote. Every CLI verb runs through one
GraphClientwith two implementations (embedded engine, HTTP) sharing a single
wire-DTO crate (omnigraph-api-types). An executable parity matrix runs every
verb against both and asserts identical results, so local and remote no longer
drift. - Declared capabilities + honest addressing. Every command declares the
capability it needs —any(run against a graph, served or embedded),
served(needs a server),direct(direct storage access),control
(manage/inspect a cluster), orlocal(no graph) — and the CLI enforces it.
Wrong-capability addressing now fails loudly with a declared message (e.g.
--serveronoptimize) instead of being silently ignored, and a maintenance
verb pointed at a remote target is rejected.omnigraph --helpgroups commands
by capability with a legend. - Address cluster graphs for maintenance.
optimize/repair/cleanup
accept--cluster <dir|s3://…> --graph <id>(--clusteris a cluster directory,
storage-root URI, or aclusters:name from~/.omnigraph/config.yaml),
resolving the graph's storage URI from the served cluster state (no need to
hand-type<storage>/graphs/<id>.omni).--graphis the single graph selector
across server and cluster scopes. Conversely,omnigraph initrefuses a
cluster-managed path and points atcluster apply— graphs in a cluster are
created with ledger/recovery/approvals, not by hand.schema applyrefuses a
cluster-managed graph for the same reason (and the server rejects a cluster-
backed schema apply with409, pointing atcluster apply). - Write diagnostics + destructive-write safety (RFC-011 Decision 9). Every
write (load,mutate,branch create|delete|merge,schema apply,
optimize,repair,cleanup) echoes its resolved target + access path to
stderr — e.g.omnigraph load → s3://…/knowledge.omni (direct, remote)—
suppressible with the global--quiet. Destructive writes against a
non-local scope (cleanup, overwriteload,branch deleteagainst an
http(s)://server ors3://store/cluster) require explicit consent: the
global--yes, an interactive TTY prompt, or — for a non-interactive /
--jsonrun — a hard refusal instead of silently proceeding. Local (file://)
writes are unaffected. - Route alignment: canonical
POST /load. The server gains a canonical
POST /load;POST /ingestis now a deprecated alias that emits RFC 9745
Deprecation: true+ RFC 8288Link: <load>; rel="successor-version"
headers (a sibling-relative reference that resolves under/graphs/{id}/…).
The CLI'sloadtargets/load. - Operator aliases get their own namespace (
omnigraph alias <name>). A
personal binding to a stored query on a named server is invoked as
omnigraph alias <name> [args](RFC-011 Decision 4), so an alias can never
shadow — or be shadowed by — a built-in verb.aliasrejects global scope
flags (--server/--graph/--store/--cluster/--profile/--as) its
binding already owns. - No-graph addressing lists candidates (RFC-011 Decision 7). When a scope
has no--graphand nodefault_graph, the CLI never silently picks. A
cluster scope with exactly one applied graph uses it automatically and
otherwise lists the candidates (from the served catalog). A multi-graph
server lists the candidates (fromGET /graphs) and requires--graph <id>. - Invoke stored queries by name (RFC-011 Decision 3).
omnigraph query <name>/mutate <name>invoke a stored query by name from the served
catalog —omnigraph query find_peopleinstead of--query find.gq --name find_people. The verb asserts the query's kind (anexpect_mutationflag on
POST /queries/{name}:query <a-mutation>is rejected with'<name>' is a mutation — use omnigraph mutate <name>, and vice-versa)..gqfiles become
the explicit ad-hoc lane (-e/--query), with the positional selecting
which query in the source.
Engine & substrate
- Lance 6.0.1 → 7.0.0. The columnar substrate is bumped to Lance 7.x with
correct-by-design alignment: the unenforced primary key is immutable once set,
`Writ...
v0.6.2
Maintenance-safety release. Full notes: docs/releases/v0.6.2.md.
Highlights
- New
omnigraph repair— previews uncovered manifest/head drift (per-table classification, versions, Lance ops);--confirmpublishes verified maintenance-only drift,--force --confirmfor suspicious drift after review. optimizerefuses to compact over drift — tables with Lance HEAD ahead of__manifestwithout a recovery sidecar reportskipped: DriftNeedsRepairuntilrepairclassifies them.optimizepublishes compaction through the manifest — covered by a recovery sidecar, so a crash between compaction and manifest publish converges instead of leaving hidden drift.- Recovery roll-back converges the manifest — manifest-visible version realigns after a table restore.
- Legacy
__run__branch sweep — one-time v2→v3 manifest migration on first read-write open deletes stale__run__*staging branches (pre-v0.4.0 graphs); they no longer pollutebranch listor blockschema apply. - Pretty-printed JSON load input —
loadaccepts multi-line JSON objects, not just one-per-line JSONL.
Notes
- No manual migration; existing graphs open directly. The v2→v3 sweep is automatic, idempotent, and write-path-only (read-only deployments migrate on their next read-write open).
repairrequires a clean recovery state — reopen the graph first, then repair if drift remains.
Install
brew install ModernRelay/tap/omnigraph # Homebrew (macOS arm64 / Linux x86_64)
cargo install --locked omnigraph-cli # Cargo
Pre-built binaries (Linux / macOS / Windows) in the assets below.
v0.6.1
Operational polish after v0.6.0. Full notes: docs/releases/v0.6.1.md.
Highlights
- Stored-query registries — declare curated
queries:per graph inomnigraph.yaml; type-checked at startup, listed viaomnigraph queries list/GET /queries, invoked viaPOST /queries/{name}without accepting ad-hoc.gqfrom clients. - Stored-query policy gate — new Cedar action
invoke_query; stored mutations are double-gated (invoke_query+change). - Safer branch deletion — manifest is the single authority, visibility flips atomically; interrupted reclaims are reconciled by
cleanup, and reusing the name early reports an actionable error. - Blob-safe
optimize— tables withBlobproperties are skipped (visible asskippedin output/JSON/logs) instead of failing the sweep on Lance's blob-v2 compaction bug; other tables compact normally. - Windows support — releases now ship Windows x86_64 archives (
omnigraph.exe,omnigraph-server.exe) with a PowerShell installer. - Container entrypoint —
OMNIGRAPH_TARGET_URIcomposes withOMNIGRAPH_CONFIG, so the graph URI can live in env while policy/query config is mounted.
Notes
- Named graphs (
--target/server.graph) readgraphs.<name>.policy/.queries; top-levelpolicy:/queries:blocks are for bare-URI single-graph mode only and now fail loudly with a named graph. - Blob tables aren't compacted until the upstream Lance fix lands; reads/writes/queries are unaffected.
- The original notes listed the legacy
__run__cleanup (MR-770) here — it actually shipped in v0.6.2.
Install
brew install ModernRelay/tap/omnigraph # Homebrew (macOS arm64 / Linux x86_64)
cargo install --locked omnigraph-cli # Cargo
Pre-built binaries (Linux / macOS / Windows) in the assets below.
v0.6.0
Graph terminology, multi-graph server mode, and canonical query/mutate endpoints. This release renames the Repo concept to Graph across the Cedar resource model, policy API, and query-lint schema source; lets one omnigraph-server process serve up to 10 graphs behind cluster routes with per-graph and server-level policy; and promotes POST /query / POST /mutate (and the CLI's -e/--query-string) as the canonical inline surfaces, with /read and /change kept as deprecated aliases.
Full notes: docs/releases/v0.6.0.md.
Install
- Homebrew (macOS arm64 / Linux x86_64):
brew install ModernRelay/tap/omnigraph - Cargo (all five workspace crates on crates.io):
cargo install --locked omnigraph-cli - Pre-built binaries: see assets below (
omnigraph-linux-x86_64.tar.gz,omnigraph-macos-arm64.tar.gz).
Highlights
- Multi-graph server mode — one process serves 1–10 graphs concurrently under
/graphs/{graph_id}/..., each with its own Cedar policy. Start it with a non-emptygraphs:map inomnigraph.yamland no single-mode selector. Read-onlyGET /graphsenumerates the registry;omnigraph graphs listmirrors it on the CLI. - Graph terminology rename —
Omnigraph::Repo→Omnigraph::Graphin the Cedar resource model,repo_id→graph_idin the policy API, and"repo"→"graph"in the query-lintschema_source.kind. Operator group/action YAML is unaffected (entities are generated); only hand-rolled raw Cedar files need updating. - Canonical
POST /query+POST /mutate— inline query/mutation endpoints with a clean{ query, name, params, branch, snapshot }body./queryrejects mutations with a typed 400./readand/changecontinue indefinitely as deprecated aliases carrying RFC 9745Deprecation: trueand RFC 8288 successorLinkheaders. - Inline source on the CLI —
omnigraph query/omnigraph mutatewith-e/--query-stringrun ad-hoc.gqwithout a temp file; top-levelomnigraph lint(aliascheck) replaces the nestedomnigraph query lint. Legacy spellings warn once to stderr. - Per-graph + server-level Cedar policy —
graphs.<id>.policy.filegoverns each graph;server.policy.filegoverns the server-scopedgraph_listaction. Loaders reject a mismatched action-in-wrong-file at startup. - Strict init —
omnigraph initagainst an already-initialized URI now errors withAlreadyInitializedinstead of silently overwriting;omnigraph init --forceopts back in (it does not purge existing Lance datasets).
Behavior changes worth knowing
- No on-disk migration — existing v0.5.0 (and earlier)
.omnigraphs open directly. All formats unchanged. - Multi-graph deployments lose flat routes — bare
/query,/snapshot, … return 404 in multi mode; everything moves under/graphs/{graph_id}/.... SDK clients generated against a single-mode spec must regenerate. Single-graph invocation is unchanged. - Open servers require an explicit opt-in — a server with no bearer tokens and no policy refuses to start unless passed
--unauthenticated/OMNIGRAPH_UNAUTHENTICATED=1. Tokens-without-policy default-deny every non-readaction, andGET /graphsrequiresserver.policy.filein every runtime state. - Programmatic embedders —
ServerConfiggains amode,AppStateexposesrouting(),AuthenticatedActorbecomesResolvedActor,PolicyEngine::loadsplits intoload_graph/load_server, andPolicyRequest::actor_idmoves to a separateauthorize(actor_id, &request)parameter. The HTTP/bearer contract is unchanged. ChangeRequestfield rename —query_source→query,query_name→name; the legacy keys keep deserializing via serde aliases, so existing clients are unaffected.
Upgrade notes, the single → multi migration, and the full breaking-change detail are in docs/releases/v0.6.0.md.
v0.5.0
Lance 6 substrate, Cedar policy engine, schema-lint v1. This release jumps the storage substrate from Lance 4 to Lance 6.0.1 (DataFusion 53, Arrow 58), wires engine-wide Cedar policy enforcement into every authoring path, and ships a structured schema-lint v1 chassis with code-tagged diagnostics and explicit destructive-drop opt-in.
Full notes: docs/releases/v0.5.0.md.
Install
- Homebrew (macOS arm64 / Linux x86_64):
brew install ModernRelay/tap/omnigraph - Cargo (all five workspace crates on crates.io):
cargo install --locked omnigraph-cli - Pre-built binaries: see assets below (
omnigraph-linux-x86_64.tar.gz,omnigraph-macos-arm64.tar.gz).
Highlights
- Lance 6.0.1 substrate — DataFusion 52 → 53 and Arrow 57 → 58 come along for the ride. DF 53 optimizer rules (vectorized
IN-list,PhysicalExprSimplifier, push-limit-into-hash-join, CASE-NULL shortcut) now reach predicates via the new structured Expr pushdown. - Cedar policy engine — every
_aswriter (mutate, load, schema-apply, branch create/merge/delete) flows throughOmnigraph::enforce(action, scope, actor). Server defaults to deny-all without a Cedar YAML policy; actor identity comes only from signed token claims. - Schema-lint v1 — diagnostics now carry stable
OG-XXX-NNNcodes. Property and type drops support soft semantics; destructive drops require the new--allow-data-lossflag (CLI) or{"allow_data_loss": true}(HTTP). - Structured filter pushdown — query-language predicates push down via Lance's
Scanner::filter_exprinstead of stringified SQL.CompOp::Containsagainst list-typed columns now pushes down viaarray_has(previously fell through to in-memory filtering). - Inline
.gqsources — CLI and HTTP read/mutate endpoints accept query source inline, not just file paths. - CORS layer — optional CORS middleware on
omnigraph-serverfor browser-based UIs (OMNIGRAPH_CORS_ORIGINS). - Bug fixes — merge-insert dup-rowid (
SourceDedupeBehavior::FirstSeen+check_batch_unique_by_keysprecondition), branch-merge coordinator recovery on error paths, blob-column materialization during branch merge.
Behavior changes worth knowing
- On-disk format unchanged — existing v0.4.2 datasets open directly. No data migration.
- Server with policy enabled rejects writes by default — supply a Cedar YAML to authorize.
- Schema-lint diagnostics now have stable codes — CI parsers keying off prior free-form text need to switch to
OG-XXX-NNNmatching. - Destructive schema drops require explicit
allow_data_loss— soft-drop-or-reject is the new default. NOT INon nullable anti-join columns is now SQL-correct — side effect of the DF 53 bump.
Upgrade notes, the full migration/client/operator detail, and the test-coverage delta are in docs/releases/v0.5.0.md.
v0.4.2
v0.4.2
v0.4.1
release: bump version to 0.4.1
v0.3.1
Highlights
New CLI maintenance commands
omnigraph optimize— compacts small Lance fragments across every node and edge table in the repo. Run after large ingests to keep read paths fast.omnigraph cleanup— removes old Lance versions to reclaim storage. Requires--confirmto actually delete, and at least one of--keep N(recent versions to retain per table) or--older-than DURATION(e.g.,7d,24h,90m).
Performance
- Parallel per-type load writes:
omnigraph loadandomnigraph ingestnow write node and edge tables concurrently. Default concurrency 8, tunable viaOMNIGRAPH_LOAD_CONCURRENCY. (#46) - Dense
u32IDs through expand: graph traversal (execute_expand) now passes dense IDs internally instead of round-tripping throughString, reducing allocations on large traversals. (#47)
Bug fixes
- Deduplicate destination IDs before hydrating nodes in
execute_expand— previously could over-hydrate when the same destination appeared in multiple expand contexts. (#45)
Other
- OpenAPI spec polish for downstream SDK generation.
- README updates.