OmniGraph v0.10.0
OmniGraph 0.10 adds graph-level Blob delivery, commit diffs and change feeds,
conditional writes, benchmark tooling, and merge/read performance improvements.
It upgrades the storage substrate from v0.9.0's Lance 9 to Lance 11.0.0 while
retaining graph storage format v6, recovery sidecars v9, and Lance
V2_2 data files. Existing v0.9 graphs do not require entity export/import.
Coordinated upgrade required. Public graph-facing APIs now describe nodes,
edges, types, entities, and properties; ambiguous table/row names are removed.
CLI/HTTP JSON and public Rust vocabulary have changed, so upgrade the CLI,
server, and client integrations together. Stop the old fleet before opening the
store with the new one. Do not mix Lance 9/10 binaries with Lance 11 readers or
writers.
Rebuild full-text indexes on every branch that needs search. Old indexes
without compatible analyzer proof now fail explicitly instead of silently
missing matches. Ordinary reads, traversal, and vector search remain available;
historical snapshots are retained but may refuse full-text search. Preserve a
verified whole-root backup for rollback and follow the
upgrade procedure.
Highlights
- Blob reads through the engine, HTTP, and CLI. Select a node or edge Blob
property at a branch or snapshot.blob getstreams bytes;blob statreads
metadata. HTTP GET/HEAD support ranges and conditional delivery. Managed
readers are bounded; valid empty values remain distinct from null through
compaction. Whole-object external references return their URI without target
I/O; HTTP redirects, while the CLI never follows them. See Blobs. - Secure external-Blob ingress. New URI references are denied by default and
require configured, normalized allow bases. Server execution never admits
file://; direct-store CLI access remains deny-only. Existing references
remain readable/exportable. Writes probe allowed sources before durable
effects and bound new external references across the whole operation. - Exact write receipts and conditional mutations. Mutation and Load return
theGraphCommitpublished by that attempt, not a later branch-head lookup.
A no-op mutation returnscommit: null. Canonical reads expose their exact
graph_commit_id;mutate --if-commitand dedicated HTTP conditional routes
reject stale tokens before effects with HTTP 412 / CLI exit 4. Older servers
reject the dedicated route rather than executing an unguarded write. See
mutations. - Commit diffs, change feeds, and baselines. Embedded, HTTP, and CLI surfaces
provide exact first-parent entity diffs, bounded at-least-once feed pages,
and streamed baseline/reset handshakes. Cursors bind graph, branch lifetime,
filters, and a captured cut; consumers own their checkpoints. Retention gaps
return typed HTTP 410, not a partial diff. Install the complete baseline
durably before saving its terminal resume cursor. See
changes and change feeds. - Less work for merges and reads. Eligible merges classify candidates from
fragment lineage instead of scanning every entity; unsupported cases retain
the full-scan fallback. Warm catalog refresh reads newly published commits
incrementally. Ordinary node scans omit unneeded properties where safe, and
constructive multi-type mutations prepare independent datasets concurrently
while still publishing one graph commit. - Safer branch reuse and merge behavior. Deleting and recreating a branch
no longer reuses its predecessor's native paths/cache identity. Deletion
acknowledges graph publication;cleanupreclaims the retired forks.
Branches whose edits net to zero can be merged, and traversal now includes
stored self-loop edges. See branching. - String predicates and stricter query validation. String
containsand
starts_withprovide exact, case-sensitive substring/prefix matching, with
index acceleration where available and correct scan fallback. Missing
required parameters fail before execution; undeclared variables in inline
property matches, including negation, fail during typechecking. Nullable
parameters remain optional. Blob projection, ordering, aggregation, and
newly declared Blob uniqueness constraints are rejected explicitly. - Initialization and recovery hardening. Graph creation publishes its
initial contents and schema stamp together. An initialization failure no
longer deletes a graph that actually committed; typed outcomes distinguish
committed, indeterminate, and claimed roots. A live write handle can heal
stranded, effect-free recovery intents without reopening, within the existing
single-mutation-process boundary. - Lance correctness fixes. Includes stable-row-ID delete/index alignment,
Blob compaction/selection, decoder hardening, and overwrite/restore identity
high-water marks. Ambiguous commit outcomes remainUnknown, never a reason
to replay automatically. OmniGraph retains its final-output-partition fence
for exactnearestranking. The h2 0.4 dependency line advances to 0.4.16;
the AWS SDK's separate legacy h2 0.3 line is unchanged.
Developer-facing
- Declarative benchmarks.
omnigraph-benchruns YAML-defined merge cases
from identical frozen fixture state in isolated workers, recording phase
timings, logical store calls, peak RSS, fixture identity, and source/binary
provenance. Clean release builds can archive complete or explicitly censored
canonical JSON; a disposable OmniGraph projection provides bounded queries
over that archive. Censored runs remain failures. Local records remain
claim_eligible: falsewithout a controlled, digest-bound build receipt;
logical calls are not physical cloud requests. The separate FinGraph
node-and-edge diagnostic supports qualified APFS or EC2 instance-store
NVMe/XFS fixtures but does not publish durable telemetry or dispatch AWS jobs.
See the harness guide and
fixture commands. - Merge timing hooks.
MergeWriteProbes::merge_timing_snapshotexposes
stable phase identifiers, total/max microseconds, and completed intervals.
TableWalkidentifies general three-way staging walks; proven-insert paths
emit none. Normal production execution leaves the task-local probe unset and
performs no timing clock reads. - Compiled operation descriptors.
lint --jsonnow includes ordered result
fields and compiler-owned read/write dependencies, including nested negation,
inserted-edge endpoints, and node-delete cascades. Invalid queries emit no
partial descriptor. Server/SDK descriptor projection is not included. - Typed storage failures. Rust callers can inspect
StorageFailureKind
(Transient,Configuration,NotFound,Precondition,Permanent,
Unknown) instead of parsing Lance error strings. Positive transient
evidence is not replay authorization; erased or ambiguous evidence remains
Unknown. Generic storage errors still map to HTTP 500.
Compatibility
CLI, HTTP, and embedded API
- Breaking graph vocabulary. Graph-facing names now distinguish nodes/edges,
types, entities, properties,
graph-manifest versions, and published dataset versions. Legacy aliases such
astable_key,row_id, ambiguousmanifest_version,rows_loaded, and
export --tableare removed rather than dual-emitted. Update consumers to
the release's OpenAPI contract and
CLI reference; this is not a rolling-safe
v0.9/v0.10 boundary.embed --jsonuses record-named counters, and cluster
observations usegraph_manifest_version. - New branch lives use internal native refs named
{name}.{ulid};
native_dataset_branchreports that name. Existing bare refs remain valid.
Public branch paths ending a segment in.plus 26 upper-case letters/digits
are reserved and refused. See RFC 0042. OmniError::Lance(String)becomesOmniError::Storage(StorageFailure).
Direct storage consumers must handleStorageError::Backend, replace tuple
Iomatches withIo { failure, source }, and remove
CreateIfAbsentUnsupportedmatches. Exhaustive error matches must also cover
new initialization, Blob, precondition, change-feed, and full-text rebuild
outcomes; see the error definitions.- The embedded
read_blobAPI exposing Lance'sBlobFileis removed. Use
read_blob_at; each managed range is at most 4 MiB. CLI/HTTP stream larger
values. CLIgetoutputs raw bytes and a failed transfer can leave a partial
file; use temporary-file replacement when required. Treat resolved Blob
snapshot witnesses and ETags as opaque, not as commit IDs or content hashes.
Full-text indexes and retained history
rebuild-full-text-indexes replaces every planned index on the selected branch
from current entities and publishes them in one graph commit. It uses the
default English analyzer, warns that custom tokenizer settings are replaced,
and refuses unknown physical index kinds before publication. Rebuild each live
branch that needs search; other branches and historical snapshots are not
rewritten. Restoring an older snapshot can require another rebuild.
Ordinary optimize preserves existing full-text indexes rather than folding
uncertified postings. Unindexed entities remain searchable through a scan; use
the explicit rebuild command to refresh coverage. Other index reconciliation
and compaction continue normally. See maintenance.
Blob identity and rollback
- New or schema-rebuilt fields carry stable property-lifetime metadata.
Schema-preserving writes retain an upgraded dataset's old unmarked schema;
full-type Overwrite adopts the new metadata. An unmarked pre-0.10 Blob field
is readable only at its exact current physical dataset entry; older entries
are refused even if no rename occurred. Historical property renames and
older branch-owned snapshots can also be refused when identity cannot be
proven. Current type aliases can resolve pure type-renamed history; retired
aliases do not remain available. Details: Blob identity. - Readers hold their selected version but do not pin it against branch deletion
or destructive cleanup. Quiesce readers when they must finish. Persisted
ranged external descriptors are refused, never widened to whole objects. - A pre-existing v6 schema with a Blob uniqueness constraint remains openable
for inspection/export, but that constraint is not made functional. Rebuild
with the invalid constraint removed. - Configuring
external_blobschanges cluster state in a way v0.9 rejects.
A downgrade also loses default-deny external ingress. Removing the field is
not a full-text rollback: restore the whole pre-upgrade graph backup with
the old fleet. Follow cluster rollback.
Support boundaries and distribution
- Azure Blob storage is a preview, not production-supported. Native
az://roots, Azurite qualification, and a managed-identity smoke proof are
available. The disruptive live-Azure qualification matrix remains pending.
Preview writes must useomnigraph-azure-admission; its lease admits one
writer process and is not a distributed engine fence. - One mutation process per graph remains the supported topology. Conditional
writes and recovery do not introduce distributed writer fencing. Full-text
rebuilding is direct-storage maintenance, not an HTTP endpoint;--asis
attribution, not server-policy authorization. - crates.io publication remains paused. Historical
omnigraph-*registry
packages are frozen at 0.8.0 and are not this release. Binaries ship through
GitHub Releases, the installer, Homebrew, and Docker. See
distribution policy.