Skip to content

v1.4.6

Choose a tag to compare

View all tags
@github-actions github-actions released this 07 Oct 23:52
· 6 commits to main since this release
v1.4.6
da12ac7

Release Notes v1.4.6

Released: 2026-10-07

This release makes the cold graph page proof cheap enough for large graphs: snapshot and entity lookups now run in index-served clumps instead of handing SurrealDB 3.2 thousand-element IN lists, cutting a 12K-entity proof from 79s to 25s. Alongside it, the lifecycle acceptance gate gains end-to-end proof that a committed write survives response loss and a real daemon restart.

Highlights

Cold graph page proves in 25s instead of 79s

SurrealDB 3.2 serves an IN list from an index only up to 32 values; past that it scans the table and tests every row against the whole list, so cost grew as rows times ids. The graph page's cold proof handed it thousands of ids per statement, which pushed a 12K-entity graph past a team server's gateway timeout. The same graph now proves in 25s with an identical set of entities and edges.

Index-served lookup clumps

operational_relationships.org_lookup_clumps() builds [$org, ids] pairs via array::clump(array::distinct($ids), 32), so each lookup stays on the index. Duplicates are removed first: an id split across two clumps would return its row twice where a single IN returns it once. Each clump carries its organization in the closure argument, because a closure body that references any other binding silently evaluates to nothing on at least one engine. Rows are sorted in an outer select, since an ORDER BY beside the IN turns the plan into a full ordered index walk.

Snapshots without vectors for read-only comparisons

_snapshot() takes a new include_embeddings flag. Callers that only compare rows (graph_read_availability, graph_view_availability) pass include_embeddings=False and get rows back without embedding, name_embedding or fact_embedding, and without the write fingerprint, which covers stored vectors and cannot be computed without them. Vectors are storage, not evidence, and neither side of the comparison reads them now.

Overlapping relationship batches on pooled clients

New _each_batch() helper in graph_read_availability.py runs batch reads concurrently with asyncio.gather when the client pool has more than one slot, and sequentially otherwise. The single-slot path is deliberate: an embedded store only queues overlapping reads, and a queued waiter binds the pool's queue to the current event loop, which a later loop reusing the same client cannot wait on.

Lifecycle recovery proven across native restarts

Twenty complete capture, revise, delete, restore and recall chains now run over native storage in apps/e2e/tests/cli/test_memory_lifecycle_recovery.py: two scopes (private, project) across four injection phases (capture x4, revise x2, delete x2, restore x2). Each case loses the acknowledgement after the write commits, restarts the daemon, flushes the retained pending write with its original idempotency key, and compares full receipts, with recall as the post-recovery verification step rather than an injection point.

Graph Performance

  • operational_relationships.py: _SNAPSHOT is now generated by _snapshot_lookups(vectors=True), with a matching _SNAPSHOT_WITHOUT_VECTORS variant. All four lookups (entity, memory_derivations, source_states, relates_to) go through clumps.
  • graph_entity_search.py: entity get_many replaces per-uuid point lookups with array::clump($uuids, 32) over the unique index, costing a fraction of one lookup per uuid.
  • graph_read_availability.py: relationship endpoint reads use the clumped query and run concurrently per batch; initial capture and recapture both skip vectors.
  • graph_view_availability.py: available_graph_view takes its snapshot with include_embeddings=False.

Correctness Evidence

  • test_operational_snapshot_plans.py keeps a _SINGLE_IN_SNAPSHOT oracle, the exact statement shape the clumped lookups replaced, and asserts the clumped snapshot returns the same targets, associations, states, relationships and fingerprint over 70 entities and 40 edges, including foreign-group decoys, missing ids, and a duplicate straddling a clump boundary.
  • A live-server test (SIBYL_LIVE_SURREAL_TESTS=1 with SIBYL_SURREAL_URL) runs EXPLAIN on each clumped lookup and asserts the expected index (idx_entity_uuid, memory_derivation_target, source_state_identity, idx_relates_uuid) appears and TableScan does not.
  • Shape tests assert _LOOKUP_CLUMP <= 32, that every IN binds $lookup[1], and that no closure body names $org or contains an ORDER BY.

Lifecycle Testing

  • apps/e2e/tests/cli/lifecycle_runtime.py: LifecycleRuntime owns one persistent native store, a sibyld serve --embedded daemon, a recording HTTP proxy, and an isolated CLI HOME; OwnedCLI runs each CLI subprocess with its own environment and queue. Startup is verified against a per-run SIBYL_GIT_COMMIT nonce in /health, and shutdown asserts the proxy thread is reaped.
  • Loss injection is narrow: the proxy drops an acknowledgement only for a POST to the targeted path that returned success with a JSON mutation_receipt whose applied is true and whose idempotency_key matches the request header.
  • apps/e2e/tests/cli/test_lifecycle_proxy.py: seven forwarding cases (plain-text error, empty 204, ordinary JSON, error-with-receipt, unapplied receipt, mismatched key, malformed receipt) confirm the proxy preserves status, body and content type without arming the fault. A genuine upstream disconnect records RemoteProtocolError and returns an explicit 502, keeping transport failure distinct from deliberate loss.
  • Embeddings are disabled through missing provider credentials, so the acceptance campaign requires no paid inference.

Release and Deployment Artifacts

  • VERSION, apps/api/pyproject.toml and apps/cli/pyproject.toml move to 1.4.6.
  • charts/sibyl/Chart.yaml bumps both version and appVersion to 1.4.6.
  • charts/surrealdb/Chart.yaml bumps version to 1.4.6 only; its appVersion stays "3.2.4", which tracks the SurrealDB release it wraps.
  • docker-compose.quickstart.yml defaults sibyl-api, sibyl-worker and sibyl-web images to 1.4.6; infra/ansible/roles/sibyl/defaults/main.yml sets sibyl_version: "1.4.6".
  • Deployment and installation docs (docs/cli/docker.md, docs/deployment/ansible.md, docs/deployment/helm-chart.md, docs/deployment/kubernetes.md, docs/deployment/monitoring.md, docs/guide/installation.md) reference the new tag.

Compatibility

No breaking changes. For the full _SNAPSHOT statement, rows, their order and the operational write fingerprint are unchanged from the single-IN shape, as asserted against the oracle statement in test_operational_snapshot_plans.py. The no-vector variant deliberately omits the fingerprint, so that equivalence applies only to callers that read with embeddings.

Upgrade Notes

  • Update image tags to 1.4.6, or set SIBYL_IMAGE_TAG / sibyl_version explicitly if you pin.
  • Helm users: sibyl-surrealdb chart version moves to 1.4.6 while its appVersion stays 3.2.4. No SurrealDB engine change is implied.
  • No schema migration and no index changes are required; the gains come from query shape alone.
  • To exercise the live planner assertions locally, run the core test suite with SIBYL_LIVE_SURREAL_TESTS=1 and SIBYL_SURREAL_URL pointed at a SurrealDB server (the embedded engine plans IN lists differently and those cases skip).
  • The lifecycle acceptance suite needs the owned backend environment prepared with moon run api:sync; set SIBYL_LIFECYCLE_RECEIPT_DIR to keep per-case receipt JSON for triage.

Install

Local server

curl -fsSL https://raw.githubusercontent.com/hyperb1iss/sibyl/main/install.sh | sh -s -- --version 1.4.6

Remote CLI

curl -fsSL https://raw.githubusercontent.com/hyperb1iss/sibyl/main/install.sh | sh -s -- --remote --version 1.4.6
sibyl init --remote https://sibyl.example.com
sibyl auth login

Homebrew

brew install hyperb1iss/tap/sibyl
sibyl up

Arch Linux (AUR)

paru -S sibyl
sibyl up

Headless server

curl -fsSL https://raw.githubusercontent.com/hyperb1iss/sibyl/main/install.sh | sh -s -- --version 1.4.6 --no-open

Kubernetes (Helm)

helm repo add sibyl https://raw.githubusercontent.com/hyperb1iss/sibyl/gh-pages
helm repo update sibyl
helm upgrade --install sibyl sibyl/sibyl --version 1.4.6

Artifacts

This release includes Python wheels and sdists, the generated
Homebrew formula, the generated AUR PKGBUILD, Helm charts, Docker
SBOMs, aggregate dual-registry cosign receipts, and a SHA256 checksum
manifest.

Assets 19
Loading