Skip to content

Releases

Release v0.6.0

Choose a tag to compare

@homer6 homer6 released this 16 Nov 07:06
b2861be

Released: 2026-11-16 · Branch: dev · Previous: v0.5.0 (2026-10-23)

v0.5.0 was the security release. v0.6.0 prepares the repository for its open-source release (#1500), lands the first batch of fixes for the pre-launch security review (#1388), and makes score and explain work on segmentation models.

This release was not planned with a release epic. It carries the pull requests merged into dev since v0.5.0. The epic #1500 is its tracking issue.

14 pull requests since v0.5.0:

  • 4 security fixes;
  • 1 build fix;
  • 1 machine-learning fix;
  • 5 release-readiness changes;
  • 2 docs;
  • 1 test.

The generated OpenAPI still has 190 /v1 paths, and no capability was added or removed.


Upgrade: read this first

  • A rolling upgrade is fine. Nothing in this release changes replication, the peer protocol or a replicated record's shape. A v0.5.0 node and a v0.6.0 node can share a cluster.
  • Clerk now needs its authorized parties (#1423, in #1503). A node with a Clerk issuer set accepts a Clerk token only when the token carries an integer exp and an azp listed in ZETTA_CLERK_AUTHORIZED_PARTIES, a comma-separated list of this node's Studio origins. With an issuer and no list, every Clerk token is refused. Bearer and session tokens are unaffected. No deployment sets a Clerk issuer today.
  • An inbox needs its own grant (#1432, in #1505). A topic-prefix grant no longer reaches an inbox. A grant meant to open an inbox must name inbox:<name>.
  • The Studio web app is no longer in the repository (#1527). webapp/ was removed because it needs a rebuild from its source (#1526). A node started without --webapp-dir serves its built-in server-rendered UI at /, as before. A deployment that bakes webapp/ into its image must stop copying it. The Dockerfile added here handles both cases.

Behaviour changes

  • Cookie routes check the session before reading the body (#1395, in #1502).
    • A request with no session is answered without its body being read, and the connection is closed.
    • POST /login, POST /session and POST /blob/delete accept at most 64 KiB, and answer 413 above that.
    • A blob upload checks sql:write before reading the file.
  • Redirects (#1503):
    • A web run redirects only to a path on this origin (#1435).
    • The outbound HTTP client keeps secret headers across a redirect only when the scheme, host and port all match (#1429).
  • list_datasets lists only what the caller may read (#1439, in #1503). The rule is the same as system.datasets: an administrator or the local session sees every dataset, and anyone else only the tables it holds sql:read on.
  • storage_usage shows a caller who is not an administrator only the buckets it may read (#1430, in #1505). Per-kind totals are 0 for such a caller.
  • The web shell runs as the signed-in user (#1434, in #1505): statements are authorized, executed and audited under the user, not the local session.
  • Parsers of untrusted input are bounded (#1504):
    • Thrift metadata nests at most 64 levels (#1424).
    • Parquet counts are checked against the bytes present, and read_parquet_rows refuses a footer claiming more than 10,000,000 rows (#1425). A Parquet file over 10 million rows uploaded through add_dataset is now refused.
    • A tokenizer vocabulary whose ids run far past its size is refused (#1431).
    • A JPEG with a second frame is refused (#1418).
    • A Python pipeline expression nests at most 200 levels (#1427).

Added

  • score and explain on segmentation (M-3) models (#1497, in #1499).
    • score places each row in its nearest segment: kind: "segmentation" and assignments.
    • explain names the segment, the runner-up, and each column's contribution to the choice.
  • The image Dockerfile is in this repository (#1530). It was in zetta-bridge. Build from the repository root: docker buildx build --platform linux/amd64 -t <registry>/zetta:<version> .. See docs/build.md.
  • Licensing (#1524):
    • MIT LICENSE and COPYRIGHT (OramaSearch, Inc.).
    • SPDX-License-Identifier: MIT in every source file.
    • THIRD_PARTY_LICENSES, with 15 components and none incompatible with MIT.
    • An SBOM in SPDX and CycloneDX form, regenerated with tools/sbom.py.
    • CONTRIBUTING.md, with contributions under a copyright-assignment CLA (CLA.md).
    • CODE_OF_CONDUCT.md.
  • SECURITY.md (#1506): report vulnerabilities privately to security@orama.com.

Build and repository

  • Every FetchContent source is pinned by commit or hash (#1508): SDL3, Dear ImGui, json-schema-validator and GoogleTest by commit, and the Boost archive by SHA-256.
  • Submodule remotes are https (#1530). Every submodule repository is public, so an anonymous git clone --recurse-submodules works.
  • Schema $ids use one namespace, https://oramasearch.github.io/zetta/schemas/ (#1530). Nothing resolves or compares them.
  • Client and internal material is out of the tree (#1523, #1525, #1527). The churn fixture's company is now the invented Contoso, and the demo data generators describe an invented business. The agent files are rewritten for the public repository, and finding ids and private links are removed. docs/full-context.md and docs/architecture/code-audit.md are gone.
  • An object-log test's hooks now outlive the log they are installed in (#1491, in #1492). The v0.5.0 notes record the image digest and the AddressSanitizer result (#1493).

Not in this release

These pull requests are open and were held back as high risk. Their issues are closed as in-pr, but their fixes do not ship in v0.6.0:

  • #1507:

    • read-only SQL enforced by the engine (#1501);
    • the identity root for burst nodes (#1343);
    • feedback bundles scoped to the caller (#1455);
    • bounded pipeline graphs (#1426);
    • the audit surface and the background actor (#1328, #1327);
    • study trials run as their caller (#1322);
    • action delivery off the owner loop (#1406).

    It moves the audit protocol to 3.

  • #1509 and the stacked #1511:

    • an index keeps its source database (#1428);
    • a default bucket quota (#1421);
    • bounded invalid-token work (#1417);
    • discovery docs per caller (#1357);
    • a grant per saved connection (#1339);
    • the connection root secret outside data_dir (#1314);
    • delivery credentials only to approved endpoints (#1453);
    • egress refusals naming the run principal (#1323);
    • the decided audit, principal and credential follow-ups.
  • #1510: re-reading the egress allowlist without a restart (#1332).

  • #1512:

    • identity root rotation (#1342);
    • the identity epoch (#1360);
    • CTL-2 V2 enrollment (#1319);
    • per-node revocation (#1320).

    The second enrollment release is #1514 (#1513).

  • #1516:

  • Backup routes for the operator's backup system: #1521 and #1522.

  • The Studio web app, until it is rebuilt from a cleaned source (#1526).

Known issues

Security-relevant first.

  • The security fixes listed under Not in this release are not in this tag. They are tracked under #1388 and the epic #1500.
  • The repository and the image oramasearch/zetta were made public on 2026-11-16, with the full history and every earlier image tag. GitHub private vulnerability reporting is on, and security@orama.com also takes reports.
  • The new cookie-route bounds (#1531) and Parquet count bounds (#1532) have no tests of their own yet.
  • score over a table on a multi-member cluster scores only the rows the answering member owns (#1533).
  • An intermittent segfault in Uc1Journey.ARunWithoutADatasetReadsItsPipelinesTable (#1485).
  • Latency-budget tests fail on a loaded host (#1256).
  • WHERE NOT (x = 5) matches rows where x is null (#1473).
  • Pending outbox and report rows from before v0.5.0's outbox change are not delivered after the upgrade (#1474).

Where it has run

Not on any cluster, and not on the dev environment. This release was cut without a dev environment run first, on Steve's decision (2026-11-16), and the image is built but not deployed. The image oramasearch/zetta:v0.6.0 is built from the tag on the cluster BuildKit (linux/amd64) with this repository's Dockerfile, from a detached checkout of the tag (b2861be5) with every submodule at its pin:

oramasearch/zetta:v0.6.0         sha256:97fa856e5a2c7e920498ecdd0cce3dbac2a2b05f5055b28210007fde18a72215
oramasearch/zetta:main-b2861be5  (the same digest)

Verification

Full test suite, run serially (ctest --test-dir build, no -j) on the release branch at 072a21ad (dev at 9ea9f3a8 plus the version move), macOS 14.6.1 arm64, Homebrew LLVM Clang, static Release build. Submodules were cloned fresh over their https remotes. The run took 1,098 s, on a host with a virtual machine running (load average 25 to 60):

3,475 registered · 3,474 run · 3,4...
Read more

Release v0.5.0

Choose a tag to compare

@homer6 homer6 released this 23 Oct 22:45
e11c02f

v0.5.0

Released: 2026-10-23 · Branch: dev · Previous: v0.4.0 (2026-10-19)

v0.4.0 was about datasets. v0.5.0 is the security release: the fixes from the auth review
(#759–#769), the multi-tenant audit, TC-5 and the pre-launch security review (LSR findings,
#1388), plus the features that landed beside them: three
new source connectors (MySQL, TiDB, lake's SQLite/DuckDB/Polars services), a per-database safe mode,
WHERE with AND/OR/NOT, published pipelines, and run proposals a person approves.

73 pull requests since v0.4.0 (52 fixes, 12 features, 2 tests, 1 performance change, 6 docs).
The generated OpenAPI has 190 /v1 paths (was 177), the capability registry has 179
capabilities (was 168), and the test sources hold 3,554 test cases (was 3,042).


Upgrade: read this first

  • Upgrade every node of a cluster together. Several changes are not compatible with a node from
    v0.4.0 in the same cluster:
    • Sealed identity (#1340, #1359): accounts, roles, tokens, grants and policies are sealed under a
      cluster identity root, and every record carries a version. A v0.4.0 node refuses token records
      that carry _seal, so its callers get 401 until it is upgraded.
    • Text entity-key mappings as a table (#1442).
    • Indexed search (#1459): a v0.4.0 peer refuses the database field a v0.5.0 coordinator
      sends, so distributed search fails closed until every node is upgraded.
    • Distributed WHERE (#1366): a v0.4.0 partition owner refuses a compound WHERE with an
      error (never partial results); a single comparison keeps the old shape.
  • #1442 cannot be rolled back. Text entity-key mappings written by v0.5.0 are stored in the new
    table; a v0.4.0 node does not read them.
  • The identity root is created on the first v0.5.0 start (#1340, #1480), from
    --admin-token/ZETTA_ADMIN_TOKEN (or generated), and pinned with its seed in
    <data_dir>/security/identity-root.json (mode 0600). Under the Orama operator every node has the
    same token, so every node derives the same root and nothing changes in the operator. Records held
    before the upgrade are sealed once on that first start. See
    identity-seal.md.
  • Outbound connections are denied by default (#1321): the node reaches only loopback, the Clerk
    issuer and --egress-allow/ZETTA_EGRESS_ALLOW. Under a Lake the operator sets * and the
    namespace's NetworkSandbox enforces egress
    (orama-operator#723, rolled).
  • Connector sources need an approved destination (#1469): a private or loopback source is dialled
    only when ZETTA_CONNECTOR_ALLOW_DESTINATIONS lists it as an exact host:port. Roll
    orama-operator#890 before this image
    ,
    or every in-cluster source (lake's SQLite/DuckDB/Polars, in-cluster Postgres, TiDB) is refused. A
    lake SQL endpoint must now name http:// or https://.
  • Action and report delivery dial only public addresses (#1452) unless
    ZETTA_ACTION_ALLOW_DESTINATIONS names a class.
  • Pending action-outbox and report rows written before the upgrade cannot be delivered (#1467):
    their records carry no actor binding. Re-enqueue them after the upgrade
    (#1474).
  • Agent checkpoints must be registered (#1465): agent.model must be base-agent,
    /data/models/base-agent or an entry of ZETTA_AGENT_CHECKPOINTS. Lake's default base model path
    is registered; agents on other model strings stop serving.
  • The corosio pin moves to orama-main be514224 (corosio#15 peer certificate, corosio#16
    full-duplex TLS, and the reactor fixes #6, #8, #11, #13).

Behaviour changes

  • Safe mode per database (#1373): set_safe_mode / get_safe_mode set Off, Warn Writes or Warn
    All on a Zetta database or a saved connection. A warned statement is refused with
    safe_mode_warning (REST 428) carrying the exact statements and a confirmation bound to them; the
    same call with safe_mode_confirmation runs it. A database defaults to Off and a connection to
    Warn Writes
    (connections are read-only, so Warn Writes prompts for nothing until Warn All). See
    safe-mode.md. The consent gate stays off by default; its approval
    surface (Governance → Consent requests) is new.
  • Authorization is per caller in more places: the system catalog and /v1/specs (#1358), indexed
    search with row policies (#1459), analyze_dataset/plan_features (#1451), virtual tables
    (admin-only, #1449), trace diagnostics (admin-only, #1462), the cookie blob routes (#1448), and a
    legacy AccessKey (row security and grants, #1361). Each graph step runs as the run's principal
    (#1330). A reader may see fewer rows or get unauthorized where it used to read.
  • Role management is bounded by the caller's own rights, and built-in roles are fixed (#1355).
  • A row policy missing its condition denies instead of admitting every row (#1352).
  • agent_run runs a plan only after a person approves it (#913, #1337), and the chat agent
    proposes runs a person approves (#1384).
  • The owner token is stored as PBKDF2-SHA256 (#1311); seeding resumes after a crash.
  • Blob grants match on a / path boundary (#1312): a grant on report no longer admits
    report-secret-2026.
  • Refused reads are audited and do not reveal whether a table exists (TC-5, #1345); the node
    audits its own catalogue changes and names the caller on every row (#847, #1333).
  • The SQL parser never throws (#1336), so no statement can end the node.
  • Peers: a HELLO is bounded and shape-checked (#1313), its node id must be its certificate's
    common name (#1326), setup has one deadline and accept errors are recovered (#1457), and mTLS links
    are full duplex (#1347, the fix for the silent SYNC stall #835).
  • Imports and exports resolve only under the node's import and export roots (#1450, #1454); model
    downloads stay in their directory (#1466); Parquet input is bounded and type-checked (#1463, #1464);
    model imports are bounded (#1447, #1472); dataset generation has a ceiling (#1456).
  • Browser sessions and actions are bound to their origin (#1471); JSON embedded in HTML is encoded
    (#1461); public LLM docs carry no inventory (#1458).
  • The join token is never printed (#1446), reaches cloud VMs from each vendor's secret store
    (#1318, #1346), and cluster secrets are created private (#1468).

Added

  • Source connectors: native MySQL, read-only and TLS by default (#1371); TiDB over the MySQL
    protocol, fenced the way TiDB allows (#1387); lake's SQLite, DuckDB and Polars services (#1385).
    Catalog filters are encoded per SQL dialect (#1460).
  • WHERE with AND, OR, NOT and parentheses (#912, #1366), with three-valued logic.
  • Published pipelines (#1289, #1444): expose_pipeline pins a pipeline as
    POST /v1/pipelines/{name}/run; a call runs with the caller's own grants and a per-node rate limit.
  • Safe mode (#1373), above.
  • list_projects returns graph summaries (#1436); a lean list_datasets summary (#1438);
    60-second long polls for receive_events and browse_topic (#1437).
  • Full-text search: a query naming a field searches only that field (#1363), and every shard is
    scored with the merged corpus statistics (#1372, with oramacore-cpp#2).
  • Chaos experiments (#1353): an mTLS cluster tormented on a seeded schedule; the long runs and the
    every-fault-class smoke are opt-in (ZETTA_CHAOS_SMOKE, ZETTA_CHAOS_SCHEDULES).
  • The authz regression harness for #759–#769 (#1325).

Performance

  • Gossip sends only the changes a peer lacks (#794, #1350): 13–17× fewer bytes per round.
  • Catalogue reads no longer wait for a save (#1248, #1440).

Not in this release

  • Root rotation, and handing the identity root to a node added after an admin-token rotation
    (#1479).
  • Safe mode in the desktop SQL console and the governed console, in a pipeline run that reads a
    saved connection, and a Run / Cancel dialog in Zetta's own web console (Lake has one, lake#1080).
  • Provisioned burst nodes are not given the identity root
    (#1343).

Known issues

Security-relevant first.

  • The remaining pre-launch review findings are open under
    #1388, among them: LSR-173 synchronous action delivery
    can block the owner loop (#1406); LSR-174 system SQL views expose metadata without their read
    permission (#1407); LSR-162 cookie routes buffer unauthenticated bodies (#1395); and the p3 findings
    LSR-244 to LSR-262 (#1417–#1435).
  • A node that never saw a newer sealed record accepts an older one when no holder is left (#1360).
  • Node certificates can be minted by any join-token holder, and there is no per-node revocation
    (#1319, #1320); network and filesystem reach is node-wide, not per run principal (#1323); a saved
    connection's credential is usable by anyone holding the connection capability (#1339).
  • /v1/openapi.json and /llms*.txt are public and list every capability (#1357).
  • An intermittent segfault in Uc1Journey.ARunWithoutADatasetReadsItsPipelinesTable (#1485),
    under investigation with AddressSanitizer.
  • Latency-budget tests fail on a loaded host (#1256).
  • WHERE NOT (x = 5) matches rows where x is null (#1473).
  • Pending outbox and report rows from before LSR-237 are not delivered after the upgrade (#1474).

Where it has run

Not yet on any cluster. The image above is rolled by the oper...

Read more

Release v0.4.0

Choose a tag to compare

@homer6 homer6 released this 20 Oct 02:35
104e8a9

v0.4.0

Released: 2026-10-19 · Branch: dev · Previous: v0.3.0 (2026-10-17)

v0.3.0 was Zetta's part of secure networking. v0.4.0 is about datasets: they keep the
column order of the file they came from and report typed columns, LIMIT is pushed to the
partitions instead of cutting a full scan, text columns become categorical features, the
dataset contract is typed from the partition catalog, and a dataset can be loaded from a
stored object up to 1 GiB instead of 16 MiB inline.

Four pull requests since v0.3.0: three features (#1299, #1297, #1303) and the v0.3.0 image
digest in the release notes (#1295). The generated OpenAPI has 177 /v1 paths (was 176),
the capability registry has 168 capabilities (was 167), and the test sources hold 3,042 test
cases (was 2,992).


Behaviour changes

  • get_dataset answers columns as typed objects, [{name, type, nullable, null_fraction}],
    where it answered a list of names (#1299). list_datasets still answers names, so its callers
    are unchanged. A caller reading get_dataset's columns as strings must read .name. Lake's
    dataset page reads the new shape from oramasearch/lake#834 (lake
    v0.3.0).
  • Columns are listed in the source's order, not alphabetically, in get_dataset,
    list_datasets, system.datasets and SELECT *: the CSV header, the Parquet schema, the order a
    JSON or JSON Lines source first writes each key, a connector query's SELECT list, or a
    CREATE TABLE column list (#1299). A dataset loaded before this release keeps its
    alphabetical order
    and says column_order: "first_seen"; loading the file again gives the
    source order. Nothing restores the old order for new loads.
  • Text columns with 2 to 32 levels become one-hot categorical features of the M-1 GBDT
    (#1297). A model trained on the same data can now use columns it used to drop with
    not numeric; categorical columns are not supported yet, so its features and scores differ. The
    model stores the fitted levels, and score, explain and POST /v1/models/{name}/predict take
    the original text column: a row missing a categorical column, or giving it a number, is
    refused
    with the column named. An unseen level or a null encodes as all zeros. Models trained
    before this release are unchanged.
  • The dataset contract is typed from the partition catalog where every row is in a partition
    that recorded its schema (#1297). The decisions match the 256-row sample in every case but one:
    a column the catalog types numeric whose first 256 rows are all null now becomes a feature.
  • upload_dataset's refusal of a body over 16 MiB now names load_dataset_object (#1303).
  • A column the caller may not read answers type, nullable and null_fraction as null in
    get_dataset, and ORDER BY on it is refused, as WHERE and GROUP BY already were (#1299).

A v0.3.0 node and a v0.4.0 node answer the same SQL with the same rows: a v0.3.0 responder
refuses the new keep and order_by fields of sql.scan, and a v0.4.0 coordinator asks it again
without them (#1299). A whole mixed-version cluster was not tested; upgrade every node together.


Added

load_dataset_object: a dataset from a stored object — #1303

Loads a CSV or JSON Lines object from a bucket as a dataset, past upload_dataset's 16 MiB inline
cap, for Lake's chunked upload (#1302; oramasearch/lake#846). By
@thatjuan.

  • A Job using the add_dataset operation. The caller also needs objects:read on the bucket. It
    answers a run_id at once; poll it with get_run.
  • Refusals come before a run is created, in this order: invalid_input, unauthorized,
    not_found (/bucket), conflict (/table_name), engine_error.
  • The object is read in 4 MiB slices on a worker and handed to the owner in 10,000-row batches.
    The table is partition-only: rows live in partition bodies on their owners, and SELECT
    gathers across them. A failed load drops its table; remove_object runs after either outcome.
  • upload_dataset, add_dataset_file and load_dataset_object decode through one line decoder,
    so they refuse in the same words.
  • Limits: 1 GiB per object (kMaxDatasetObjectBytes), 16 MiB per line.
  • The source's column order is kept on this path too: the CSV header, or the order the JSON
    Lines rows first write each key (ObjectLoad.KeepsTheSourceColumnOrder, added when #1303 was
    rebased onto #1299).

Measured by the PR on a 67 MiB CSV (1,405,770 rows, one node, over REST): 41 s, 1.7 GB of node
memory added, a longest owner-loop stall of 2.7 s, against 126 s, 5.8 GB and 8.5 s when the rows
were also copied into the replicated document. See
docs/data/add-dataset.md.

LIMIT pushed to the partitions; OFFSET and FETCH — #1299

  • FETCH {FIRST|NEXT} [n] {ROW|ROWS} ONLY is LIMIT n; OFFSET m takes ROW/ROWS. TOP n is
    still refused.
  • Over a partition-only table, a non-aggregating SELECT with a LIMIT sends offset + limit and
    its ORDER BY keys to each owner, which returns at most that many rows per partition. The
    coordinator still sorts, skips and cuts, so the answer equals the full scan's.
  • Over a replicated table without ORDER BY, rows past offset + limit are not copied out. An
    aggregate is never cut.

See docs/data/query.md and
docs/data/scatter-gather-select.md.

Source column order and typed columns — #1299 (#1021)

  • The order is recorded at load time under a new CRDT key cols:<table-id>, kept in table.json,
    migrated by migrate_to_replication and removed by drop_table.
  • Every summary carries column_order: "source" | "first_seen".
  • type is the JSON family (integer, number, string, boolean, object, array, null,
    mixed, unknown). A partition-only table is profiled from the column_stats that
    write_partitioned now records; a partition written before has null_fraction null.

See docs/data/column-order.md.

Categorical features and catalog typing — #1297 (#1203)

  • A text column is a categorical feature when every value is text, it has 2 to 32 levels, at most
    half its values are distinct, and the values are not dates. Anything else stays excluded, and the
    run log says why.
  • Levels are fitted on training rows only (after the as_of cutoff). One-hot reads no label, so
    no target information leaks into a held-out fold.
  • feature_schema.categorical_encoding holds the fitted levels; feature_schema.inputs is the
    predict input contract by original column name.
  • resolve_dataset reads only the columns it cannot type from the catalog, projected with
    SELECT <columns>, and says in its first note where the types came from.

Measured by the PR on the 15,000-row demo table: resolve_dataset went from 86–88 ms to
34–39 ms; held-out AUC on the demo is unchanged at 0.731. On the getting-started churn data the
categories lift held-out AUC to 0.845, from 0.464 with the numeric column alone. See
docs/data/categorical-features.md and
docs/data/dataset-path.md.


Documentation

  • New: docs/data/column-order.md,
    docs/data/categorical-features.md.
  • Updated: add-dataset.md, object-store.md, query.md, scatter-gather-select.md,
    system-datasets.md, dataset-path.md, partition-manifest.md, partitioned-write-path.md,
    the API pages and the code map. schemas/openapi.json,
    schemas/partition_scan_request.schema.json and schemas/predict_request.schema.json are
    regenerated.
  • v0.3.0 records its image digest (#1295).

Not included

  • Other file formats from an object: Excel (#1307), Parquet one row group at a time (#1306),
    a JSON array (#1305), gzip and other delimiters (#1304).
  • LIMIT on the dataset-typing read: resolve_dataset still scans the table and cuts the
    typing sample to 256 rows in memory; appending LIMIT to that read is a one-line follow-up now
    that #1299 pushes it down.
  • Class imbalance warnings and weighting (#1298).
  • No GPU build, real-checkpoint or external-engine test was run, and the Linux test suite was not
    run (see Verification).

Where it has run

This release has not run on a cluster. v0.3.0 (main-d7551c75) was never rolled either: on
2026-10-19 every Zetta on the development cluster still runs v0.2.0. The new capabilities were
exercised by the test suite only, and #1303's 67 MiB load by its author on one local node.

The release image is oramasearch/zetta:v0.4.0, the same digest as main-104e8a98
(sha256:73e613748c1d6114a31fd8c462b55a09109de1b7c73323042429cb8a8509e1a9), built from a detached checkout of the tag on the cluster BuildKit (linux/amd64, Ubuntu
24.04, Clang 19, zetta target only, no tests) with the same compiler warning as on macOS. The Docker
Hub repository answers 404 to an anonymous request. Rolling it is a separate step,
through the operator's default Zetta image, and needs Steve's go-ahead.


Verification

Full test suite, run serially (ctest --test-dir build, no -j) on the release branch,
macOS 14 arm64, Homebrew LLVM Clang, s...

Read more

Release v0.3.0

Choose a tag to compare

@homer6 homer6 released this 17 Oct 23:51
d7551c7

v0.3.0

Released: 2026-10-17 · Branch: dev · Previous: v0.2.0 (2026-10-12)

v0.2.0 was the engine a hosted workspace runs on. v0.3.0 is its part of secure
networking (orama-bridge#186):
a workspace Zetta now keeps the workspace's network history, gives each app connection its
own scoped credential, and joins Zetta nodes in separate private networks through
zetta-tunnel peer gateways. The mutual-TLS and framing code the cluster uses is now a
library that zetta-tunnel links too.

Seven pull requests since v0.2.0: four features (#1284, #1285, #1286, #1287), a test
fixture fix (#1293), the release procedure (#1278), and the merge of main back into
dev after v0.2.0 (#1279).
The generated OpenAPI has 176 /v1 paths (was 172), and the test sources hold 2,992
test cases (was 2,977).


Behaviour changes

There is no behaviour change that a v0.2.0 caller has to adapt to. Two things are new at
startup and in the permission vocabulary:

  • A node declares four network record collections when it starts, and refuses to start
    if it cannot (zetta: cannot initialize network record collections: ...). The
    declarations are idempotent and byte-identical on every node.
  • Two permission atoms are added, network:read and network:write. The canonical
    vocabulary has 34 atoms (was 32). No built-in role is changed.

The cluster's inter-node protocol is unchanged: the handshake order, wire bytes, frame
bound and plaintext mode are the same as in v0.2.0 (#1284). A cluster with nodes of both
versions was not tested.


Added

Workspace network records — #1281

The records people inspect about a workspace's network live in that workspace's Zetta;
Lake's Postgres keeps only the desired state the operator enforces.

Record type Collection Retention
connection network_connections 365 days
grant network_grants 365 days
tunnel_session network_tunnel_sessions 90 days
activity network_activity 30 days
  • write_network_record (REST and MCP) takes the producer's record as it is: Lake's
    connection and grant lifecycle snapshots, and zetta-tunnel's relay session, refusal and
    egress, peer, ingress and mesh decision records. It waits for the owner's fsync and for
    every reachable live peer to install the write, and needs network:write.
  • list_network_records reads a type newest first, or by one of its indexes (event,
    workspace, ids, role, decision, rule), with a page cursor. It needs network:read.
  • A tunnel session's start and end share one key, so the end replaces the live row with
    its final byte totals. Other records are keyed by the SHA-256 of their canonical JSON,
    so an identical retry is idempotent.
  • The JSON format is zetta.NetworkRecord version 1, published as
    schemas/network_record.schema.json. Unknown fields and malformed identifiers are
    refused before storage.
  • Neither permission grants the generic record API.

See docs/network/network-records.md.

Scoped app credentials — #1280

An app reaching a workspace Zetta no longer inherits workspace-wide access.

  • issue_app_credential turns a connection's credential_scope (read or write;
    a database, tables, topic prefixes) into a connection-owned user and least-privilege
    role, and returns a bearer credential once. Only its salted hash is stored. A second
    live credential for the same connection is a conflict. It needs tokens:manage.
  • revoke_app_credential revokes every token of that connection and removes its role
    assignment. Repeating it answers revoked: 0, so an operator can reconcile absence.
  • read grants SQL reads on only the named database or tables, and topic reads and
    receives on only the named prefixes. write adds SQL writes and publish, update and
    delete-event rights on the same resources. Neither grants DDL, user or token
    administration, topic creation or deletion, or network records.
  • Every use goes through the existing registry checks and audit trail; revocation fails
    authentication immediately.

See docs/network/scoped-app-access.md.

Zetta peer links through peer gateways — #1282

Zetta nodes in separate private networks form one cluster through zetta-tunnel peer
gateways. A node points --peer at its local import listener and runs --dial-out-only,
so it advertises no private address that could collide with another cluster's pod range.
Zetta's own mutual TLS 1.3 stays end to end: the gateways copy bytes and never hold the
join token.

PeerGateway.OverlappingPrivateNodesReplicateThroughOpaqueGateways proves it with two
nodes on the same modelled private address and two opaque forwarders: one cluster forms,
and CRDT writes on either node converge. See
docs/network/peer-gateway-links.md.

The transport library — #1283

zetta::transport (zetta_transport) is the cluster's mutual-TLS and framing code as a
library of its own:

  • PEM credential loading, and checks before a connection starts: the key matches the
    leaf, the leaf verifies against the CA bundle now, and no system trust anchor is added;
  • a Corosio context that is TLS 1.3 only and verifies the peer in both roles;
  • FramedStream, newline-framed reads and writes with a frame bound.

-DZETTA_TRANSPORT_ONLY=ON configures only the library and its direct dependencies (Capy,
Corosio, OpenSSL), static or shared, so zetta-tunnel can add_subdirectory it. Zetta's
cluster uses the same target. See
docs/cluster/transport-library.md.

The version, held by a test

project(VERSION) in CMakeLists.txt now says the release version (it said 0.0.1, a
known issue of v0.2.0). tests/test_release.cpp reads kVersion from
include/zetta/version.hpp and checks that it equals the CMake project version, the
committed schemas/openapi.json info.version, and that this page exists and is listed
in the index.


Fixed

A test crashed on a race its fixture created — #1292

RestClusterFixture.RunReportAnswersForADatasetlessRun ended in a segmentation fault or an
abort now and then. Under AddressSanitizer it was a heap-use-after-free: the churn run's
thread appended to the Run's log while the test thread serialized the Run. The fixture
never ticked its node, and a node that no loop has ticked runs owner work inline on the
calling thread. The fixture now ticks once after attaching its cluster, as both apps do
from their first loop iteration (#1293). Test-only: production nodes were not
affected.
Under ASan with 8 parallel copies it crashed once in 80 runs before, and
0 times in 200 after.

A documentation drift check skipped silently

CliDocs.EveryFlagMainOffersIsDocumented found src/main.cpp only through an environment
variable or the working directory, and under ctest the working directory is the build
tree, so it skipped. It now falls back to the source directory the tests were built from.


Documentation


Not included

  • zetta-tunnel itself: its executable, gateway authorization, gateway certificates and
    readiness live in zetta-tunnel. This
    release gives it the library, the record API and the peer contract.
  • Projecting the app credential to the app and reconciling connections: the operator
    and Lake (orama-bridge#186).
  • Moving Lake's users and tokens into workspace Zettas
    (#1172).
  • Clustered pub/sub (#1038).
  • No GPU build, real-checkpoint or external-engine test was run, and the Linux test suite
    was not run (see Verification).

Where it has run

Before this tag, the post-v0.2.0 tree has not run on a cluster. On 2026-10-17 every
Zetta on the development cluster (orama-production-cluster-01: the Lake bus
lake/lake-prod-zetta-zetta-0 and four workspace Zetta pods) runs
oramasearch/zetta:main-4015ef76, which is v0.2.0. The new capabilities were exercised
by the test suite only, in-process, with real Corosio TCP and mutual TLS between nodes.

The release image is oramasearch/zetta:v0.3.0 (the same digest as main-d7551c75,
sha256:0dad6a7fba1842a1406a3430394fd98a06d3241d9989a69470f6ad921a54328a). Rolling it to the cluster is a separate step, through the
operator's default Zetta image, and needs Steve's go-ahead.


Verification

Full test suite, run serially (ctest --test-dir build, no -j), macOS 14 with 16
hardware threads, Release builds, while other builds kept the host's load average between
5 and 30. Run 3 is the release commit; runs 1, 2 and the shared run are the same tree
without the fixture fix (#1293) and, for run 1, without the CliDocs fix.

Run Build Tests Passed Failed Skipped Disabled Time
1 static 2,888 2,870 1 17 1 880 s
...
Read more

Release v0.2.0

Choose a tag to compare

@homer6 homer6 released this 12 Oct 19:41
4015ef7

Released: 2026-10-12 · Branch: dev · Previous: v0.1.1 (2026-08-27)

v0.1.1 was one methodology (churn) on one node API. v0.2.0 is the engine a hosted
workspace runs on: 21 methodologies behind one registry, composed solutions that decide and
measure, AutoML studies, pub/sub inside the node, a keyed record store and an object store
for workspace data, agents described by records, and an audit trail that no longer lives in
the replicated catalogue. Every change here was already running on the development cluster
as oramasearch/zetta:dev-2cfdac21 before this release was cut.

236 commits across 63 pull requests, 750 files. The generated OpenAPI has 172 /v1
paths (was 78), and the test sources hold 2,977 test cases (was 2,119).


⚠️ Behaviour change: upgrade every node of a cluster together

Audit events moved out of the replicated catalogue into per-origin segments
(#1164). Mixed versions deliberately do not converge audit history. Stop the old
writers, upgrade every node, then restart the cluster.

  • On first start the node migrates the audit events it finds in the old catalogue into its
    own segments, under the reserved origin legacy. The migration is restartable, and a
    failure leaves the old persisted catalogue in place.
  • An unreadable store or a failed migration prevents startup. That is deliberate: a node
    that started anyway would drop history.
  • Valid legacy audit/<uuid>.json files are folded in and removed; invalid ones stay where
    they are and are reported.

There is no flag that restores the old layout. See
docs/governance/audit-trail.md, "Replication and upgrade
policy".

⚠️ Behaviour change: every SQL statement that changes data checks its right

Before this release only INSERT was authorized in the executor (#1229). Now:

Statement Right needed
UPDATE, DELETE, TRUNCATE sql:write, at table or database scope
CREATE / DROP DATABASE and TABLE sql:ddl, at table or database scope
CREATE VIRTUAL TABLE flat sql:ddl
CREATE USER, CREATE TOKEN users:manage, tokens:manage

Column grants and row-level security apply to UPDATE and DELETE; TRUNCATE of a
policy-protected table is refused; a legacy AccessKey needs the flat right. Each such
statement lands on the audit trail as the new data_changed action.

A principal that could run these statements with read rights is now refused. Grant the
right; there is no setting that turns the check off.

⚠️ Behaviour change: a gate check that did not apply says not_applicable

A gate check with nothing to check on a graph used to answer pass, so a churn report read
as about 30 passes that verified nothing (#1240). CheckStatus gains not_applicable,
which never moves the verdict. The report's JSON now lists only the checks that applied,
with the counts applied and not_applicable. A caller that counted pass entries, or
expected every check in the list, sees fewer. Pass include_not_applicable to the gate
capability to get the full list back.

⚠️ Behaviour change: smaller ones that can break a caller

  • A compiled pipeline is named from its intent (#1238), not churn skeleton or
    <methodology> pipeline. A second compile in the same project is numbered (2) instead
    of failing. Pass name to compile_intent to keep a fixed name; the two delivery
    runbooks now do.
  • create_agent and update_agent take a closed definition (#1220): unknown
    properties are refused at the property. A file sent as {path, content} is refused
    instead of being stored empty; the field is contents.
  • upload_dataset refuses a taken name with conflict (409) unless if_exists is
    "rename", and the CSV decoder refuses ragged rows, unnamed or repeated header columns
    and open quotes, naming the line, instead of padding or truncating (#1221).
  • A SQL column that no row carries is an error naming the column and the table's
    columns (#1218).
  • A study whose trials all fail ends failed, with the last trial's reason, instead of
    complete with an empty reason (#1217). A positive cost_cap under 60 trial-seconds is
    rejected at /budgets/cost_cap, because it could never admit a trial (#1211).
  • A request no route matches gets an answer (#1011): 405 method_not_allowed for a
    capability called with the wrong method, 404 unknown_capability or 404 not_found
    otherwise. It used to get no bytes at all.
  • delete_dataset refuses while a pending or running run, or a running study, reads the
    dataset
    (#1258), naming each one.

Added

The methodology registry: 21 methodologies, one way to compile and run them

compile_intent and run route through a registry instead of a churn-only path.
Methodologies M-1 to M-21 are registered and runnable: churn classification, forecasting,
segmentation, anomaly detection, entity resolution (distributed across members), survival,
next-best-action, causal uplift, LambdaMART ranking, language, document extraction, graph,
vision, speech, bandits, optimization, Bayesian uncertainty, out-of-fold stacking, label
efficiency, synthetic data with differentially private GBDT, and online streaming learning.

  • analyze_dataset and plan_features describe a dataset and plan its features. The plan
    is enforced for all 21 methodologies
    , and a run reports the provenance of what it used.
  • Text entity keys and boolean columns are accepted, and the original entity ids come back
    in every output. This closes the v0.1.1 known issue
    #957.
  • A workspace can register its own methodology (register_methodology,
    update_methodology, delete_methodology, record_methodology_run), run by Lake's
    runner (#1261, lake#655).

Solutions: from a score to a decision and a readout

A composed solution registry (create_solution, run_solution) executes through governed
foundation stages. Around it: a replicated holdout experiment manager, a versioned decision
policy engine (create_policy, decide, approve_decision), a replicated audit ledger
(list_ledger), a drift and guardrail monitor (get_monitor), point-in-time snapshots
(snapshot), action delivery (enqueue_action_task, deliver_action_task,
record_outcome) and the ROI readout (get_roi_readout).

Runnable: S-1 churn, S-2 lead scoring, S-3 lifetime value, S-4 collections, S-5 demand
forecasting, S-6 replenishment, S-7 KPI anomaly, predictive maintenance, S-9 automated
outbound, S-10 fraud decisioning, S-11 next-best-action, S-12 claims; and the modules B-1
dynamic pricing, B-2 employee attrition, B-3 ticket triage, B-4 document workflows, B-5
supplier risk, B-6 healthcare operations, B-7 credit underwriting and B-8 security
operations.

Studies, run metrics and models

  • Run-backed AutoML studies: create_study, validate_study, list_trials,
    promote_trial, stop_study, stop_trial, get_study_analysis, delete_study. Trials
    run through the background training driver, under a cost cap in trial-seconds.
  • Experiments over a workspace methodology's parameters, with trials run by Lake's
    runner (lake#689).
  • Durable run metrics: log_run_metrics, log_run_params, log_run_artifact,
    get_run_metrics, compare_runs. A GBDT run logs its held-out scores, confusion matrix,
    ROC and calibration on the Run (#1239).
  • One metric name. The run report's key (held_out_auc, held_out_log_loss,
    held_out_accuracy, held_out_rmse, held_out_mae) is what a study objective names; the
    old spellings (validation.auc, ROC-AUC, auc) are accepted as aliases.
  • Standalone models: train_model, retrain_model, register_model and
    move_model_to_project work without a project
    (lake#654).
  • Bring a model exactly: ONNX, XGBoost and LightGBM models are imported as they are, and
    export_model writes one out (lake#681).
  • Large artifacts move in chunks: a methodology run's artifact over 24 MiB is appended
    with append_methodology_artifact, and read_model_weights reads weights back in chunks
    (lake#690).

Pub/sub inside the node

Stage 1 of the design recorded as decisions D5 to D39 under
docs/plans/pubsub/: typed addresses, an email-style envelope, ring
buffers with a disk tier, topics, subscriptions, inboxes, governance bounds, limits, and
best-effort survival across a restart.

  • create_topic, update_topic, drop_topic, list_topics, get_topic, browse_topic,
    publish_event, receive_events, update_event, delete_events, over REST and MCP with
    parity.
  • A long poll suspends without holding a thread, and the node keeps accepting connections
    while polls are held.
  • Stage 1 is one node. Topic definitions do not replicate
    (#1038).

A keyed record store, an object store, and POST sql

  • Records (put_record, get_record, list_records, delete_record,
    create_collection) with conditional writes and secondary-index lists, and labelled time
    series (create_metric, record_metric, query_metric) (#1231).
  • Records live in a Bitcask-style log per node under <data_dir>/records/, not in the
    replicated catalogue. The first version kept them in the catalogue: after 200,000 puts
    with three live keys the saved document was 141 MiB and the process held 5.2 GB. The soak
    that found it is now a test: 1,000,000 puts ...
Read more

Release v0.1.1

Choose a tag to compare

@homer6 homer6 released this 27 Aug 15:37
12d66bb

Released: 2026-08-27 · Branch: dev · Previous: v0.1.0 (2026-08-12)

A repair release. v0.1.0 shipped a Postgres source connector that could not be used —
three independent defects sat on top of each other, and each was only visible once the one
above it stopped refusing the call. Everything here came out of running the delivery
runbooks end to end against a real database rather than reading them.

10 commits, 40 files, 76 capabilities (was 75).


⚠️ Behaviour change: consent enforcement is OFF by default

NodeConfig::enforce_consent now defaults to false. A requires_consent capability
— create_dataset_from_query, cloud_inspect, detect_provisioners, a cloud add_nodes
— executes on the first call instead of returning a consent challenge.

Ungated is not unrecorded. Each such call lands on the audit trail as a new
consent_bypassed event carrying the summary the approver would have read, plus a
consent_enforced=false span attribute. "What touched a cloud on this node?" is still
answerable.

To keep the old behaviour, start the node with --governance (or ZETTA_GOVERNANCE=1).

Why: the protocol was correct and its approval surfaces were not reachable. The refusal
pointed at the web console's /governance page, which has no consent list; the desktop
app's approval surface is on Admin & audit, not Governance; and a one-person node has
no second principal, so a challenge could be issued and then decided by nobody. The gate
blocked the action instead of gating it. Turning it back on is tracked by
#999, which lists what has to ship first.

The desktop app's own cloud gates are unaffected and remain unconditional — those are a
human clicking in front of the machine.


Fixed

create_dataset_from_query had never worked on Postgres — #997

run_distributed_connector_ingest submitted every connector config through Snowflake's
SQL API v2 regardless of type, so a Postgres connection built a Snowflake request out of
its empty account and failed with snowflake: could not resolve .snowflakecomputing.com
— a vendor the caller never named, for a database on 127.0.0.1.

Tiling a result set needs a result manifest (statementHandle + partitionInfo[]) to
weight the fan-out; Postgres has no equivalent. A source without one now takes a single-shot
path that lands the same dataset shape — same scheduler, partition count, replication
factor, partition_only marker and fetch-affinity placement. It fans out over cores and
disks but not over uplinks, which is a limit of a source that returns one stream.

The ranked at-risk list did not join back to its own data — #1000

row.value("account", 0) deduces int from the literal, so entity keys above 2³¹ were
narrowed to 32 bits. get_run_report disagreed with score and with the node's own
churn_results.at_risk_* table for the same run:

Surface rank 2 entity risk
get_run_report -1949009714 0.8807439523346903
score / SQL 2345957582 0.8807439523346903

A derived integer key — the documented remedy for a string business id — lands above 2³¹
about half the time, so this fired on the first realistic dataset. Fixed in the report
builder and in the chat agent's grounding text.

REST dropped error.details — #998

The structured failure payload rode the MCP envelope and was dropped from the REST one, so
a consent_required refusal over /v1 carried a challenge id in prose and no challenge.
Now emitted (absent, not null, when there is none) and documented in error_envelope_schema.

Build warnings — #992

Found and first fixed by Charles Philippe in
PR #993, opened 2026-08-18. That branch
was closed as superseded by a5c7888, which was written independently eight days later
and reaches the same five fixes across the same eight files. The history therefore does
not carry his commit; the credit is his.

tests/api/test_clerk_auth.cpp did not compile (OpenSSL::Crypto linked PRIVATE in
zetta-core never reached zetta-tests); eight web-UI handlers discarded push_to's
io_result, so a body-stream failure was handled as "no fields supplied"; two
async_event::wait() results were discarded; four switches missed JobKind::Ingest,
including the GUI label, which showed "Compute job" for an ingest. zetta and zetta-tests
now build with zero warnings.


Added

list_source_tables — browse a source without ingesting from it

Finding a table required ingesting one: the only way to answer "what is in this database?"
was create_dataset_from_query against information_schema, which materializes a
partitioned, replicated dataset to read a catalog and leaves it behind.

A Read — one connection, one catalog query, nothing landed:

curl -s -G -H "Authorization: Bearer $TOK" "$ZETTA/v1/list_source_tables" \
  --data-urlencode "connection=local-pg" --data-urlencode "match=score"

Returns each table with its columns and types. match is a case-insensitive substring;
schema= narrows; include_columns=false returns names only. Columns come from one second
catalog query rather than one per table, so a 200-table schema is two round trips.

model_from_source MCP prompt

The node advertised exactly one prompt, build_and_run, which drives agent_run and
therefore cannot run on a node with no agent installed — which is every fresh node
(#977). The only guided path a new node
offered was the one it could not execute.

model_from_source is the deterministic path end to end and needs no agent or checkpoint.
It carries the three column rules — integer entity key, numeric {0,1} label, numeric
features
— at the step where they are still fixable, rather than leaving them to be
discovered from a mid-run failure inside load_data_frame.

Consent challenges name the real connector

create_dataset_from_query had no ConsentDescriber, so its challenge described every
source as Snowflake whatever the connector's type was. It now reads the resolved connector
(postgres db.example.com:5432/motive) and lists the caller's own query as what would run.
A loopback Postgres is not a cloud and is not gated even with --governance on
(#994).


Documentation

Two delivery runbooks — from-clone-to-deployed-model.md
(customer-neutral) and the Motive-specific one — were dry-run end to end and did not survive
contact, independently of the bugs above:

  • The call the connector section exists for was missing entirely.
  • The read-back example used LIMIT, which the node's SQL subset does not support.
  • A trust-auth Postgres cannot connect (token is required even when empty).
  • The documented run example used a text account_id and a boolean churned — which
    is what a real accounts table has, and which both fail.
  • POSTing a read returns an empty body that reads as a hung node
    (#954).

All corrected, with a cast recipe and a measured query. Every $ZETTA/v1/… curl in both
files is checked against the node's own OpenAPI for the correct HTTP method.


Verification

Driven over MCP the way a coding agent would — tools/call only, no human correction:

create_connection · list_source_tables · create_dataset_from_query · create_project
capture_intent · compile_intent · approve_plan · gate(pass) · run · get_run(succeeded)
get_run_report(held_out_auc 0.734) · score · explain(5 contributions, residual 0.0)
report/score entity agreement                          === 14/14 steps passed ===

Also over REST against a real Postgres 17: 800 accounts across four raw tables, one
feature-engineering SELECT, held-out AUC 0.769 (in-sample 0.966), additive-exact TreeSHAP
(residual 8.9e-16).

Tests: 2060/2060.


Known issues

  • #999 — there is still no reachable
    consent approval surface. --governance reproduces the original wall on a one-person
    node. This gates returning the default to true.
  • #1001 — the Snowflake connector is
    PAT/OAuth only. An account that authenticates with an RSA key pair cannot be reached
    at all; enterprise service accounts commonly permit key-pair only.
  • #957 — a non-numeric label fails
    late, mid-run, inside load_data_frame with a message that names no remedy. The entity
    column's equivalent failure is immediate and does name one.
  • Only Snowflake ingests with the cluster's aggregate bandwidth. A Postgres pull is one
    node reading one stream and then partitioning it.
  • ClusterSecurity.MeshConvergesOverMutualTlsUnderSustainedLoad is load-sensitive under
    parallel ctest and passes on an idle machine
    (#786 has WIP).

Upgrading

Nothing to migrate — no schema, storage or wire-format change. The one thing to decide is
whether you want --governance. If you relied on cloud-touching calls being refused by
default, set it explicitly; otherwise those calls now execute and audit as
`consent...

Read more

Release v0.1.0

Choose a tag to compare

@homer6 homer6 released this 12 Aug 19:23
b54c008

Released: 2026-08-12 · Commit: b54c008 · The first tagged release of the engine

1,082 commits. One statically-linked C++26 binary that is both the database node and,
headed, its own UI. 75 capabilities on a single registry rendered three ways — REST, MCP
tools, and the OpenAPI document — so the three cannot drift.

These notes were written retroactively (2026-08-27). Every number below is a figure
measured against a running v0.1.0 node while writing
from-clone-to-deployed-model.md, not a
claim read out of a design document. Where v0.1.0 does not do something, it says so.


What it does

Intent → Pipeline → Verification → Run → ranked scores with reasons, on infrastructure
you control, with the model trained natively on the node rather than prompted from a rented
one.

create_project → capture_intent → compile_intent → approve_plan → gate → run
              → get_run_report → score → explain

Six calls to a trained model, and every one is on both surfaces — one capability
registry renders the REST route and the MCP tool from a single entry.

The engine underneath

  • Equal-peer, CRDT-replicated store. A from-scratch C++ Automerge; the document is the
    source of truth and the on-disk store is a projection of it. No leader, no election.
  • Native distributed ML. Histogram-allreduce GBDT, native TreeSHAP, point-in-time
    featurize running compute-near-data over owned partitions. No Python, no XGBoost, no
    model API.
  • Partitioned datasets with a scheduler that targets a partition size rather than a
    constant count, and fetch-affinity placement so a slice is not shuffled after it lands.
  • Source connectors — Snowflake (SQL API v2) and Postgres — plus local files (CSV,
    Parquet, JSONL, JSON).
  • Distributed full-text search by per-partition scatter-gather.
  • Two run modes from one binary: headed (SDL3 + Dear ImGui; the desktop app is itself a
    cluster member) and headless.
  • Governance: default-deny access keys, an immutable append-only audit trail, and
    system.* read views computed at query time from the live source.
  • Observability: every operation opens a span; a caller gets a trace id back and can
    read the whole trace.
  • Auth is always on — a native bearer/session token, or Clerk JWTs verified against
    JWKS when CLERK_ISSUER is set. Inter-node traffic is always mTLS; there is no
    plaintext mode.

Measured on a running node

The same run over MCP and over REST returned held_out_auc: 0.7911784511784512 — the
same float to the last digit. MCP additionally returns a typed structuredContent block
and surfaces failures as isError: true with the same structured error code REST returns.
The engine is exactly as deterministic over MCP; what differs is who composes the calls.

compile_intent produced a 7-step Pipeline, 6 edges, from one sentence of plain
language:

connect  Load customer accounts        model  Train churn-risk model
connect  Load usage & billing activity index  Index churn scores
clean    Build churn features          serve  Serve churn scores & reasons
                                       act    Trigger retention outreach

Verification: verdict pass, block_deploy: false, four checks.

A churn model on 600 accounts (38.3% churn), holdout 0.3:

held-out AUC 0.791 (174 rows)
in-sample AUC 0.940 (426 rows)
held-out accuracy 0.707
majority-class accuracy 0.617 — what a constant answer scores
top drivers failed_payments 105.2 · seats 39.6 · total_events 29.1 · mrr 26.2

Both figures are reported and labelled. The gap between them is the overfitting signal
and is only readable when both are present — quote the held-out one.

explain returns a signed contribution per feature, strongest first (failed_payments
+2.82, total_events +1.22, the rest 0.0), a base_value, and the resulting risk 0.983
— the per-account answer that score's population drivers cannot give.

The report says when not to trust it. On the 24-row demo dataset it returns a caveats
block: "measured on only 4 held-out rows. Below about 30 a ranking metric is noise … Do
not quote it as a result."
On the 600-row run, caveats was empty.


What v0.1.0 does not include

Stated here because an evaluator will otherwise find them.

  • No conversational agent. No checkpoint ships and a fresh node has zero agents
    (#977) — Studio's Chat screen and the
    Ask Orama rail have nothing behind them. The whole path above is the deterministic
    capability path and needs no model, which is also why it is the reliable one. Defects
    confined to the conversational surface:
    #975 (POST /v1/chat hangs over REST
    while MCP is fine), #976,
    #979.
  • Discover Opportunities is a dead end in the UI — the card offers something you cannot
    begin, because the Workspace Profile screen does not exist.
  • One methodology is executable. The compiler produces the tabular-supervised shape
    (M-1) plus part of the NLP/embeddings shape; the other nineteen in the catalogue are
    specified and not built (#984). Churn
    is M-1. A forecasting or uplift ask is outside the built surface and fails silently —
    a plausible M-1 skeleton with the nouns swapped — rather than loudly.
  • In flight, not shipped: tiered storage, compute-near-data via embedded Lua/Orly and
    the DDF API, the Postgres wire protocol, and the learning loop.

Known defects at time of tagging

Found immediately after, while running the delivery runbooks against a real database. All
are fixed in v0.1.1 — do not evaluate the connector path on v0.1.0:

  • create_dataset_from_query has never worked on Postgres. Every connector config went
    through Snowflake's protocol, so a Postgres connection failed with snowflake: could not resolve .snowflakecomputing.com (#997).
  • The consent gate has no reachable approver, so a query against a local Postgres was
    refused with no way to approve it (#995).
  • The ranked at-risk list does not join back to its own data — entity ids above 2³¹ were
    wrapped to 32 bits (#1000).
  • tests/api/test_clerk_auth.cpp does not compile on a clean clone
    (#992).

A version string that disagrees with its tag

A v0.1.0 binary introduces itself as zetta 0.0.1 — kVersion was never moved off
its pre-release value. Corrected in v0.1.1.


Running it

git submodule update --init --recursive
./scripts/install-deps.sh
cmake -S . -B build -G Ninja && cmake --build build
./build/bin/zetta --headless --data-dir ~/zetta-data

Needs CMake 3.28+/Ninja and a C++26 Clang 17+. modules/libpqxx must be initialized or
the Postgres connector is not in the binary
— the configure log says which. Two flags are
routinely confused: --listen binds the cluster peer port (7777); --mcp-listen binds
the API, MCP and web UI (127.0.0.1:4800).

The node documents itself at /cli.md, /mcp.md and /v1/openapi.json, all readable
without a token.

Full path from a clean clone to a deployed model:
from-clone-to-deployed-model.md.