Skip to content

Release v0.1.1

Choose a tag to compare

@homer6 homer6 released this 27 Aug 15:37
· 659 commits to main since this release
12d66bb

Released: 2026-08-27 · Branch: dev · Previous: v0.1.0 (2026-08-12)

A repair release. v0.1.0 shipped a Postgres source connector that could not be used —
three independent defects sat on top of each other, and each was only visible once the one
above it stopped refusing the call. Everything here came out of running the delivery
runbooks end to end against a real database rather than reading them.

10 commits, 40 files, 76 capabilities (was 75).


⚠️ Behaviour change: consent enforcement is OFF by default

NodeConfig::enforce_consent now defaults to false. A requires_consent capability
— create_dataset_from_query, cloud_inspect, detect_provisioners, a cloud add_nodes
— executes on the first call instead of returning a consent challenge.

Ungated is not unrecorded. Each such call lands on the audit trail as a new
consent_bypassed event carrying the summary the approver would have read, plus a
consent_enforced=false span attribute. "What touched a cloud on this node?" is still
answerable.

To keep the old behaviour, start the node with --governance (or ZETTA_GOVERNANCE=1).

Why: the protocol was correct and its approval surfaces were not reachable. The refusal
pointed at the web console's /governance page, which has no consent list; the desktop
app's approval surface is on Admin & audit, not Governance; and a one-person node has
no second principal, so a challenge could be issued and then decided by nobody. The gate
blocked the action instead of gating it. Turning it back on is tracked by
#999, which lists what has to ship first.

The desktop app's own cloud gates are unaffected and remain unconditional — those are a
human clicking in front of the machine.


Fixed

create_dataset_from_query had never worked on Postgres — #997

run_distributed_connector_ingest submitted every connector config through Snowflake's
SQL API v2 regardless of type, so a Postgres connection built a Snowflake request out of
its empty account and failed with snowflake: could not resolve .snowflakecomputing.com
— a vendor the caller never named, for a database on 127.0.0.1.

Tiling a result set needs a result manifest (statementHandle + partitionInfo[]) to
weight the fan-out; Postgres has no equivalent. A source without one now takes a single-shot
path that lands the same dataset shape — same scheduler, partition count, replication
factor, partition_only marker and fetch-affinity placement. It fans out over cores and
disks but not over uplinks, which is a limit of a source that returns one stream.

The ranked at-risk list did not join back to its own data — #1000

row.value("account", 0) deduces int from the literal, so entity keys above 2³¹ were
narrowed to 32 bits. get_run_report disagreed with score and with the node's own
churn_results.at_risk_* table for the same run:

Surface rank 2 entity risk
get_run_report -1949009714 0.8807439523346903
score / SQL 2345957582 0.8807439523346903

A derived integer key — the documented remedy for a string business id — lands above 2³¹
about half the time, so this fired on the first realistic dataset. Fixed in the report
builder and in the chat agent's grounding text.

REST dropped error.details — #998

The structured failure payload rode the MCP envelope and was dropped from the REST one, so
a consent_required refusal over /v1 carried a challenge id in prose and no challenge.
Now emitted (absent, not null, when there is none) and documented in error_envelope_schema.

Build warnings — #992

Found and first fixed by Charles Philippe in
PR #993, opened 2026-08-18. That branch
was closed as superseded by a5c7888, which was written independently eight days later
and reaches the same five fixes across the same eight files. The history therefore does
not carry his commit; the credit is his.

tests/api/test_clerk_auth.cpp did not compile (OpenSSL::Crypto linked PRIVATE in
zetta-core never reached zetta-tests); eight web-UI handlers discarded push_to's
io_result, so a body-stream failure was handled as "no fields supplied"; two
async_event::wait() results were discarded; four switches missed JobKind::Ingest,
including the GUI label, which showed "Compute job" for an ingest. zetta and zetta-tests
now build with zero warnings.


Added

list_source_tables — browse a source without ingesting from it

Finding a table required ingesting one: the only way to answer "what is in this database?"
was create_dataset_from_query against information_schema, which materializes a
partitioned, replicated dataset to read a catalog and leaves it behind.

A Read — one connection, one catalog query, nothing landed:

curl -s -G -H "Authorization: Bearer $TOK" "$ZETTA/v1/list_source_tables" \
  --data-urlencode "connection=local-pg" --data-urlencode "match=score"

Returns each table with its columns and types. match is a case-insensitive substring;
schema= narrows; include_columns=false returns names only. Columns come from one second
catalog query rather than one per table, so a 200-table schema is two round trips.

model_from_source MCP prompt

The node advertised exactly one prompt, build_and_run, which drives agent_run and
therefore cannot run on a node with no agent installed — which is every fresh node
(#977). The only guided path a new node
offered was the one it could not execute.

model_from_source is the deterministic path end to end and needs no agent or checkpoint.
It carries the three column rules — integer entity key, numeric {0,1} label, numeric
features
— at the step where they are still fixable, rather than leaving them to be
discovered from a mid-run failure inside load_data_frame.

Consent challenges name the real connector

create_dataset_from_query had no ConsentDescriber, so its challenge described every
source as Snowflake whatever the connector's type was. It now reads the resolved connector
(postgres db.example.com:5432/motive) and lists the caller's own query as what would run.
A loopback Postgres is not a cloud and is not gated even with --governance on
(#994).


Documentation

Two delivery runbooks — from-clone-to-deployed-model.md
(customer-neutral) and the Motive-specific one — were dry-run end to end and did not survive
contact, independently of the bugs above:

  • The call the connector section exists for was missing entirely.
  • The read-back example used LIMIT, which the node's SQL subset does not support.
  • A trust-auth Postgres cannot connect (token is required even when empty).
  • The documented run example used a text account_id and a boolean churned — which
    is what a real accounts table has, and which both fail.
  • POSTing a read returns an empty body that reads as a hung node
    (#954).

All corrected, with a cast recipe and a measured query. Every $ZETTA/v1/… curl in both
files is checked against the node's own OpenAPI for the correct HTTP method.


Verification

Driven over MCP the way a coding agent would — tools/call only, no human correction:

create_connection · list_source_tables · create_dataset_from_query · create_project
capture_intent · compile_intent · approve_plan · gate(pass) · run · get_run(succeeded)
get_run_report(held_out_auc 0.734) · score · explain(5 contributions, residual 0.0)
report/score entity agreement                          === 14/14 steps passed ===

Also over REST against a real Postgres 17: 800 accounts across four raw tables, one
feature-engineering SELECT, held-out AUC 0.769 (in-sample 0.966), additive-exact TreeSHAP
(residual 8.9e-16).

Tests: 2060/2060.


Known issues

  • #999 — there is still no reachable
    consent approval surface. --governance reproduces the original wall on a one-person
    node. This gates returning the default to true.
  • #1001 — the Snowflake connector is
    PAT/OAuth only. An account that authenticates with an RSA key pair cannot be reached
    at all; enterprise service accounts commonly permit key-pair only.
  • #957 — a non-numeric label fails
    late, mid-run, inside load_data_frame with a message that names no remedy. The entity
    column's equivalent failure is immediate and does name one.
  • Only Snowflake ingests with the cluster's aggregate bandwidth. A Postgres pull is one
    node reading one stream and then partitioning it.
  • ClusterSecurity.MeshConvergesOverMutualTlsUnderSustainedLoad is load-sensitive under
    parallel ctest and passes on an idle machine
    (#786 has WIP).

Upgrading

Nothing to migrate — no schema, storage or wire-format change. The one thing to decide is
whether you want --governance. If you relied on cloud-touching calls being refused by
default, set it explicitly; otherwise those calls now execute and audit as
consent_bypassed.