Repository navigation
Release v0.3.0
v0.3.0
Released: 2026-10-17 · Branch: dev · Previous: v0.2.0 (2026-10-12)
v0.2.0 was the engine a hosted workspace runs on. v0.3.0 is its part of secure
networking (orama-bridge#186):
a workspace Zetta now keeps the workspace's network history, gives each app connection its
own scoped credential, and joins Zetta nodes in separate private networks through
zetta-tunnel peer gateways. The mutual-TLS and framing code the cluster uses is now a
library that zetta-tunnel links too.
Seven pull requests since v0.2.0: four features (#1284, #1285, #1286, #1287), a test
fixture fix (#1293), the release procedure (#1278), and the merge of main back into
dev after v0.2.0 (#1279).
The generated OpenAPI has 176 /v1 paths (was 172), and the test sources hold 2,992
test cases (was 2,977).
Behaviour changes
There is no behaviour change that a v0.2.0 caller has to adapt to. Two things are new at
startup and in the permission vocabulary:
- A node declares four network record collections when it starts, and refuses to start
if it cannot (zetta: cannot initialize network record collections: ...). The
declarations are idempotent and byte-identical on every node. - Two permission atoms are added,
network:readandnetwork:write. The canonical
vocabulary has 34 atoms (was 32). No built-in role is changed.
The cluster's inter-node protocol is unchanged: the handshake order, wire bytes, frame
bound and plaintext mode are the same as in v0.2.0 (#1284). A cluster with nodes of both
versions was not tested.
Added
Workspace network records — #1281
The records people inspect about a workspace's network live in that workspace's Zetta;
Lake's Postgres keeps only the desired state the operator enforces.
| Record type | Collection | Retention |
|---|---|---|
connection |
network_connections |
365 days |
grant |
network_grants |
365 days |
tunnel_session |
network_tunnel_sessions |
90 days |
activity |
network_activity |
30 days |
write_network_record(REST and MCP) takes the producer's record as it is: Lake's
connection and grant lifecycle snapshots, and zetta-tunnel's relay session, refusal and
egress, peer, ingress and mesh decision records. It waits for the owner's fsync and for
every reachable live peer to install the write, and needsnetwork:write.list_network_recordsreads a type newest first, or by one of its indexes (event,
workspace, ids, role, decision, rule), with a page cursor. It needsnetwork:read.- A tunnel session's start and end share one key, so the end replaces the live row with
its final byte totals. Other records are keyed by the SHA-256 of their canonical JSON,
so an identical retry is idempotent. - The JSON format is
zetta.NetworkRecordversion 1, published as
schemas/network_record.schema.json. Unknown fields and malformed identifiers are
refused before storage. - Neither permission grants the generic record API.
See docs/network/network-records.md.
Scoped app credentials — #1280
An app reaching a workspace Zetta no longer inherits workspace-wide access.
issue_app_credentialturns a connection'scredential_scope(readorwrite;
a database, tables, topic prefixes) into a connection-owned user and least-privilege
role, and returns a bearer credential once. Only its salted hash is stored. A second
live credential for the same connection is aconflict. It needstokens:manage.revoke_app_credentialrevokes every token of that connection and removes its role
assignment. Repeating it answersrevoked: 0, so an operator can reconcile absence.readgrants SQL reads on only the named database or tables, and topic reads and
receives on only the named prefixes.writeadds SQL writes and publish, update and
delete-event rights on the same resources. Neither grants DDL, user or token
administration, topic creation or deletion, or network records.- Every use goes through the existing registry checks and audit trail; revocation fails
authentication immediately.
See docs/network/scoped-app-access.md.
Zetta peer links through peer gateways — #1282
Zetta nodes in separate private networks form one cluster through zetta-tunnel peer
gateways. A node points --peer at its local import listener and runs --dial-out-only,
so it advertises no private address that could collide with another cluster's pod range.
Zetta's own mutual TLS 1.3 stays end to end: the gateways copy bytes and never hold the
join token.
PeerGateway.OverlappingPrivateNodesReplicateThroughOpaqueGateways proves it with two
nodes on the same modelled private address and two opaque forwarders: one cluster forms,
and CRDT writes on either node converge. See
docs/network/peer-gateway-links.md.
The transport library — #1283
zetta::transport (zetta_transport) is the cluster's mutual-TLS and framing code as a
library of its own:
- PEM credential loading, and checks before a connection starts: the key matches the
leaf, the leaf verifies against the CA bundle now, and no system trust anchor is added; - a Corosio context that is TLS 1.3 only and verifies the peer in both roles;
FramedStream, newline-framed reads and writes with a frame bound.
-DZETTA_TRANSPORT_ONLY=ON configures only the library and its direct dependencies (Capy,
Corosio, OpenSSL), static or shared, so zetta-tunnel can add_subdirectory it. Zetta's
cluster uses the same target. See
docs/cluster/transport-library.md.
The version, held by a test
project(VERSION) in CMakeLists.txt now says the release version (it said 0.0.1, a
known issue of v0.2.0). tests/test_release.cpp reads kVersion from
include/zetta/version.hpp and checks that it equals the CMake project version, the
committed schemas/openapi.json info.version, and that this page exists and is listed
in the index.
Fixed
A test crashed on a race its fixture created — #1292
RestClusterFixture.RunReportAnswersForADatasetlessRun ended in a segmentation fault or an
abort now and then. Under AddressSanitizer it was a heap-use-after-free: the churn run's
thread appended to the Run's log while the test thread serialized the Run. The fixture
never ticked its node, and a node that no loop has ticked runs owner work inline on the
calling thread. The fixture now ticks once after attaching its cluster, as both apps do
from their first loop iteration (#1293). Test-only: production nodes were not
affected. Under ASan with 8 parallel copies it crashed once in 80 runs before, and
0 times in 200 after.
A documentation drift check skipped silently
CliDocs.EveryFlagMainOffersIsDocumented found src/main.cpp only through an environment
variable or the working directory, and under ctest the working directory is the build
tree, so it skipped. It now falls back to the source directory the tests were built from.
Documentation
docs/network/: network records, scoped app access, and peer
gateway links.docs/cluster/transport-library.md, and the cluster
security page on the shared transport.docs/releases/README.md: how a release is cut, ending with the image
rolled and tagged (#1278).
Not included
- zetta-tunnel itself: its executable, gateway authorization, gateway certificates and
readiness live in zetta-tunnel. This
release gives it the library, the record API and the peer contract. - Projecting the app credential to the app and reconciling connections: the operator
and Lake (orama-bridge#186). - Moving Lake's users and tokens into workspace Zettas
(#1172). - Clustered pub/sub (#1038).
- No GPU build, real-checkpoint or external-engine test was run, and the Linux test suite
was not run (see Verification).
Where it has run
Before this tag, the post-v0.2.0 tree has not run on a cluster. On 2026-10-17 every
Zetta on the development cluster (orama-production-cluster-01: the Lake bus
lake/lake-prod-zetta-zetta-0 and four workspace Zetta pods) runs
oramasearch/zetta:main-4015ef76, which is v0.2.0. The new capabilities were exercised
by the test suite only, in-process, with real Corosio TCP and mutual TLS between nodes.
The release image is oramasearch/zetta:v0.3.0 (the same digest as main-d7551c75,
sha256:0dad6a7fba1842a1406a3430394fd98a06d3241d9989a69470f6ad921a54328a). Rolling it to the cluster is a separate step, through the
operator's default Zetta image, and needs Steve's go-ahead.
Verification
Full test suite, run serially (ctest --test-dir build, no -j), macOS 14 with 16
hardware threads, Release builds, while other builds kept the host's load average between
5 and 30. Run 3 is the release commit; runs 1, 2 and the shared run are the same tree
without the fixture fix (#1293) and, for run 1, without the CliDocs fix.
| Run | Build | Tests | Passed | Failed | Skipped | Disabled | Time |
|---|---|---|---|---|---|---|---|
| 1 | static | 2,888 | 2,870 | 1 | 17 | 1 | 880 s |
| 2 | static | 2,888 | 2,870 | 2 | 16 | 1 | 775 s |
| shared | -DBUILD_SHARED_LIBS=ON |
2,888 | 2,871 | 1 | 16 | 1 | 861 s |
| 3 (release) | static | 2,888 | 2,872 | 0 | 16 | 1 | 762 s |
The failures in the earlier runs:
RestClusterFixture.RunReportAnswersForADatasetlessRun, a segmentation fault (run 2) and
an abort (shared): the fixture race fixed in this release (#1292, #1293). After the fix
it passed 200 of 200 (8 parallel copies, 25 repeats each) and run 3.ClusterSecurity.MeshConvergesOverMutualTlsUnderSustainedLoad(run 1): documents did not
converge within the deadline under load
(#959,
#1256). Passed 3 of 3 alone, and in
the other runs.ObjectRoutes.RangedReadsStayInsideTheirBudgets(run 2): a 64 KiB ranged GET p99 of
7.19 ms against a 2 ms budget under load
(#1256). Passed 3 of 3 alone, and in
the other runs.
Skipped: 16 tests need a real checkpoint, a GPU, a generated corpus or an external
OramaCore engine that this host does not configure, and one benchmark is disabled. Run 1
also skipped CliDocs.EveryFlagMainOffersIsDocumented, fixed in this release (see Fixed).
The new version test: ReleaseVersion.MatchesTheCMakeProjectVersion,
ReleaseVersion.MatchesTheCommittedOpenApiDocument and ReleaseVersion.IsDocumented
pass.
A running node reports 0.3.0: a headless node built from this branch logs
zetta 0.3.0: node started, answers "version": "0.3.0" on GET /v1/server_info, and
serverInfo.version 0.3.0 to an MCP initialize. zetta --help prints
zetta 0.3.0 — a database node; there is no --version flag.
Linux: the release image is built from the tag on the cluster BuildKit (linux/amd64,
Ubuntu 24.04, Clang 19, zetta target only, no tests). It built from a detached checkout
of the tag, with the one compiler warning above, and is oramasearch/zetta:main-d7551c75,
also tagged oramasearch/zetta:v0.3.0, both naming sha256:0dad6a7fba1842a1406a3430394fd98a06d3241d9989a69470f6ad921a54328a. The
Docker Hub repository still answers 404 to an anonymous request. The Linux test suite was
not run: the native amd64 build box did not answer on 2026-10-17.
The build has the same two warnings as v0.2.0: a discarded io_result at
src/core/ml/forecast_execution.cpp:99, and a duplicate-library note from the linker.
Known issues
Security first. These are open in the tracker; none is new in this release.
- #819, #793
— some malformed SQL (a malformed table reference,REVOKE TOKEN, a scoped
GRANT ON table(cols),SELECT current_user()) was reported to throw an uncaught
ParseErrorthat ends the node: a denial of service by anyone who can send SQL. On the
macOS build of this release,SELECT * FROM system.*,REVOKE TOKEN abc,
GRANT sql:read ON t(a,b) TO xandSELECT current_user()over/v1/sqleach answer
engine_errorand the node stays up (checked 2026-10-17). #793 names a toolchain
miscompile as the cause, and the Linux image (Clang 19) was not checked, so both stay
open. - #760 — column masking can be
bypassed with an aggregate (SELECT id, MAX(salary) ... GROUP BY id). - #786, #835
— mutual-TLS gossip can stall because the read and write pumps share one OpenSSL
stream, and a node can silently stop applying inbound sync. The transport moved into a
library in this release without changing that behaviour. - #762 — authorization is off by
default: a network-facing node without--require-authaccepts unauthenticated
requests. - #763, #764
— a joining node can create a second owner account, and replicated identity records are
trusted with no proof of origin. Any node holding the join token is trusted. - #761, #765, #766, #767
— role management is unbounded, a row policy with no condition allows all rows, any
authenticated user can read the policy, user and role catalogue, and blob permissions
match by text prefix rather than path. - #759, #830
— still open in the tracker, althoughv0.2.0checks every data-changing statement's
right (#1229) and seals saved
connector credentials (#1230).
Re-check and close them. - #769, #770, #771, #669
— lower-priority hardening: the admin token is not hashed, the join token in an
environment variable is exposed widely, a peer's advertised addresses are not capped,
and there is no certificate enrollment, pinning or revocation. - #1276 —
runcan fail with
engine_error(key 'system.cpu.percent' not found), or end the process on
system.throughput.per_second, when a system metric was not sampled. Seen within tens of
milliseconds of node start. Retry therun. - Tests that fail under a loaded host and pass alone, two of which failed in this
release's earlier runs (see Verification): #1288 (near-data featurize under
ctest -j 6), #1256 and
#959 (mesh and store latency
benchmarks), #1248,
#1124,
#964,
#963. - #999 — no reachable consent
approval surface, so consent enforcement stays off by default. - Still open from
v0.2.0: #1001
(Snowflake is PAT/OAuth only), #977
(a fresh node has no agent loaded), #1272,
#1255,
#1203,
#1174, pub/sub is one node
(#1038,
#1037,
#1036), and the source connectors are
Postgres and Snowflake only (#1014).
Upgrading
- No migration. A
v0.2.0data directory starts as is; the node adds the four
network record collections on first start. - Grant
network:writeto zetta-tunnel andnetwork:readto Lake's service role
(each withmcp:call) before they write or read network records. - Grant
tokens:manageto whoever issues app credentials. - Upgrade every node of a cluster together, as for
v0.2.0: mixed versions were not
tested.