v0.23.0

Breaking changes

KRONK_LIB_VERSION must be unset — a leftover pin aborts the inference engine at startup, remove the variable from compose and the environment so the digest baked into the image applies.

Highlights

Attested upstream gateways — A TEE node can verify an attested LLM gateway at boot, pin every request to that enclave, and publish the evidence for a payer to re-check; configure tee.upstream_attestation (llm.provider stays openai_passthrough, and require_upstream_providers cannot be empty).

Web tools under tee.mode — zs_web_search, zs_web_read, and zs_image_search no longer block boot; the caller opts in per request, while image_llm is still refused.

Verified engine bundles — Inference engine 1.32.7 checks its runtime bundle against a digest baked into the image and will not start on a bundle it cannot verify; older libraries on an existing volume are replaced in place.

Improvements

• /v1/zs/attestation and the TEE operator docs now state the real checks (TDX quote, event-log replay, measured compose, sealing-key binding) and that NVIDIA GPU EATs are not produced or verified. X-Zs-Require-TEE is a boolean, not a mode selector.
• zs-node tee compose includes a commented tee.upstream_attestation example and names both qualifying passthrough upstreams: xAI zero-retention, and an upstream that attests its own enclave.
• A TEE node logs one startup warning naming the builtin tools it serves.

Fixes

• Mnemonic retrieval via ZS_MNEMONIC_URLS was exposed to grpc client vulnerabilities, including HTTP/2 heap exhaustion.

Upgrade notes

1. Delete KRONK_LIB_VERSION from the compose file and the process environment, then restart. Leaving it set means the engine does not boot.
2. Do not recreate the GPU volume. An older on-disk library is replaced in place; recreating the volume re-pulls every weight file.

Details

This release is about matching what a TEE node advertises to what it can actually prove, and about not taking choices that belong to the payer.

The GPU engine is 1.32.7, which bakes a runtime-bundle digest into the image and will not decode a bundle that fails that check. That property only holds if KRONK_LIB_VERSION is unset: a leftover tag either cannot be verified or is checked against digests from the same host that served the files. The document at /v1/zs/attestation is the preimage a payer hashes, and it now names GPU attestation as the remaining gap rather than the bundle. Operator docs that said an NVIDIA EAT is verified, that confidential mode means GPU CC, that X-Zs-Require-TEE: nvidia-cc routes to such a pool, or that the sealing key is minted once at boot, were not true of this build; the key rotates about every 20 minutes.

Web tools on a TEE node are a caller decision. They exist on a request only when the caller listed them, so refusing them at boot moved that choice onto the operator and left a node without tee.mode as the only way to offer search. image_llm is still refused: it is an operator-pointed image backend with no tool in the request. Expect one startup warning that lists what you serve.

If the LLM upstream is an attested gateway, tee.upstream_attestation appraises it before the first prompt and pins every call. Boot is fatal on a failed appraisal because that appraisal is the claim you advertise. An unreachable gateway keeps the last good verdict until the keyset expires, so someone else's outage does not take attested nodes off the network at once; a verification refusal revokes, and upstream_verification_failed is not retried into a capacity error. Streams can only check the receipt after frames are on the wire — the next request is what fails. require_upstream_providers is refused empty because appraising a router does not say who ran the weights. allow_root_backdoor_env has no NODE_TEE_* override and is honoured only on a gateway image enumerated for SSH; on any other image, including GPU, setting it refuses boot. Intel collateral for this path comes from Intel, not from tee.attestation.pccs_url.