v0.46.0

0.x minor bump because it is the breaking-change vehicle: this release changes the wire.
89 commits since v0.45.3. Read the breaking section before upgrading — all four items below
require a client change, and one of them is a rollout-sequencing hazard with open streams.

Breaking

__describe__ is retired; discovery is vgi_rpc.Reflection.v1

The two __describe__ fast paths (serve_one's and the HTTP unary dispatcher's) are gone,
along with _describe_batch, _describe_metadata, the synthetic __describe__ method
registration, _DESCRIBE_SCHEMA, build_describe_batch and parse_describe_batch.

What to do: call vgi_rpc.Reflection.v1 — list_protocols, then describe. Two round
trips, because a server may host several protocols and there is no longer a single "the"
protocol to describe without asking first; pass an explicit protocol to skip the discovery
hop. introspect() and http_introspect() already do this for you, and
introspect.ServiceDescription survives unchanged, so the CLI, the describe page and the
conformance runner are unaffected.

A stale client calling __describe__ now gets a refusal that names vgi_rpc.Reflection.v1
and both of its entry points, so it is fixable from the error text alone. Every other
reserved name keeps the plain "not implemented" capability answer.

Two consequences worth knowing: discovery is now an ordinary dispatch through a real binding,
so it gets access logging, telemetry and the normal error envelope — a test that asserted
describe produced no access record now asserts it produces one. And a server that
externalizes now externalizes reflection replies too; the old fast path was exempt only by
accident of answering before dispatch. enable_describe is binding registration: without it
the server does not host vgi_rpc.Reflection.v1 and a client gets the ordinary "no such
protocol" answer.

is_exchange became stream_kind, a string. Whether a stream is an exchange is often not
knowable from the Protocol, so the state is genuinely three-valued and "unknown" is said
rather than encoded as a null.

Requests must carry the vgi_rpc.protocol routing key

Dispatch resolves on (protocol, method). vgi_rpc.protocol names the hosted protocol a call
addresses, stamped beside vgi_rpc.method, with the major version in the name
(vgi_rpc.Identity.v1) — so an incompatible major is a different name a proxy can route on
without parsing Arrow, and v1/v2 can be served side by side. protocol_version stays separate
and gated: it is what catches a signature change under an unchanged method name, which Arrow
cannot detect for itself.

Required on raw transports, optional over HTTP, where the path has already resolved the
binding. When both are present they must agree. The cost of the HTTP relaxation is recorded
rather than hidden: a path rewrite by an intermediary is undetectable when the key is absent,
since the intermediary cannot touch the metadata.

What to do: RpcClient stamps its own bound protocol, so callers using it need no change.
Anything that hand-builds requests — or any intermediary that rebuilds them — must stamp the
key. An intermediary must re-stamp vgi_rpc.protocol and protocol_version together or
neither: preserving the version while dropping the name routes to the wrong protocol carrying
a version that then gates against the wrong surface.

Three rejections, deliberately distinguishable rather than collapsed, both new types
subclassing ValueError:

protocol_not_specified	no routing key
protocol_not_supported	not hosted here (404-shaped)
method_not_implemented	hosted, but no such method

HTTP routes are now namespaced by protocol, protocol_version is gated per binding rather
than per server, and stream state and tokens are scoped to the owning binding.

protocol_hash is redefined — every pinned digest moves

It used to hash schema.serialize() of a describe batch. The batch is one encoding of the
description and not the same encoding in every port, which made the hash comparable only
against itself and useless for the one question it exists to answer.

It is now taken over the decoded description:

sha256(b"vgi_rpc.protocol_hash.v1|" + canonical_json(description))

RFC 8785 canonical JSON, with every numeric parameter folded into a type token so the preimage
contains no JSON numbers and JCS's number canonicalisation never applies. Out of the preimage:
server identity, docstrings, parameter defaults, Python type names, and the framework's own
REQUEST_VERSION / DESCRIBE_VERSION — folding the framework versions in would rotate every
protocol's hash on a release that changed no protocol. is_exchange was also dropped from the
preimage.

What to do: re-pin. Any stored or asserted digest changes. tests/golden/protocol_hash_vector.json
is the cross-port vector — the digest plus the exact preimage bytes — so a disagreeing
implementation can diff its own preimage and see which method, field or type token it spells
differently.

Related: access-log protocol_hash is now a required argument rather than defaulting to "",
and a record is logged with the owning binding's hash, not the primary's.

Upgrade hazard: in-flight HTTP tokens minted by 0.45.3 will not open

Not stated plainly in any commit subject, and it is a rollout-sequencing fact. Binding the
protocol into the AAD of the cursor and call tokens — rather than comparing it in application
code — bumped both AEAD associated-data prefixes:

token	0.45.3	0.46.0
state / cursor (no peer evidence)	vgi_rpc.state.v4	vgi_rpc.state.v6
state / cursor (with peer-evidence binding)	vgi_rpc.state.v5	vgi_rpc.state.v7
call (no peer evidence)	vgi_rpc.call.v1	vgi_rpc.call.v3
call (with peer-evidence binding)	vgi_rpc.call.v2	vgi_rpc.call.v4

The AAD also gained a trailing protocol scope (b"\x00" + protocol). _CALL_TOKEN_VERSION did
not move — the break is carried entirely by the prefixes. These are the HTTP stateless-resume
tokens in vgi_rpc/http/server/_state_token.py, so the stateless-resume path is exactly what is
affected: a continuation minted before the upgrade fails the AEAD tag check and is rejected as
an invalid token, with nothing in the error naming the upgrade as the cause. Sticky session
tokens go through the same _compute_aad (with SERVER_SCOPE) and break the same way.

What to do: drain open streams before upgrading, or accept that in-flight continuations and
sticky sessions restart. Do not expect a diagnosable error — plan the sequencing instead.

This is the design intent, not a side effect: a cross-protocol continuation now fails the tag
check, with no comparison code to get wrong and nothing to forget on the cache-hit path, where
the call token is never opened at all. _compute_aad and _compute_call_aad take the scope as
a required keyword, so no call site can silently inherit an unbound token.

Added

vgi_rpc.Identity.v1

Identity moves from POST {prefix}/__introspect_token__ — an HTTP JSON route that existed on
one transport only and had to be hand-written in every port — to a protocol. http/server/_introspect.py
and its wiring are gone, along with the make_wsgi_app / serve_http parameters and the
HTTP-shaped conformance group. Identity lives at the RPC layer because a bearer token is not a
VGI concept and the primitives it builds on (AuthContext, chain_authenticate,
AuthUnavailableError) were already here.

Two methods, guarded very differently, and the asymmetry is the design:

• introspect_token answers a question about somebody else's credential, so it is an
  oracle. Four normative guards: an allowlist with no permissive default (a server configured
  without one refuses to start), authorization checked before the credential is touched,
  uniform rejections, and a JWS-shaped subject refused before the resolver runs. Rate limiting
  bounds rather than closes what an allowlisted-but-compromised caller still has.
• issue_grant mints for the caller, so it needs neither allowlist nor limit. It takes no
  subject parameter at all — cross-subject minting is closed by construction rather than by a
  check a port can forget. A credential with no verifiable auth_time cannot mint, which stops
  a grant minting another grant and escaping the IdP. Its rejections are deliberately
  actionable, unlike introspection's: they are always about the caller, so naming the reason
  leaks nothing and is the only way a console learns to re-prompt.

A method whose hook the deployment did not configure is not hosted, and the binding's
method set — and therefore its protocol_hash — narrows with it. Absent beats
routed-and-refusing: it is what keeps a dependency upgrade from growing a credential-to-identity
oracle on every existing worker.

error_kind carries the definitive-versus-transient distinction that HTTP callers used to read
off the status (404 vs 503). identity_unavailable is deliberately not a ValueError, because
chain_authenticate advances on ValueError and a sidecar outage raised as one restarts every
session in the fleet over a thirty-second blip.

The credential cap is measured in UTF-8 bytes, not codepoints, over the trimmed
credential, against an enumerated whitespace set — all three pinned by tests, with
vgi_rpc.Identity.v1's digests checked in as the ports' shared vector.

Externalization on every transport

--fake-storage and --externalize-threshold moved off the HTTP conformance scripts and onto
the CLI, where every transport reaches them. Externalization was never an HTTP feature —
§12 says so — but there was no reference byte-stream server that externalized anything, so the
byte-stream client's externalization path (external_config threaded into
_read_batch_with_log_check at two call sites) existed and nothing ran it.

TestExternalByteStream (11 cases) closes that, gated on a
conformance_bytestream_external_target fixture. It pins the properties that actually differ
under externalization: the resolved metadata's origin, log survival through a multi-batch
payload, a dictionary-typed schema echoed onto a zero-row pointer, the separately-externalized
stream header, multi-turn exchange, and a zero-row error batch travelling among pointers. Every
test asserts the storage object count rose, so a variant that quietly stops externalizing goes
red rather than passing trivially. The group escalates rather than skips for a port that has a
fake-storage fixture — withholding it is how this half stayed untested.

Name the byte-stream transport explicitly rather than inferring it from the absence of flags:
a _worker() translation that spells "byte stream" as "no flags" reroutes to an HTTP server the
moment the fixture adds --fake-storage, and the result is a three-way stall that reads in CI
as a flaky timeout.

produce_annotated_batches

A conformance method at the one point where per-batch custom metadata and externalization meet,
where two ports shipped opposite defects that nothing could see because no conformance method
emitted per-emit metadata at all. It emits count batches carrying a varying batch_index, a
constant batch_total, and a deliberately non-ASCII emit_label. The index varies and the
tests check which batch carried which value, so a port that caches the first turn's metadata
and reuses it fails; the label is non-ASCII so a port round-tripping metadata through a latin-1
or C-string path fails here rather than in production data. Four assertions, each localizing a
different failure. Ships with an inline control in the ordinary suite.

Spec clarifications (§12, §1.5, §4.10)

Each of these was a sentence that several independent ports read several different ways — which
makes it a defect in the document, not in the implementations.

Provenance is stamped by the reader, at resolve time. vgi_rpc.location.fetch_ms and
vgi_rpc.location.source move out of the "External storage pointer batch metadata" table into
their own "Resolved batch metadata (added by the reader)" table, and provenance is promoted
into the normative list under the resolution pseudocode. A wire pointer MUST NOT carry them,
and a resolved batch MUST NOT carry vgi_rpc.location or vgi_rpc.location.sha256. The word
"diagnostics" is gone — it read as "optional", and one port declared both key constants and
referenced them nowhere on the strength of it.

Resolved metadata is the fetched data batch's, never the pointer's. The pseudocode assigned
into resolved_metadata without ever initialising it, leaving two plausible sources in scope;
three ports chose the pointer, which is the continuation cursor read off the outer pointer
instead of the inner payload — first turn succeeds, second cannot be addressed because
vgi_rpc.stream_state#b64 was left inside the uploaded object. Now stated:
resolved_metadata = data_batch.custom_metadata. Also: every log batch in the payload MUST be
dispatched (the uploaded object is a whole IPC stream), and the loop names its own failure modes
(no data batch, multiple data batches).

location.source is credential-bearing. The redaction rule under Fetch safety governs
diagnostics — errors, logs, traces, exception chains — and not this key, which carries the URL
in full. The URL is already on the wire in vgi_rpc.location, so carrying it whole exposes it
to the consuming application rather than to a new network observer; the asymmetry worth a
paragraph is that it outlives the pointer, which a resolved batch must not carry. Callers
must treat location.source as credential-bearing and keep it out of logs and telemetry. A
redacted value would also be wrong on its own terms — it names a URL that was never fetched.

Externalizing a header is optional; resolving one is mandatory. §1.5 said the header "is
externalizable like any other batch" and then that it "is uploaded" when over threshold, which
read as an obligation on servers. It is not. A server that externalizes only in its data path is
conformant and most are. A client that cannot read an externalized header is not conformant,
whether or not its own server ever produces one.

Sticky session scopes nest. §4.10 never said what session_begin does when a scope is
already open, and the natural reading is one slot; the suite nests, and four ports each built a
slot and each independently arrived at a stack. Now stated: begin pushes, end pops and restores
the enclosing scope, the accessors act on the top, an empty-stack end is a no-op, closing drains
innermost-out, and the best-effort DELETE covers the popped scope only.

Also in this release

• RpcMethodInfo is self-describing (protocol_name from a declared ClassVar, read via
  vars() so a Protocol subclass does not silently inherit its base's identity), and
  undeclared parameters are rejected. Telemetry and access logs follow the resolved method
  rather than a name baked in at hook-registration time.
• Dispatch calls implementation_for(info) everywhere rather than the primary implementation —
  every unary and stream dispatch site on both the pipe and HTTP transports was wrong for a
  secondary protocol.
• describe_diff became a six-port hash-equality drift guard: it compares every hosted protocol
  rather than just the application one, and every description field, by reflection — protocol
  names, header schemas, and each stream's declared state class.
• Protocol names are validated on both carriers, and the vgi_rpc. prefix is reserved.
• A stream call emits one access record per turn, and a secondary protocol's record carries its
  own identity.
• Reflection describes its own methods.
• Cross-port tooling folded in under tools/cross-port/, with its history: a port that was
  never audited now reports as a failure rather than as a clean run.
• Fixes: read the request body the fake storage never read; one thread per connection in the
  fake storage server; read the describe golden as UTF-8 rather than as whatever the locale
  says; restore the pydoclint, ty and docs-build gates.

Full changelog: v0.45.3...v0.46.0