RFC: Canonical streaming ABI across tractatus, Bobravoz, and the Go SDK

[lanycrost]

Summary

tractatus, bobravoz-grpc, and bubu-sdk-go do not currently enforce one canonical streaming packet ABI. The concrete gaps are already visible: structured payloads are object-only in practice, envelope-only MessageID handling is weaker than metadata-shadow handling, kind-only packets are not consistently constrained, and opaque raw binary can still be dropped when the structured-envelope path is chosen.

Why this needs a discussion

This is exactly the type of contract problem that should not be fixed repo-by-repo. The packet ABI spans tractatus, core, bobravoz-grpc, bubu-sdk-go, and any future SDKs. The existing GitHub discussion #75 already covers adjacent typed ABI work, so this RFC should either extend that thread or explicitly narrow the packet-contract slice that it does not settle.

Current evidence

• tractatus models structured Payload and Inputs with google.protobuf.Struct, so the wire-level structured path is object-shaped JSON in practice.
• bubu-sdk-go/stream_context.go also decodes Payload and Inputs through map[string]any, reinforcing the same object-only behavior.
• bubu-sdk-go/stream.go and transport_envelope.go can drop or reinterpret opaque Binary.Payload when the structured-envelope path is forced.
• bubu-sdk-go/stream.go and Bobravoz hub logic still treat private metadata-shadow keys as stronger than envelope-only MessageID / TimestampMs in some dedupe/replay paths.
• bubu-sdk-go/stream_message_builder.go, stream.go, and testkit/validators.go still leave room for inconsistent kind-only packet handling.
• Shipped engrams already reveal the current semantic spread:
  • openai-chat-engram mirrors Inputs and Payload to force the structured-envelope path.
  • openai-tts-engram mirrors JSON into both Payload and Binary.
  • silero-vad-engram and livekit-turn-detector-engram read structured content with Inputs > Payload > Binary precedence.

Proposed decisions

1. Structured Payload and Inputs are JSON objects only. Non-object JSON must travel as opaque binary with an explicit MIME, or wait for a future tractatus extension.
2. Envelope.MessageID and Envelope.TimestampMs are authoritative on the structured path. Metadata-shadow keys are cleanup targets, not part of the long-term contract.
3. Kind-only packets are reserved for control/event markers such as heartbeat, noop, and hook. Business-data packets must carry payload, inputs, binary, audio, or video.
4. Mixing opaque raw Binary.Payload with structured-envelope fields is invalid until tractatus grows a first-class dual-payload ABI. SDK/Bobravoz should fail closed instead of silently mirroring or dropping bytes.
5. tractatus owns transport/envelope ABI versioning. core carries helpers and protocol checks, but should not define an independent packet contract.
6. Because the ecosystem is pre-release, we should remove shadow-field and lossy-mirroring behavior during this ABI cleanup instead of carrying compatibility shims into the first release.

Cross-repo scope

• tractatus: owns proto/envelope versioning and the canonical packet shape.
• core: owns protocol-version helpers and startup compatibility checks.
• bubu-sdk-go: must align encode/decode, validation, dedupe, and builder semantics.
• bobravoz-grpc: must align hub decode/dedupe/replay behavior to the same ABI.
• Impacted shipped engrams include openai-chat-engram, openai-stt-engram, openai-tts-engram, silero-vad-engram, and livekit-turn-detector-engram.

Desired outcome

One canonical streaming ABI plus a pre-release migration/removal plan so Bobravoz, the Go SDK, and future SDKs stop carrying subtly different packet semantics.