Skip to content

v1.3.0 — Streaming Contracts

Choose a tag to compare

@johnnichev johnnichev released this 14 Jul 23:51
· 3 commits to main since this release
bd1ceb7

Makes final_turn_only structured output safe to rely on across upgrades (#174): behaviors that streaming chat integrators previously had to learn by reading the source are now documented, tested public contracts — enforced on every consumption surface: astream() chunks, the stream_handler callback in run()/arun(), and the selectools.serve SSE stream. The contract tests (tests/agent/test_structured_streaming_contract.py) pin every clause, so a refactor that breaks integrators breaks CI.

The streaming contract

  • The synthesis turn never streams — not as chunks, not through stream_handler. Its JSON arrives only on the terminal AgentResult (.content/.parsed). StreamChunk(event="structured_synthesis_start") fires once per run when the synthesis turn begins (validation retries don't re-emit it).
  • single_pass no longer streams the structured answer (Changed, @beta surface): content is suppressed on both channels — it's the JSON envelope by construction when the schema rides on every loop call. Tool-call chunks still stream for activity display, including on providers without native async streaming (which previously emitted no tool-call chunks at all). Migration: read AgentResult.parsed instead of accumulating streamed content.
  • New StreamChunk(event="structured_reuse"): when the converged loop answer validates and is reused, the just-streamed loop chunks WERE the structured answer — this signal lets a chat surface convert or retract the bubble.
  • serve SSE forwards event chunks as {"type": "event", "event": ...} records (previously dropped), so the convert/retract flow works through selectools.serve.

The should_finalize contract

should_finalize(messages, text) receives a documented conversation view: prior history, possible SYSTEM context injections, this turn's USER message(s), the ASSISTANT tool-call messages, and one Message(role=TOOL) per executed tool. On executor-appended TOOL messages, tool_name is always populated and tool_result carries the result text for successful executions while staying None for errors and policy/coherence denials — tool_result is not None is the supported test for "this tool produced a real result". tool_call_id is populated for native tool-calling providers only.

Compatibility

  • All additive except the single_pass streaming change (a @beta surface, 0 releases old, changed at its one consumer's request — see the migration note).
  • Message.from_dict round-trip fidelity is untouched; the tool_result=None failure marker keeps its v1.2 meaning, now documented as a contract.

Every change in this release went through two adversarial multi-agent review rounds (the second at max effort, 39 agents) — the reviews caught the contract being implemented on only one of four surfaces before it shipped. 7,787 tests green.