v0.9.0 — SSE streaming for the completions adapter

v0.9.0 — SSE streaming for the completions adapter

The chat-completions adapter (and every provider built on top of it — OpenAI, OpenRouter, Groq, Mistral, Ollama, vLLM) now streams responses by default. Text and reasoning arrive as Delta::AppendText, tool-call arguments accumulate until they parse as JSON, image outputs are committed as media parts, and the terminal usage frame drives the loop-level chat span attributes opened in 0.8.0.

What's Changed

Streaming completions

CompletionsTurn becomes an enum of Buffered { events } and Streaming(state). The streaming arm holds a BodyStream, an SSE decoder, an event translator that normalises provider-specific chunks into ModelTurnEvents, and a postprocess hook that runs once on EOF (so finish reasons, accumulated tool calls, and the final usage frame land in the right order). CompletionsProvider gains two hooks:

pub trait CompletionsProvider: Send + Sync + Clone {
    // ...
    fn streaming(&self) -> bool { true }

    fn apply_stream_options(
        &self,
        _body: &mut serde_json::Map<String, Value>,
    ) -> Result<(), LoopError> {
        Ok(())
    }
}

Providers that support terminal usage frames insert stream_options: { include_usage: true } in apply_stream_options; OpenAI, OpenRouter, Groq, Mistral, Ollama, and vLLM are all wired through to advertise streaming in their READMEs and turn it on by default. To opt out, override streaming() to return false and the adapter falls back to the original buffered path.

// Drop-in: an existing OpenRouter agent now renders tokens as they arrive.
let mut events = agent.next_turn(input).await?;
while let Some(event) = events.next_event().await? {
    match event {
        ModelTurnEvent::Delta(Delta::AppendText { text, .. }) => print!("{text}"),
        ModelTurnEvent::ToolCall(call) => dispatch(call).await?,
        ModelTurnEvent::Usage(usage) => tracing::info!(?usage, "turn done"),
        _ => {}
    }
}

Image output

Generated images returned alongside text complete the multimodal output path. The delta.images[] shape — converged on by OpenRouter (Google Nano Banana family), Vercel AI Gateway (Nano Banana, Nano Banana Pro, GPT-5 image variants), and llmgateway.io after the OpenAI spec left it underspecified — is decoded into MediaPart { modality: Image, .. } on both buffered and streaming paths. The image_url.url is preserved as-is (typically a data:image/...;base64,... URL).

Loop-level chat span gets its usage frame

The chat span opened in 0.8.0 records token usage from the response. With streaming, that usage often only arrives on the terminal frame after the body is drained — the span now stays open until the stream's EOF postprocess runs, so gen_ai.usage.input_tokens / output_tokens populate for every provider, buffered or streaming.

Docs

• agentkit-adapter-completions/README.md documents the buffered ↔ streaming surface and how chunks translate to ModelTurnEvents.
• Per-provider READMEs picked up "with streaming" tags.
• book/src/ch04-streaming-and-deltas.md extends the streaming chapter to cover completions-style providers.
• openrouter-coding-agent example switches to streaming rendering.

Migration notes

• Default behaviour changes: any provider built on CompletionsProvider now streams. To restore buffered behaviour, override streaming() on the provider.
• CompletionsTurn is now an enum behind a private inner; if you matched on it directly (unlikely outside this crate) update to use the public API.
• ResponseMessage gains an images: Vec<ResponseImage> field — non-breaking for callers, but any custom deserializer that asserted no extra fields needs updating.

Commits

• feat: add .images support to completions (9df6caa)
• feat: add completions streaming support (fae16c4)

Full Changelog: v0.8.1...v0.9.0