# Rig v0.40.0 released!

[gold-silver-copper]

Another huge Rig release is here — nearly 90 PRs merged since 0.39! THANK YOU to everyone who contributed, tested, reported issues, and helped sharpen the APIs <3

This release turns the agent loop introduced in 0.39 into a serious, composable middleware platform. Hooks can now shape requests, govern tool execution, rewrite results, share run-scoped state, and behave identically across streaming and non-streaming runs. We also made tool failures machine-readable, consolidated Rig's OpenAI-compatible providers onto one shared engine, expanded cassette coverage across the provider ecosystem, and removed a meaningful amount of stale public API.

Hook System v2: Composable Agent Middleware

The headline of 0.40 is hook system v2 (#2012). The first hook integration from #1945 connected hooks to AgentRun; v2 takes the next step and turns them into a composable middleware layer for production agents.

Every hook now receives a run-scoped HookContext containing the run id, turn, streaming mode, agent name, and a shared typed Scratchpad. Hooks observe a single StepEvent stream covering completion calls and responses, model-turn completion, tool calls and results, invalid tool calls, and streamed deltas. Multiple hooks compose through HookStack in registration order.

More importantly, hooks can now do much more than observe:

• Flow::PatchRequest changes the outgoing model request for one turn — including temperature, max tokens, tool choice, active tools, history, extra context, and additional provider parameters.
• Flow::RewriteArgs validates or rewrites model-generated tool arguments before execution.
• Flow::RewriteResult redacts, transforms, or replaces tool output before the model sees it.
• Flow::Skip, Terminate, Retry, Repair, and Fail give hooks explicit control over tool policy and invalid-call recovery.

Request patches merge in registration order, tool rewrites chain through every hook, and unsupported actions fail closed instead of silently continuing. The same semantics apply to both AgentRunner::run and AgentRunner::stream because both now use the shared drive_agent engine (#1985, #1986).

This unlocks middleware for RAG injection, guardrails, per-turn model settings, tool allow-lists, telemetry, authorization, retries, and human approval without replacing Rig's agent loop. The new force_tool_first_turn example (#2014) shows an important detail: request patches are deliberately per-turn and non-sticky, so a forced tool choice should be gated to the intended turn.

We also added human-in-the-loop examples for approving, denying, editing, and aborting tool calls (#1967), plus a 30-scenario, live-recorded Gemini hook stress suite (#2013).

Structured Tool Execution

Tool execution is no longer flattened into an opaque string. #2015 introduces a structured result envelope carrying:

• the model-visible output;
• a machine-readable success or failure outcome;
• categorized failures such as timeouts or invalid arguments; and
• hidden typed extensions for application metadata.

A hook can now distinguish a timeout from a permission failure without parsing error text, count repeated failures in its scratchpad, attach internal metadata, and decide whether to retry, rewrite, or terminate.

Per-call runtime context also travels all the way through the agent loop via ToolCallExtensions (#1954). Applications can attach values such as authentication tokens, session identifiers, tenant context, or conversation state for tools to consume without exposing any of it to the model. This works across ordinary tools, MCP tools, and sub-agents.

The Tool authoring API itself is now substantially simpler (#2029). Tools expose name, description, and parameters directly instead of constructing prompt-dependent definitions. Provider-facing ToolDefinitions are generated only at registry and request boundaries, making tool metadata easier to implement and reason about.

One Agent Engine, Exact Semantics

The unification of streaming and non-streaming agents was also furthered, with the majority of remaining divergences being resolved (#1985). Completion hooks, invalid-call recovery, tool execution, history, memory, and finalization all pass through the same control flow. #1986 finished consolidating the seams and fixed Anthropic streaming requests silently dropping output_schema.

Concurrent tools received the same treatment. Streaming agents gained tool_concurrency parity (#1957), and streamed results can be surfaced independently while deterministic call-order history is retained (#1981). Hook v2 subsequently tightened this into atomic batch semantics so failed batches do not leave partial history or orphan lifecycle events.

We also corrected max_turns to mean exactly what it says: an exact total model-call budget, including the initial request, retries, and continuations (#2093). max_turns(0) now rejects before making a request, and blocking and streaming enforce the same boundary.

Finally, blocking and streaming no longer return two nearly identical final-response types. #2056 unifies PromptResponse and FinalResponse, so switching between .prompt() and .stream_prompt() no longer requires renaming every accessor.

Provider Architecture & Coverage

Rig had accumulated many hand-written OpenAI-compatible provider implementations, each carrying its own request conversion, streaming logic, and edge cases. That duplication is gone.

Starting with llamafile (#2038), then continuing across Groq, DeepSeek, Mistral, Together, Moonshot, Perplexity, Hyperbolic, Mira, Azure, Hugging Face, and others (#2040), these providers now use GenericCompletionModel<Ext>. OpenRouter joined the shared implementation in #2054 while retaining its routing preferences, prompt caching, multimodal conversion, generated images, and reasoning metadata.

The result is thousands fewer lines of duplicated provider code and one shared place to improve OpenAI-compatible request conversion, streaming, telemetry, and errors. Provider-specific behavior remains explicit through extension hooks rather than entire forked implementations.

We paired that consolidation with a major expansion in cassette-backed integration coverage:

• OpenAI Responses (#2002)
• Anthropic Messages (#2003)
• Gemini generateContent (#2004)
• ChatGPT/Codex (#2005)
• OpenRouter (#2006)
• xAI/Grok (#2008)
• DeepSeek (#2009)
• Mistral (#2010)
• Groq (#2011)
• Perplexity (#2049)
• AWS Bedrock (#2084)

These suites exercise long histories, sequential and parallel tools, complex argument shapes, tool-choice modes, structured output, reasoning, usage accounting, and streaming/non-streaming parity against recorded real provider responses. Several real provider bugs were found and fixed while building them, including Mistral's previously buffered “streaming” path, provider-specific tool-choice encodings, missing structured-output mappings, and incomplete usage preservation.

Provider Correctness

There are many smaller provider fixes in this release, but several are worth calling out:

• OpenAI Responses now preserve top-level instructions (#1995), reasoning text (#1999), prompt-cache parameters (#1830), null metadata (#1993), and unknown provider-native output items in both normal and streaming responses (#1950, #1951). Strict function tools are now opt-in rather than silently enabled (#1991).
• Gemini now tolerates omitted proto3 defaults instead of failing to deserialize empty or blocked responses (#1936, #1984).
• Vertex AI now preserves signed thought text and image history, and maps Gemini-compatible generationConfig parameters with validation (#2052, #2086).
• Ollama now preserves assistant thinking in non-streaming histories (#1927), supports the max thinking level (#1982), and leaves think absent when callers want the model default (#1990).
• Anthropic now accepts citations: null (#1972) and guarantees that outgoing tool_use.input is an object (#1964).
• ChatGPT now reconstructs empty terminal SSE responses from streamed events (#2001) and preserves non-success HTTP status and body information (#2053).

Provider error inspection, introduced in #1859, is now populated consistently across the workspace (#1944). Applications can inspect provider response status, body, and parsed JSON without depending on provider-specific error shapes.

Structured Output, Vectors & Ergonomics

Agents can now combine tools with structured output through OutputMode (#1929). Instead of applying a provider-native schema to every turn — which can suppress tool calls — an agent can produce its final structured response through a synthetic tool after completing its ordinary work.

Neo4j and LanceDB now implement the common InsertDocuments trait (#1960, #1961). The in-memory vector index also finally honors filters and score thresholds, and Filter::satisfies now correctly handles realistic multi-field documents and numeric comparisons (#1987).

For everyday applications, rig::prelude is now actually comprehensive (#2057). A basic agent or RAG program can import the common agent, completion, streaming, embedding, and tool APIs with use rig::prelude::* instead of assembling imports from several modules.

API Cleanup

Because Rig is still pre-1.0, we are using this opportunity to remove stale and duplicative APIs instead of carrying permanent compatibility aliases.

This release removes the experimental pipeline module (#1941), the unused evaluation module and experimental feature (#2036), the Galadriel provider (#2041), unused Anthropic decoders (#2082), unused generation wrapper traits (#2083), the low-level stream-to-stdout helper (#2085), and unused derive and extractor APIs (#2087).

#2055 also removes duplicate names such as StreamingPromptRequest::multi_turn, consolidates prompt setters, moves ThinkTool under the canonical tool module, and shares authentication errors between ChatGPT and Copilot.

Please check the changelogs when upgrading: hook v2, flattened tool metadata, the unified final response, exact max_turns accounting, provider consolidation, and the removed modules are intentionally breaking changes.

Privacy, CI & Final Notes

GenAI telemetry spans no longer record serialized input or output messages (#2066). The span fields remain available but empty, avoiding accidental content disclosure and high-cardinality telemetry until an explicit opt-in design is introduced. In the future we plan to add the ability to opt into message telemetry for those who need it.

CI now executes doctests instead of merely building documentation (#1939), and default-feature integration-test targets are compiled in addition to the all-features graph (#2007). The examples were also reorganized into independent workspace packages with focused dependency sets (#1937).

Huge thank you to everyone who made 0.40 happen:

• @arsenyinfo
• @CharmingGroot
• @Kade-Powell
• @m-dreiling
• @SarthakB11
• @Shaurya-Sethi
• @wey-gu
• @ForeverAngry
• @arsenyinfo
• @caibirdme
• @danechitoaie
• @domenic-donato
• @kanghouchao
• @kawayiYokami
• @liamcrumm
• @mdelete
• @ratzrattillo
• @redroy44
• @tidely

And thank you to everyone who opened issues, reviewed the large architectural changes, tested providers, and helped improve the release.

We build Rig, Rig builds us!

Full changelog: v0.39.0...v0.40.0