feat(agent): add update_models() to swap STT/VAD/LLM/TTS at runtime#6440
Merged
Conversation
Agent.update_models(*, stt, vad, llm, tts) swaps a pipeline model in place, for cases like changing the STT language or TTS voice mid-call without a full agent handoff. When the agent isn't running it replaces the stored model; when running it applies to the live pipeline: STT/VAD streams are rewired, LLM/TTS take effect on the next generation/synthesis, and metrics/error listeners follow the new model. None disables a model, matching Agent(stt=None). Swapping to or from a RealtimeModel while running raises (its session is created at start); use AgentSession.update_agent instead. Adds KeytermDetector.swap_stt to rebind the recognizer in place so the swap stays synchronous.
theomonnom
reviewed
Jul 17, 2026
| chat_ctx, exclude_invalid_function_calls=exclude_invalid_function_calls | ||
| ) | ||
|
|
||
| def update_models( |
Member
There was a problem hiding this comment.
What do you think about naming it update_options to keep it consistent with how we do it in other parts of the codebase?
Contributor
Author
There was a problem hiding this comment.
updated to update_options for Agent, agent activity keeps _update_models
theomonnom
approved these changes
Jul 17, 2026
Member
|
Just double checking, when switching to another TTS mid-conversation? Will an "active" generation still use the old TTS, or the new one? |
…ive swap AudioRecognition._update_stt now optionally refreshes _stt_model/_stt_provider (used for user_turn trace-span attributes) and resets stt_context, so a mid-call STT swap doesn't keep the previous provider's trace labels or stale speaker metadata. Existing callers (_start, _stop, _clear_user_turn) reuse the same STT and are unaffected.
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Contributor
Author
update the TTS won't cancel the active generation, the active one still uses the old TTS. |
Aligns with update_options elsewhere in the codebase (AgentSession, AgentActivity, stt/tts) and leaves room to add turn_handling/tool_handling to the same method later. The internal AgentActivity._update_models helper keeps its name (private, models-only, and avoids colliding with the activity's existing update_options).
… atomic _update_models previously mutated STT before the VAD branch, where _update_vad's streaming-turn-detector min_silence check can raise ValueError, leaving the pipeline half-swapped despite the documented all-or-nothing contract. _check_vad_silence_requirement now accepts a candidate vad so the check runs up front alongside the RealtimeModel check, before any mutation.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Agent.update_models(*, stt, vad, llm, tts)swaps a model in place. Only the models passed are changed; strings resolve to inference models like the constructor, andNonedisables a model (overriding the session), matchingAgent(stt=None).When the agent isn't running it replaces the stored model, applied on the next start. When running, it applies to the live pipeline: the STT and VAD streams are rewired, the LLM and TTS take effect on the next generation and synthesis respectively, and each model's
metrics_collected/errorlisteners follow the new instance. The call is synchronous and atomic.The Agent is the source of truth — the activity resolves
agent.<model> if given else session.<model>live — soupdate_modelsjust writes to the agent. This is distinct fromAgentSession.update_agent(full handoff, new instance) and from a model's ownupdate_options(same instance, provider options).Alternative to #6235
Realtime models
A
RealtimeModelowns a live session created when the agent starts, so it can't be swapped in place. While running, passing aRealtimeModelasllm(or swapping away from one) raisesRuntimeErrorpointing toAgentSession.update_agent. Validation happens before any mutation, so the call stays all-or-nothing.