Effort / thinking control for Anthropic models (Haiku 5.5) in the Voice Agent think provider #1704
Replies: 4 comments
|
Thanks for asking your question. Please be sure to reply with as much detail as possible so the community can assist you efficiently. |
|
Hey there! It looks like you haven't connected your GitHub account to your Deepgram account. You can do this at https://community.deepgram.com - being verified through this process will allow our team to help you in a much more streamlined fashion. |
|
It looks like we're missing some important information to help debug your issue. Would you mind providing us with the following details in a reply?
|
|
Hello @dmpalmer81, Thank you for the detailed report and the latency measurements. Apologies for the delay in a reply. On effort control: the documented examples for the On the Until this is resolved, you have three options:
|
Uh oh!
There was an error while loading. Please reload this page.
Hey all,
We run a phone receptionist on the Voice Agent API using the managed Anthropic provider, currently on
claude-haiku-4-5. This week we tried moving toclaude-haiku-5-5and hit a wall: there's no way to set its effort level through the think provider, and at its default (medium) the extra thinking adds enough delay that we can't ship it.What we measured (time from the end of the caller's turn to the agent's first audio, same prompt and settings otherwise):
Simple turns, like confirming a name, are about the same on both. The gap shows up exactly when the caller is waiting on a real answer, and on a phone call an extra half second to a second of dead air is really noticeable.
On Anthropic's API, Haiku 5.5 takes an effort setting (
output_config.effort, low through max) and can also run with thinking turned off. For OpenAI models you already expose this asreasoning_mode. But withtype: "anthropic", Settings gets rejected for anything beyond model and temperature. We tried:reasoning_mode: none / low / medium / higheffort: low / highreasoning_effortthinking: {"type": "disabled"}output_config: {"effort": "low"}Every one comes back with:
Minimal example:
{ "type": "Settings", "agent": { "think": { "provider": { "type": "anthropic", "model": "claude-haiku-5-5", "reasoning_mode": "low" }, "prompt": "..." } } }The ask: could
reasoning_mode(or a new field) be supported for Anthropic models and mapped to Anthropic's effort setting? Even justlow, or a way to turn thinking off, would make Haiku 5.5 workable for voice for us.One related thing worth flagging: if you send
temperaturewithclaude-haiku-5-5, Settings is accepted (SettingsApplied), but the first turn then fails with "Failed to think. Please check your agent.think settings." On a live call that means the agent greets the caller and then goes silent. Haiku 5.5 rejects non-default temperature on Anthropic's side, so it would be great if Deepgram caught this at Settings instead of mid-call.Thanks!
All reactions