What's the intended way to fallback from anthropic to openai

[Nick Thapen (Hellebore)]

I've been running into problems trying to add a fallback from anthropic to openai, in case of rate limiting or other issues with anthropic.

When I use ModelFallbackMiddleware with Anthropic primary + OpenAI fallback, the fallback path inherits Anthropic-specific model_settings from AnthropicPromptCachingMiddleware (specifically cache_control), and the OpenAI chat completions client then fails with TypeError: AsyncCompletions.create() got an unexpected keyword argument 'cache_control'.

Also it seems difficult to add fallbacks for e.g. subagents and summarisation.

Whats the intended way of doing this falling back in general?

Thanks!

[Mason Daugherty (mdrxy)]

Hey Nick Thapen (@Hellebore), what version are you running? This should work out of the box when you pass ModelFallbackMiddleware through create_deep_agent's middleware= parameter. If you bump to the latest version and observe that this isn't the case, please open an issue.

[Nick Thapen (Hellebore)]

Hi - this doesn't work since falling back from anthropic to openai it is including the anthropic specific cache control headers which cause a crash.

[Mason Daugherty (mdrxy)]

Nick Thapen (@Hellebore) which version are you running?

[Nick Thapen (Hellebore)]

Sorry was on holiday - 0.4.7

[Mason Daugherty (mdrxy)]

Nick Thapen (@Hellebore) would you mind opening an issue, including a MRE?