FallbackModel does not handle UnexpectedModelBehavior raised during response handling in agent graph

[m7mdhka]

Initial Checks

• I confirm that I'm using the latest version of Pydantic AI
• I confirm that I searched for my issue in https://github.com/pydantic/pydantic-ai/issues before opening this issue

Description

When using FallbackModel, the fallback is only triggered if the error occurs during the model request itself.
However, if the model returns an invalid/empty response and pydantic_ai raises UnexpectedModelBehavior during response handling inside the agent’s graph, the fallback is not used.

This makes FallbackModel less useful, since many real-world failures happen at the parsing/response validation stage, not just at request time.

Example Code

from pydantic_ai.exceptions import UnexpectedModelBehavior
from pydantic_ai.models import Model
from pydantic_ai.models.bedrock import BedrockConverseModel, BedrockModelProfile
from pydantic_ai.models.fallback import FallbackModel
from pydantic_ai.providers.bedrock import BedrockProvider
from pydantic_ai.settings import ModelSettings


def get_fallback_model() -> Model:
    return BedrockConverseModel(
        model_name="meta.llama3-3-70b-instruct-v1:0",
        provider=BedrockProvider(region_name="us-east-2"),
        profile=BedrockModelProfile(bedrock_supports_tool_choice=False),
    )


def get_model(model_name: str, **settings) -> Model:
    return FallbackModel(
        BedrockConverseModel(
            model_name=model_name,
            provider=BedrockProvider(region_name="us-west-2"),
            profile=BedrockModelProfile(bedrock_supports_tool_choice=False),
            settings=ModelSettings(**settings),
        ),
        get_fallback_model(),
        fallback_on=(UnexpectedModelBehavior,),
    )


Running this sometimes raises:


pydantic_ai.exceptions.UnexpectedModelBehavior: Received empty model response


But instead of falling back to the second model, the exception is propagated.

Python, Pydantic AI & LLM client version

Python: 3.13.3
PydanticAI: Latest

[DouweM]

@m7mdhka Thanks for filing the issue. By definition FallbackModel cannot automatically retry things that happen upstream from it, i.e. not in a model request, but I agree it would be useful to let the Received empty model response error be handled by trying a different model.

Before we figure out how that would be realized though, it would be useful to understand why you're seeing this error at all and if there's not a better way to handle it. Did you happen to capture the ModelResponse object or HTTP response? Did it indeed not contain any tool calls, text, or thinking?

[lcy0321]

I'm not certain whether this is related to the root cause identified in this case, and it appears that configuring "retries" also does not trigger retries for pydantic_ai.exceptions.UnexpectedModelBehavior: Received empty model response.

[DouweM]

@lcy0321 Which retries are you referring to?

If the retries argument on Agent: that only affects failed tool calls

If https://ai.pydantic.dev/retries/: that works at the request level, so like FallbackModel it's not capable of recovering from this error that happens at a higher level.

I'll see if I can get rid of this exception, though.

[DouweM]

OK, #2961 addresses the specific pydantic_ai.exceptions.UnexpectedModelBehavior: Received empty model response error by giving the model a chance to try again. If it keeps producing empty responses, we will eventually hit a pydantic_ai.exceptions.UnexpectedModelBehavior: Exceeded maximum retries ({max_result_retries}) for output validation exception that is still not handled by FallbackModel. So I'll keep the issue open, even though it's slightly less urgent now.

[lcy0321]

@lcy0321 Which retries are you referring to?

If the retries argument on Agent: that only affects failed tool calls

If https://ai.pydantic.dev/retries/: that works at the request level, so like FallbackModel it's not capable of recovering from this error that happens at a higher level.

I'll see if I can get rid of this exception, though.

Thank you for your clarification. The description of the retries for Agent is:

The default number of retries allowed before raising an error.

So I thought it would retry for most errors. Perhaps the description could be made clearer?

[DouweM]

@lcy0321 You're right, I updated the docstring in #2963.