Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/agents.md
Original file line number Diff line number Diff line change
Expand Up @@ -444,12 +444,12 @@ user_id=123 message='Hello John, would you be free for coffee sometime next week

## Model errors

If models behave unexpectedly (e.g., the retry limit is exceeded, or their API returns `503`), agent runs will raise [`UnexpectedModelBehaviour`][pydantic_ai.exceptions.UnexpectedModelBehaviour].
If models behave unexpectedly (e.g., the retry limit is exceeded, or their API returns `503`), agent runs will raise [`UnexpectedModelBehavior`][pydantic_ai.exceptions.UnexpectedModelBehavior].

In these cases, [`agent.last_run_messages`][pydantic_ai.Agent.last_run_messages] can be used to access the messages exchanged during the run to help diagnose the issue.

```py
from pydantic_ai import Agent, ModelRetry, UnexpectedModelBehaviour
from pydantic_ai import Agent, ModelRetry, UnexpectedModelBehavior

agent = Agent('openai:gpt-4o')

Expand All @@ -464,7 +464,7 @@ def calc_volume(size: int) -> int: # (1)!

try:
result = agent.run_sync('Please get me the volume of a box with size 6.')
except UnexpectedModelBehaviour as e:
except UnexpectedModelBehavior as e:
print('An error occurred:', e)
#> An error occurred: Retriever exceeded max retries count of 1
print('cause:', repr(e.__cause__))
Expand Down
2 changes: 1 addition & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ async def main():

1. This [agent](agents.md) will act as first-tier support in a bank. Agents are generic in the type of dependencies they accept and the type of result they return. In this case, the support agent has type `#!python Agent[SupportDependencies, SupportResult]`.
2. Here we configure the agent to use [OpenAI's GPT-4o model](api/models/openai.md), you can also set the model when running the agent.
3. The `SupportDependencies` dataclass is used to pass data, connections, and logic into the model that will be needed when running [system prompt](agents.md#system-prompts) and [retriever](agents.md#retrievers) functions. PydanticAI's system of dependency injection provides a type-safe way to customise the behaviour of your agents, and can be especially useful when running unit tests and evals.
3. The `SupportDependencies` dataclass is used to pass data, connections, and logic into the model that will be needed when running [system prompt](agents.md#system-prompts) and [retriever](agents.md#retrievers) functions. PydanticAI's system of dependency injection provides a type-safe way to customise the behavior of your agents, and can be especially useful when running unit tests and evals.
4. Static [system prompts](agents.md#system-prompts) can be registered with the [`system_prompt` keyword argument][pydantic_ai.Agent.__init__] to the agent.
5. Dynamic [system prompts](agents.md#system-prompts) can be registered with the [`@agent.system_prompt`][pydantic_ai.Agent.system_prompt] decorator, and can make use of dependency injection. Dependencies are carried via the [`CallContext`][pydantic_ai.dependencies.CallContext] argument, which is parameterized with the `deps_type` from above. If the type annotation here is wrong, static type checkers will catch it.
6. [Retrievers](agents.md#retrievers) let you register "tools" which the LLM may call while responding to a user. Again, dependencies are carried via [`CallContext`][pydantic_ai.dependencies.CallContext], and any other arguments become the tool schema passed to the LLM. Pydantic is used to validate these arguments, and errors are passed back to the LLM so it can retry.
Expand Down
4 changes: 2 additions & 2 deletions pydantic_ai/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

from .agent import Agent
from .dependencies import CallContext
from .exceptions import ModelRetry, UnexpectedModelBehaviour, UserError
from .exceptions import ModelRetry, UnexpectedModelBehavior, UserError

__all__ = 'Agent', 'CallContext', 'ModelRetry', 'UnexpectedModelBehaviour', 'UserError', '__version__'
__all__ = 'Agent', 'CallContext', 'ModelRetry', 'UnexpectedModelBehavior', 'UserError', '__version__'
__version__ = version('pydantic_ai')
4 changes: 2 additions & 2 deletions pydantic_ai/_retriever.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@

from . import _pydantic, _utils, messages
from .dependencies import AgentDeps, CallContext, RetrieverContextFunc, RetrieverParams, RetrieverPlainFunc
from .exceptions import ModelRetry, UnexpectedModelBehaviour
from .exceptions import ModelRetry, UnexpectedModelBehavior

# Usage `RetrieverEitherFunc[AgentDependencies, P]`
RetrieverEitherFunc = _utils.Either[
Expand Down Expand Up @@ -101,7 +101,7 @@ def _on_error(self, exc: ValidationError | ModelRetry, call_message: messages.To
self._current_retry += 1
if self._current_retry > self.max_retries:
# TODO custom error with details of the retriever
raise UnexpectedModelBehaviour(f'Retriever exceeded max retries count of {self.max_retries}') from exc
raise UnexpectedModelBehavior(f'Retriever exceeded max retries count of {self.max_retries}') from exc
else:
if isinstance(exc, ValidationError):
content = exc.errors(include_url=False)
Expand Down
6 changes: 3 additions & 3 deletions pydantic_ai/agent.py
Original file line number Diff line number Diff line change
Expand Up @@ -660,7 +660,7 @@ async def _handle_model_response(
return _MarkFinalResult(result_data)

if not model_response.calls:
raise exceptions.UnexpectedModelBehaviour('Received empty tool call message')
raise exceptions.UnexpectedModelBehavior('Received empty tool call message')

# otherwise we run all retriever functions in parallel
messages: list[_messages.Message] = []
Expand Down Expand Up @@ -723,7 +723,7 @@ async def _handle_streamed_model_response(
pass
structured_msg = model_response.get()
if not structured_msg.calls:
raise exceptions.UnexpectedModelBehaviour('Received empty tool call message')
raise exceptions.UnexpectedModelBehavior('Received empty tool call message')
messages: list[_messages.Message] = [structured_msg]

# we now run all retriever functions in parallel
Expand All @@ -748,7 +748,7 @@ async def _validate_result(
def _incr_result_retry(self) -> None:
self._current_result_retry += 1
if self._current_result_retry > self._max_result_retries:
raise exceptions.UnexpectedModelBehaviour(
raise exceptions.UnexpectedModelBehavior(
f'Exceeded maximum retries ({self._max_result_retries}) for result validation'
)

Expand Down
4 changes: 2 additions & 2 deletions pydantic_ai/exceptions.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

import json

__all__ = 'ModelRetry', 'UserError', 'UnexpectedModelBehaviour'
__all__ = 'ModelRetry', 'UserError', 'UnexpectedModelBehavior'


class ModelRetry(Exception):
Expand Down Expand Up @@ -30,7 +30,7 @@ def __init__(self, message: str):
super().__init__(message)


class UnexpectedModelBehaviour(RuntimeError):
class UnexpectedModelBehavior(RuntimeError):
"""Error caused by unexpected Model behavior, e.g. an unexpected response code."""

message: str
Expand Down
14 changes: 7 additions & 7 deletions pydantic_ai/models/gemini.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@
from pydantic import Discriminator, Field, Tag
from typing_extensions import NotRequired, TypedDict, TypeGuard, assert_never

from .. import UnexpectedModelBehaviour, _pydantic, _utils, exceptions, result
from .. import UnexpectedModelBehavior, _pydantic, _utils, exceptions, result
from ..messages import (
ArgsObject,
Message,
Expand Down Expand Up @@ -192,7 +192,7 @@ async def _make_request(self, messages: list[Message], streamed: bool) -> AsyncI
async with self.http_client.stream('POST', url, content=request_json, headers=headers) as r:
if r.status_code != 200:
await r.aread()
raise exceptions.UnexpectedModelBehaviour(f'Unexpected response from gemini {r.status_code}', r.text)
raise exceptions.UnexpectedModelBehavior(f'Unexpected response from gemini {r.status_code}', r.text)
yield r

@staticmethod
Expand Down Expand Up @@ -223,7 +223,7 @@ async def _process_streamed_response(http_response: HTTPResponse) -> EitherStrea
break

if start_response is None:
raise UnexpectedModelBehaviour('Streamed response ended without content or tool calls')
raise UnexpectedModelBehavior('Streamed response ended without content or tool calls')

if _extract_response_parts(start_response).is_left():
return GeminiStreamStructuredResponse(_content=content, _stream=aiter_bytes)
Expand Down Expand Up @@ -287,7 +287,7 @@ def get(self, *, final: bool = False) -> Iterable[str]:
for part in parts:
yield part['text']
else:
raise UnexpectedModelBehaviour(
raise UnexpectedModelBehavior(
'Streamed response with unexpected content, expected all parts to be text'
)

Expand Down Expand Up @@ -334,7 +334,7 @@ def get(self, *, final: bool = False) -> ModelStructuredResponse:
combined_parts.extend(parts)
elif not candidate.get('finish_reason'):
# you can get an empty text part along with the finish_reason, so we ignore that case
raise UnexpectedModelBehaviour(
raise UnexpectedModelBehavior(
'Streamed response with unexpected content, expected all parts to be function calls'
)
return _structured_response_from_parts(combined_parts, timestamp=self._timestamp)
Expand Down Expand Up @@ -534,14 +534,14 @@ def _extract_response_parts(
Returns Either a list of function calls (Either.left) or a list of text parts (Either.right).
"""
if len(response['candidates']) != 1:
raise UnexpectedModelBehaviour('Expected exactly one candidate in Gemini response')
raise UnexpectedModelBehavior('Expected exactly one candidate in Gemini response')
parts = response['candidates'][0]['content']['parts']
if _all_function_call_parts(parts):
return _utils.Either(left=parts)
elif _all_text_parts(parts):
return _utils.Either(right=parts)
else:
raise exceptions.UnexpectedModelBehaviour(
raise exceptions.UnexpectedModelBehavior(
f'Unsupported response from Gemini, expected all parts to be function calls or text, got: {parts!r}'
)

Expand Down
6 changes: 3 additions & 3 deletions pydantic_ai/models/openai.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
from openai.types.chat.chat_completion_chunk import ChoiceDeltaToolCall
from typing_extensions import assert_never

from .. import UnexpectedModelBehaviour, _utils, result
from .. import UnexpectedModelBehavior, _utils, result
from ..messages import (
ArgsJson,
Message,
Expand Down Expand Up @@ -184,7 +184,7 @@ async def _process_streamed_response(response: AsyncStream[ChatCompletionChunk])
try:
first_chunk = await response.__anext__()
except StopAsyncIteration as e: # pragma: no cover
raise UnexpectedModelBehaviour('Streamed response ended without content or tool calls') from e
raise UnexpectedModelBehavior('Streamed response ended without content or tool calls') from e
timestamp = datetime.fromtimestamp(first_chunk.created, tz=timezone.utc)
delta = first_chunk.choices[0].delta
start_cost = _map_cost(first_chunk)
Expand All @@ -194,7 +194,7 @@ async def _process_streamed_response(response: AsyncStream[ChatCompletionChunk])
try:
next_chunk = await response.__anext__()
except StopAsyncIteration as e:
raise UnexpectedModelBehaviour('Streamed response ended without content or tool calls') from e
raise UnexpectedModelBehavior('Streamed response ended without content or tool calls') from e
delta = next_chunk.choices[0].delta
start_cost += _map_cost(next_chunk)

Expand Down
2 changes: 1 addition & 1 deletion pydantic_ai/result.py
Original file line number Diff line number Diff line change
Expand Up @@ -279,7 +279,7 @@ async def validate_structured_result(
assert self._result_schema is not None, 'Expected _result_schema to not be None'
match = self._result_schema.find_tool(message)
if match is None:
raise exceptions.UnexpectedModelBehaviour(
raise exceptions.UnexpectedModelBehavior(
f'Invalid message, unable to find tool: {self._result_schema.tool_names()}'
)

Expand Down
10 changes: 5 additions & 5 deletions tests/models/test_gemini.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
from pydantic import BaseModel
from typing_extensions import Literal, TypeAlias

from pydantic_ai import Agent, ModelRetry, UnexpectedModelBehaviour, UserError, _utils
from pydantic_ai import Agent, ModelRetry, UnexpectedModelBehavior, UserError, _utils
from pydantic_ai.messages import (
ArgsObject,
ModelStructuredResponse,
Expand Down Expand Up @@ -511,7 +511,7 @@ def handler(_: httpx.Request):
m = GeminiModel('gemini-1.5-flash', http_client=gemini_client)
agent = Agent(m, system_prompt='this is the system prompt')

with pytest.raises(UnexpectedModelBehaviour) as exc_info:
with pytest.raises(UnexpectedModelBehavior) as exc_info:
await agent.run('Hello')

assert str(exc_info.value) == snapshot('Unexpected response from gemini 401, body:\ninvalid request')
Expand All @@ -535,7 +535,7 @@ async def test_heterogeneous_responses(get_gemini_client: GetGeminiClient):
gemini_client = get_gemini_client(response)
m = GeminiModel('gemini-1.5-flash', http_client=gemini_client)
agent = Agent(m)
with pytest.raises(UnexpectedModelBehaviour) as exc_info:
with pytest.raises(UnexpectedModelBehavior) as exc_info:
await agent.run('Hello')

assert str(exc_info.value) == snapshot(
Expand Down Expand Up @@ -573,7 +573,7 @@ async def test_stream_text_no_data(get_gemini_client: GetGeminiClient):
gemini_client = get_gemini_client(stream)
m = GeminiModel('gemini-1.5-flash', http_client=gemini_client)
agent = Agent(m)
with pytest.raises(UnexpectedModelBehaviour, match='Streamed response ended without con'):
with pytest.raises(UnexpectedModelBehavior, match='Streamed response ended without con'):
async with agent.run_stream('Hello'):
pass

Expand Down Expand Up @@ -691,5 +691,5 @@ async def test_stream_text_heterogeneous(get_gemini_client: GetGeminiClient):

msg = 'Streamed response with unexpected content, expected all parts to be text'
async with agent.run_stream('Hello') as result:
with pytest.raises(UnexpectedModelBehaviour, match=msg):
with pytest.raises(UnexpectedModelBehavior, match=msg):
await result.get_data()
4 changes: 2 additions & 2 deletions tests/models/test_openai.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
from openai.types.completion_usage import CompletionUsage, PromptTokensDetails
from typing_extensions import TypedDict

from pydantic_ai import Agent, ModelRetry, UnexpectedModelBehaviour, _utils
from pydantic_ai import Agent, ModelRetry, UnexpectedModelBehavior, _utils
from pydantic_ai.messages import (
ArgsJson,
ModelStructuredResponse,
Expand Down Expand Up @@ -424,6 +424,6 @@ async def test_no_content(allow_model_requests: None):
m = OpenAIModel('gpt-4', openai_client=mock_client)
agent = Agent(m, result_type=MyTypedDict)

with pytest.raises(UnexpectedModelBehaviour, match='Streamed response ended without con'):
with pytest.raises(UnexpectedModelBehavior, match='Streamed response ended without con'):
async with agent.run_stream(''):
pass
6 changes: 3 additions & 3 deletions tests/test_agent.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
from inline_snapshot import snapshot
from pydantic import BaseModel

from pydantic_ai import Agent, CallContext, ModelRetry, UnexpectedModelBehaviour, UserError
from pydantic_ai import Agent, CallContext, ModelRetry, UnexpectedModelBehavior, UserError
from pydantic_ai.messages import (
ArgsJson,
ArgsObject,
Expand Down Expand Up @@ -445,7 +445,7 @@ def empty(_: list[Message], _info: AgentInfo) -> ModelAnyResponse:

agent = Agent(FunctionModel(empty))

with pytest.raises(UnexpectedModelBehaviour, match='Received empty tool call message'):
with pytest.raises(UnexpectedModelBehavior, match='Received empty tool call message'):
agent.run_sync('Hello')


Expand All @@ -455,7 +455,7 @@ def empty(_: list[Message], _info: AgentInfo) -> ModelAnyResponse:

agent = Agent(FunctionModel(empty))

with pytest.raises(UnexpectedModelBehaviour, match=r'Exceeded maximum retries \(1\) for result validation'):
with pytest.raises(UnexpectedModelBehavior, match=r'Exceeded maximum retries \(1\) for result validation'):
agent.run_sync('Hello')
assert agent.last_run_messages == snapshot(
[
Expand Down
8 changes: 4 additions & 4 deletions tests/test_streaming.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
import pytest
from inline_snapshot import snapshot

from pydantic_ai import Agent, UnexpectedModelBehaviour, UserError
from pydantic_ai import Agent, UnexpectedModelBehavior, UserError
from pydantic_ai.messages import (
ArgsJson,
ArgsObject,
Expand Down Expand Up @@ -154,7 +154,7 @@ async def text_stream(_messages: list[Message], _: AgentInfo) -> AsyncIterator[s

agent = Agent(FunctionModel(stream_function=text_stream), result_type=tuple[str, str])

with pytest.raises(UnexpectedModelBehaviour, match=r'Exceeded maximum retries \(1\) for result validation'):
with pytest.raises(UnexpectedModelBehavior, match=r'Exceeded maximum retries \(1\) for result validation'):
async with agent.run_stream(''):
pass

Expand Down Expand Up @@ -232,7 +232,7 @@ async def stream_structured_function(_messages: list[Message], _: AgentInfo) ->

agent = Agent(FunctionModel(stream_function=stream_structured_function), result_type=tuple[str, int])

with pytest.raises(UnexpectedModelBehaviour, match='Received empty tool call message'):
with pytest.raises(UnexpectedModelBehavior, match='Received empty tool call message'):
async with agent.run_stream('hello'):
pass

Expand All @@ -247,7 +247,7 @@ async def stream_structured_function(_messages: list[Message], _: AgentInfo) ->
async def ret_a(x: str) -> str:
return x

with pytest.raises(UnexpectedModelBehaviour, match=r'Exceeded maximum retries \(1\) for result validation'):
with pytest.raises(UnexpectedModelBehavior, match=r'Exceeded maximum retries \(1\) for result validation'):
async with agent.run_stream('hello'):
pass
assert agent.last_run_messages == snapshot(
Expand Down
Loading