Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 44 additions & 42 deletions docs/agents.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,10 @@ but multiple agents can also interact to embody more complex workflows.
The [`Agent`][pydantic_ai.Agent] class has full API documentation, but conceptually you can think of an agent as a container for:

* A [system prompt](#system-prompts) — a set of instructions for the LLM written by the developer
* One or more [retrievers](#retrievers) — functions that the LLM may call to get information while generating a response
* One or more [retrieval tool](#tools) — functions that the LLM may call to get information while generating a response
* An optional structured [result type](results.md) — the structured datatype the LLM must return at the end of a run
* A [dependency](dependencies.md) type constraint — system prompt functions, retrievers and result validators may all use dependencies when they're run
* Agents may optionally also have a default [model](api/models/base.md) associated with them; the model to use can also be specified when running the agent
* A [dependency](dependencies.md) type constraint — system prompt functions, tools and result validators may all use dependencies when they're run
* Agents may optionally also have a default [LLM model](api/models/base.md) associated with them; the model to use can also be specified when running the agent

In typing terms, agents are generic in their dependency and result types, e.g., an agent which required dependencies of type `#!python Foobar` and returned results of type `#!python list[str]` would have type `#!python Agent[Foobar, list[str]]`.

Expand All @@ -31,7 +31,7 @@ roulette_agent = Agent( # (1)!
)


@roulette_agent.retriever
@roulette_agent.tool
async def roulette_wheel(ctx: CallContext[int], square: int) -> str: # (2)!
"""check if the square is a winner"""
return 'winner' if square == ctx.deps else 'loser'
Expand All @@ -49,7 +49,7 @@ print(result.data)
```

1. Create an agent, which expects an integer dependency and returns a boolean result. This agent will have type `#!python Agent[int, bool]`.
2. Define a retriever that checks if the square is a winner. Here [`CallContext`][pydantic_ai.dependencies.CallContext] is parameterized with the dependency type `int`; if you got the dependency type wrong you'd get a typing error.
2. Define a tool that checks if the square is a winner. Here [`CallContext`][pydantic_ai.dependencies.CallContext] is parameterized with the dependency type `int`; if you got the dependency type wrong you'd get a typing error.
3. In reality, you might want to use a random number here e.g. `random.randint(0, 36)`.
4. `result.data` will be a boolean indicating if the square is a winner. Pydantic performs the result validation, it'll be typed as a `bool` since its type is derived from the `result_type` generic parameter of the agent.

Expand Down Expand Up @@ -166,23 +166,23 @@ print(result.data)

_(This example is complete, it can be run "as is")_

## Retrievers
## Function Tools

Retrievers provide a mechanism for models to request extra information to help them generate a response.
Function tools provide a mechanism for models to retrieve extra information to help them generate a response.

They're useful when it is impractical or impossible to put all the context an agent might need into the system prompt, or when you want to make agents' behavior more deterministic or reliable by deferring some of the logic required to generate a response to another (not necessarily AI-powered) tool.

!!! info "Retrievers vs. RAG"
Retrievers are basically the "R" of RAG (Retrieval-Augmented Generation) — they augment what the model can do by letting it request extra information.
!!! info "Function tools vs. RAG"
Function tools are basically the "R" of RAG (Retrieval-Augmented Generation) — they augment what the model can do by letting it request extra information.

The main semantic difference between PydanticAI Retrievers and RAG is RAG is synonymous with vector search, while PydanticAI retrievers are more general-purpose. (Note: we may add support for vector search functionality in the future, particularly an API for generating embeddings. See [#58](https://github.com/pydantic/pydantic-ai/issues/58))
The main semantic difference between PydanticAI Tools and RAG is RAG is synonymous with vector search, while PydanticAI tools are more general-purpose. (Note: we may add support for vector search functionality in the future, particularly an API for generating embeddings. See [#58](https://github.com/pydantic/pydantic-ai/issues/58))

There are two different decorator functions to register retrievers:
There are two different decorator functions to register tools:

1. [`@agent.retriever_plain`][pydantic_ai.Agent.retriever_plain] — for retrievers that don't need access to the agent [context][pydantic_ai.dependencies.CallContext]
2. [`@agent.retriever`][pydantic_ai.Agent.retriever] — for retrievers that do need access to the agent [context][pydantic_ai.dependencies.CallContext]
1. [`@agent.tool`][pydantic_ai.Agent.tool] — for tools that need access to the agent [context][pydantic_ai.dependencies.CallContext]
2. [`@agent.tool_plain`][pydantic_ai.Agent.tool_plain] — for tools that do not need access to the agent [context][pydantic_ai.dependencies.CallContext]

`@agent.retriever` is the default since in the majority of cases retrievers will need access to the agent context.
`@agent.tool` is the default since in the majority of cases tools will need access to the agent context.

Here's an example using both:

Expand All @@ -202,13 +202,13 @@ agent = Agent(
)


@agent.retriever_plain # (3)!
@agent.tool_plain # (3)!
def roll_die() -> str:
"""Roll a six-sided die and return the result."""
return str(random.randint(1, 6))


@agent.retriever # (4)!
@agent.tool # (4)!
def get_player_name(ctx: CallContext[str]) -> str:
"""Get the player's name."""
return ctx.deps
Expand All @@ -221,8 +221,8 @@ print(dice_result.data)

1. This is a pretty simple task, so we can use the fast and cheap Gemini flash model.
2. We pass the user's name as the dependency, to keep things simple we use just the name as a string as the dependency.
3. This retriever doesn't need any context, it just returns a random number. You could probably use a dynamic system prompt in this case.
4. This retriever needs the player's name, so it uses `CallContext` to access dependencies which are just the player's name in this case.
3. This tool doesn't need any context, it just returns a random number. You could probably use a dynamic system prompt in this case.
4. This tool needs the player's name, so it uses `CallContext` to access dependencies which are just the player's name in this case.
5. Run the agent, passing the player's name as the dependency.

_(This example is complete, it can be run "as is")_
Expand Down Expand Up @@ -297,19 +297,19 @@ sequenceDiagram
Note over Agent: Send prompts
Agent ->> LLM: System: "You're a dice game..."<br>User: "My guess is 4"
activate LLM
Note over LLM: LLM decides to use<br>a retriever
Note over LLM: LLM decides to use<br>a tool

LLM ->> Agent: Call retriever<br>roll_die()
LLM ->> Agent: Call tool<br>roll_die()
deactivate LLM
activate Agent
Note over Agent: Rolls a six-sided die

Agent -->> LLM: ToolReturn<br>"4"
deactivate Agent
activate LLM
Note over LLM: LLM decides to use<br>another retriever
Note over LLM: LLM decides to use<br>another tool

LLM ->> Agent: Call retriever<br>get_player_name()
LLM ->> Agent: Call tool<br>get_player_name()
deactivate LLM
activate Agent
Note over Agent: Retrieves player name
Expand All @@ -323,27 +323,29 @@ sequenceDiagram
Note over Agent: Game session complete
```

### Retrievers, tools, and schema
### Function Tools vs. Structured Results

Under the hood, retrievers use the model's "tools" or "functions" API to let the model know what retrievers are available to call. Tools or functions are also used to define the schema(s) for structured responses, thus a model might have access to many tools, some of which call retrievers while others end the run and return a result.
As the name suggests, function tools use the model's "tools" or "functions" API to let the model know what is available to call. Tools or functions are also used to define the schema(s) for structured responses, thus a model might have access to many tools, some of which call function tools while others end the run and return a result.

### Function tools and schema

Function parameters are extracted from the function signature, and all parameters except `CallContext` are used to build the schema for that tool call.

Even better, PydanticAI extracts the docstring from retriever functions and (thanks to [griffe](https://mkdocstrings.github.io/griffe/)) extracts parameter descriptions from the docstring and adds them to the schema.
Even better, PydanticAI extracts the docstring from functions and (thanks to [griffe](https://mkdocstrings.github.io/griffe/)) extracts parameter descriptions from the docstring and adds them to the schema.

[Griffe supports](https://mkdocstrings.github.io/griffe/reference/docstrings/#docstrings) extracting parameter descriptions from `google`, `numpy` and `sphinx` style docstrings, and PydanticAI will infer the format to use based on the docstring. We plan to add support in the future to explicitly set the style to use, and warn/error if not all parameters are documented; see [#59](https://github.com/pydantic/pydantic-ai/issues/59).

To demonstrate a retriever's schema, here we use [`FunctionModel`][pydantic_ai.models.function.FunctionModel] to print the schema a model would receive:
To demonstrate a tool's schema, here we use [`FunctionModel`][pydantic_ai.models.function.FunctionModel] to print the schema a model would receive:

```py title="retriever_schema.py"
```py title="tool_schema.py"
from pydantic_ai import Agent
from pydantic_ai.messages import Message, ModelAnyResponse, ModelTextResponse
from pydantic_ai.models.function import AgentInfo, FunctionModel

agent = Agent()


@agent.retriever_plain
@agent.tool_plain
def foobar(a: int, b: str, c: dict[str, list[float]]) -> str:
"""Get me foobar.

Expand All @@ -356,10 +358,10 @@ def foobar(a: int, b: str, c: dict[str, list[float]]) -> str:


def print_schema(messages: list[Message], info: AgentInfo) -> ModelAnyResponse:
retriever = info.retrievers['foobar']
print(retriever.description)
tool = info.function_tools['foobar']
print(tool.description)
#> Get me foobar.
print(retriever.json_schema)
print(tool.json_schema)
"""
{
'description': 'Get me foobar.',
Expand All @@ -386,22 +388,22 @@ agent.run_sync('hello', model=FunctionModel(print_schema))

_(This example is complete, it can be run "as is")_

The return type of retriever can be any valid JSON object ([`JsonData`][pydantic_ai.dependencies.JsonData]) as some models (e.g. Gemini) support semi-structured return values, some expect text (OpenAI) but seem to be just as good at extracting meaning from the data. If a Python object is returned and the model expects a string, the value will be serialized to JSON.
The return type of tool can be any valid JSON object ([`JsonData`][pydantic_ai.dependencies.JsonData]) as some models (e.g. Gemini) support semi-structured return values, some expect text (OpenAI) but seem to be just as good at extracting meaning from the data. If a Python object is returned and the model expects a string, the value will be serialized to JSON.

If a retriever has a single parameter that can be represented as an object in JSON schema (e.g. dataclass, TypedDict, pydantic model), the schema for the retriever is simplified to be just that object. (TODO example)
If a tool has a single parameter that can be represented as an object in JSON schema (e.g. dataclass, TypedDict, pydantic model), the schema for the tool is simplified to be just that object. (TODO example)

## Reflection and self-correction

Validation errors from both retriever parameter validation and [structured result validation](results.md#structured-result-validation) can be passed back to the model with a request to retry.
Validation errors from both tool parameter validation and [structured result validation](results.md#structured-result-validation) can be passed back to the model with a request to retry.

You can also raise [`ModelRetry`][pydantic_ai.exceptions.ModelRetry] from within a [retriever](#retrievers) or [result validator function](results.md#result-validators-functions) to tell the model it should retry generating a response.
You can also raise [`ModelRetry`][pydantic_ai.exceptions.ModelRetry] from within a [tool](#tools) or [result validator function](results.md#result-validators-functions) to tell the model it should retry generating a response.

- The default retry count is **1** but can be altered for the [entire agent][pydantic_ai.Agent.__init__], a [specific retriever][pydantic_ai.Agent.retriever], or a [result validator][pydantic_ai.Agent.__init__].
- You can access the current retry count from within a retriever or result validator via [`ctx.retry`][pydantic_ai.dependencies.CallContext].
- The default retry count is **1** but can be altered for the [entire agent][pydantic_ai.Agent.__init__], a [specific tool][pydantic_ai.Agent.tool], or a [result validator][pydantic_ai.Agent.__init__].
- You can access the current retry count from within a tool or result validator via [`ctx.retry`][pydantic_ai.dependencies.CallContext].

Here's an example:

```py title="retriever_retry.py"
```py title="tool_retry.py"
from fake_database import DatabaseConn
from pydantic import BaseModel

Expand All @@ -420,7 +422,7 @@ agent = Agent(
)


@agent.retriever(retries=2)
@agent.tool(retries=2)
def get_user_by_name(ctx: CallContext[DatabaseConn], name: str) -> int:
"""Get a user's ID from their full name."""
print(name)
Expand Down Expand Up @@ -455,7 +457,7 @@ from pydantic_ai import Agent, ModelRetry, UnexpectedModelBehavior
agent = Agent('openai:gpt-4o')


@agent.retriever_plain
@agent.tool_plain
def calc_volume(size: int) -> int: # (1)!
if size == 42:
return size**3
Expand All @@ -467,7 +469,7 @@ try:
result = agent.run_sync('Please get me the volume of a box with size 6.')
except UnexpectedModelBehavior as e:
print('An error occurred:', e)
#> An error occurred: Retriever exceeded max retries count of 1
#> An error occurred: Tool exceeded max retries count of 1
print('cause:', repr(e.__cause__))
#> cause: ModelRetry('Please try again.')
print('messages:', agent.last_run_messages)
Expand Down Expand Up @@ -513,6 +515,6 @@ except UnexpectedModelBehavior as e:
else:
print(result.data)
```
1. Define a retriever that will raise `ModelRetry` repeatedly in this case.
1. Define a tool that will raise `ModelRetry` repeatedly in this case.

_(This example is complete, it can be run "as is")_
4 changes: 2 additions & 2 deletions docs/api/agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,6 @@
- override_model
- last_run_messages
- system_prompt
- retriever
- retriever_plain
- tool
- tool_plain
- result_validator
15 changes: 7 additions & 8 deletions docs/dependencies.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,12 @@
# Dependencies

PydanticAI uses a dependency injection system to provide data and services to your agent's [system prompts](agents.md#system-prompts), [retrievers](agents.md#retrievers) and [result validators](results.md#result-validators-functions).
PydanticAI uses a dependency injection system to provide data and services to your agent's [system prompts](agents.md#system-prompts), [tools](agents.md#tools) and [result validators](results.md#result-validators-functions).

Matching PydanticAI's design philosophy, our dependency system tries to use existing best practice in Python development rather than inventing esoteric "magic", this should make dependencies type-safe, understandable easier to test and ultimately easier to deploy in production.

## Defining Dependencies

Dependencies can be any python type. While in simple cases you might be able to pass a single object
as a dependency (e.g. an HTTP connection), [dataclasses][] are generally a convenient container when your dependencies included multiple objects.
Dependencies can be any python type. While in simple cases you might be able to pass a single object as a dependency (e.g. an HTTP connection), [dataclasses][] are generally a convenient container when your dependencies included multiple objects.

Here's an example of defining an agent that requires dependencies.

Expand Down Expand Up @@ -102,7 +101,7 @@ _(This example is complete, it can be run "as is")_

### Asynchronous vs. Synchronous dependencies

System prompt functions, retriever functions and result validator are all run in the async context of an agent run.
[System prompt functions](agents.md#system-prompts), [function tools](agents.md#function-tools) and [result validators](results.md#result-validators-functions) are all run in the async context of an agent run.

If these functions are not coroutines (e.g. `async def`) they are called with
[`run_in_executor`][asyncio.loop.run_in_executor] in a thread pool, it's therefore marginally preferable
Expand Down Expand Up @@ -159,7 +158,7 @@ _(This example is complete, it can be run "as is")_

## Full Example

As well as system prompts, dependencies can be used in [retrievers](agents.md#retrievers) and [result validators](results.md#result-validators-functions).
As well as system prompts, dependencies can be used in [tools](agents.md#tools) and [result validators](results.md#result-validators-functions).

```py title="full_example.py" hl_lines="27-35 38-48"
from dataclasses import dataclass
Expand Down Expand Up @@ -188,7 +187,7 @@ async def get_system_prompt(ctx: CallContext[MyDeps]) -> str:
return f'Prompt: {response.text}'


@agent.retriever # (1)!
@agent.tool # (1)!
async def get_joke_material(ctx: CallContext[MyDeps], subject: str) -> str:
response = await ctx.deps.http_client.get(
'https://example.com#jokes',
Expand Down Expand Up @@ -220,7 +219,7 @@ async def main():
#> Did you hear about the toothpaste scandal? They called it Colgate.
```

1. To pass `CallContext` and to a retriever, us the [`retriever`][pydantic_ai.Agent.retriever] decorator.
1. To pass `CallContext` to a tool, use the [`tool`][pydantic_ai.Agent.tool] decorator.
2. `CallContext` may optionally be passed to a [`result_validator`][pydantic_ai.Agent.result_validator] function as the first argument.

_(This example is complete, it can be run "as is")_
Expand Down Expand Up @@ -324,7 +323,7 @@ joke_agent = Agent(
factory_agent = Agent('gemini-1.5-pro', result_type=list[str])


@joke_agent.retriever
@joke_agent.tool
async def joke_factory(ctx: CallContext[MyDeps], count: int) -> str:
r = await ctx.deps.factory_agent.run(f'Please generate {count} jokes.')
return '\n'.join(r.data)
Expand Down
2 changes: 1 addition & 1 deletion docs/examples/bank-support.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Demonstrates:

* [dynamic system prompt](../agents.md#system-prompts)
* [structured `result_type`](../results.md#structured-result-validation)
* [retrievers](../agents.md#retrievers)
* [tools](../agents.md#tools)

## Running the Example

Expand Down
4 changes: 2 additions & 2 deletions docs/examples/rag.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,12 @@ RAG search example. This demo allows you to ask question of the [logfire](https:

Demonstrates:

* [retrievers](../agents.md#retrievers)
* [tools](../agents.md#tools)
* [agent dependencies](../dependencies.md)
* RAG search

This is done by creating a database containing each section of the markdown documentation, then registering
the search tool as a retriever with the PydanticAI agent.
the search tool with the PydanticAI agent.

Logic for extracting sections from markdown files and a JSON file with that data is available in
[this gist](https://gist.github.com/samuelcolvin/4b5bb9bb163b1122ff17e29e48c10992).
Expand Down
2 changes: 1 addition & 1 deletion docs/examples/weather-agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ Example of PydanticAI with multiple tools which the LLM needs to call in turn to

Demonstrates:

* [retrievers](../agents.md#retrievers)
* [tools](../agents.md#tools)
* [agent dependencies](../dependencies.md)

In this case the idea is a "weather" agent — the user can ask for the weather in multiple locations,
Expand Down
Loading
Loading