From 3e1c00319f30bd96b1d91e1ece53cfc9c9ff14b3 Mon Sep 17 00:00:00 2001 From: Samuel Colvin Date: Tue, 19 Nov 2024 09:42:27 +0000 Subject: [PATCH] tweaking index and other docs --- docs/index.md | 77 +++++++++++++++++----------- docs/install.md | 2 +- docs/logfire.md | 23 +++++++-- pydantic_ai_examples/bank_support.py | 2 +- 4 files changed, 67 insertions(+), 37 deletions(-) diff --git a/docs/index.md b/docs/index.md index 5e3dbdb8ce..e0230f3a9a 100644 --- a/docs/index.md +++ b/docs/index.md @@ -14,35 +14,43 @@ PydanticAI is a Python Agent Framework designed to make it less painful to build * Multi-model — currently with OpenAI and Gemini are support, Anthropic [coming soon](https://github.com/pydantic/pydantic-ai/issues/63), simply interface to implement other models or adapt existing ones * Type-safe * Built on tried and tested best practices in Python -* Structured response validation with Pydantic -* Streamed responses, including validation of streamed structured responses with Pydantic -* Novel, type-safe dependency injection system -* Logfire integration +* [Structured response](results.md#structured-result-validation) validation with Pydantic +* [Streamed responses](results.md#streamed-results) , including validation of streamed structured responses with Pydantic +* Novel, type-safe [dependency injection system](dependencies.md) +* [Logfire integration](logfire.md) for debugging and performance monitoring !!! example "In Beta" PydanticAI is in early beta, the API is subject to change and there's a lot more to do. [Feedback](https://github.com/pydantic/pydantic-ai/issues) is very welcome! -## Example — Hello World +## Hello World Example Here's a very minimal example of PydanticAI. ```py title="hello_world.py" from pydantic_ai import Agent -agent = Agent('gemini-1.5-flash', system_prompt='Be concise, reply with one sentence.') +agent = Agent( # (1)! + 'gemini-1.5-flash', + system_prompt='Be concise, reply with one sentence.', # (2)! +) -result = agent.run_sync('Where does "hello world" come from?') +result = agent.run_sync('Where does "hello world" come from?') # (3)! print(result.data) """ The first known use of "hello, world" was in a 1974 textbook about the C programming language. """ ``` -_(This example is complete, it can be run "as is")_ + +1. Define a very simple agent, here we configure the agent to use [Gemini 1.5's Flash](api/models/gemini.md) model, you can also set the model when running the agent. +2. Static [system prompts](agents.md#system-prompts) can be registered as keyword arguments to the agent. For more complex system prompts, see the example below. +3. [Run the agent](agents.md#running-agents) synchronously, conducting a conversation with the LLM, here the exchange should be very short: PydanticAI will send the system prompt and the user query to the LLM, the model will return a text response. + +4. _(This example is complete, it can be run "as is")_ Not very interesting yet, but we can easily add retrievers, dynamic system prompts and structured responses to build more powerful agents. -## Example — Retrievers and Dependency Injection +## Retrievers & Dependency Injection Example Small but complete example of using PydanticAI to build a support agent for a bank. @@ -59,10 +67,10 @@ from bank_database import DatabaseConn @dataclass class SupportDependencies: # (3)! customer_id: int - db: DatabaseConn + db: DatabaseConn # (12)! -class SupportResult(BaseModel): +class SupportResult(BaseModel): # (13)! support_advice: str = Field(description='Advice returned to the customer') block_card: bool = Field(description='Whether to block their') risk: int = Field(description='Risk level of query', ge=0, le=10) @@ -101,31 +109,34 @@ async def customer_balance( ... # (11)! -deps = SupportDependencies(customer_id=123, db=DatabaseConn()) -result = support_agent.run_sync('What is my balance?', deps=deps) # (8)! -print(result.data) # (10)! -""" -support_advice='Hello John, your current account balance, including pending transactions, is $123.45.' block_card=False risk=1 -""" +async def main(): + deps = SupportDependencies(customer_id=123, db=DatabaseConn()) + result = await support_agent.run('What is my balance?', deps=deps) # (8)! + print(result.data) # (10)! + """ + support_advice='Hello John, your current account balance, including pending transactions, is $123.45.' block_card=False risk=1 + """ -result = support_agent.run_sync('I just lost my card!', deps=deps) -print(result.data) -""" -support_advice="I'm sorry to hear that, John. We are temporarily blocking your card to prevent unauthorized transactions." block_card=True risk=8 -""" + result = await support_agent.run('I just lost my card!', deps=deps) + print(result.data) + """ + support_advice="I'm sorry to hear that, John. We are temporarily blocking your card to prevent unauthorized transactions." block_card=True risk=8 + """ ``` -1. An [agent](agents.md) that acts as first-tier support in a bank, agents are generic in the type of dependencies they take and the type of result they return, in this case `Deps` and `SupportResult`. -2. Here we configure the agent to use [OpenAI's GPT-4o model](api/models/openai.md), you can also customise the model when running the agent. -3. The `SupportDependencies` dataclass is used to pass data and connections into the model that will be needed when running [system prompts](agents.md#system-prompts) and [retrievers](agents.md#retrievers). PydanticAI's system of dependency injection provides a powerful, type safe way to customise the behaviour of your agents, including for unit tests and evals. -4. Static [system prompts](agents.md#system-prompts) can be registered as keyword arguments to the agent -5. dynamic [system prompts](agents.md#system-prompts) can be registered with the `@agent.system_prompot` decorator and benefit from dependency injection. -6. [Retrievers](agents.md#retrievers) let you register "tools" which the LLM may call while responding to a user. You inject dependencies into the retriever with [`CallContext`][pydantic_ai.dependencies.CallContext], any other arguments become the tool schema passed to the LLM, Pydantic is used to validate these arguments, errors are passed back to the LLM so it can retry. -7. The docstring is also passed to the LLM as a description of the tool. -8. [Run the agent](agents.md#running-agents) synchronously, conducting a conversation with the LLM until a final response is reached. +1. An [agent](agents.md) that acts as first-tier support in a bank, agents are generic in the type of dependencies they take and the type of result they return, in this case support agent has type `#!python Agent[SupportDependencies, SupportResult]`. +2. Here we configure the agent to use [OpenAI's GPT-4o model](api/models/openai.md), you can also set the model when running the agent. +3. The `SupportDependencies` dataclass is used to pass data, connections and logic into the model that will be needed when running [system prompts](agents.md#system-prompts) and [retrievers](agents.md#retrievers). PydanticAI's system of dependency injection provides a powerful, type safe way to customise the behaviour of your agents, including when unit tests and evals. +4. Static [system prompts](agents.md#system-prompts) can be registered as [keyword arguments][pydantic_ai.Agent.__init__] to the agent. +5. dynamic [system prompts](agents.md#system-prompts) can be registered with the [`@agent.system_prompt`][pydantic_ai.Agent.system_prompt] decorator and benefit from dependency injection. Dependencies are carried via the [`CallContext`][pydantic_ai.dependencies.CallContext] argument, this is parameterised with the `deps_type` from above, if the type annotation here is wrong, static type checkers will catch it. +6. [Retrievers](agents.md#retrievers) let you register "tools" which the LLM may call while responding to a user. Again dependencies are carried via [`CallContext`][pydantic_ai.dependencies.CallContext], any other arguments become the tool schema passed to the LLM, Pydantic is used to validate these arguments, errors are passed back to the LLM so it can retry. +7. The docstring is also passed to the LLM as a description of the tool. Parameter descriptions are [extracted](agents.md#retrievers-tools-and-schema) from the docstring and added to the tool schema sent to the LLM. +8. [Run the agent](agents.md#running-agents) asynchronously, conducting a conversation with the LLM until a final response is reached. Even in this fair simply case, the agent will exchange multiple messages with the LLM as retrievers are called to each a result. 9. The response from the agent will, be guaranteed to be a `SupportResult`, if validation fails [reflection](agents.md#reflection-and-self-correction) will mean the agent is prompted to try again. 10. The result will be validated with Pydantic to guarantee it is a `SupportResult`, since the agent is generic, it'll also be typed as a `SupportResult` to aid with static type checking. -11. In real use case, you'd add many more retrievers to the agent to extend the context it's equipped with and support it can provide. +11. In a real use case, you'd add many more retrievers and a longer system prompt to the agent to extend the context it's equipped with and support it can provide. +12. This is a simple sketch of a database connection, used to keep the example short and readable. In reality, you'd be connecting to an external database (e.g. PostgreSQL) to get information about customers. +13. This [Pydantic](https://docs.pydantic.dev) model is used to constrain the structured data returned by the agent. From this simple definition, Pydantic builds teh JSON Schema that tells the LLM how to return the data, and performs validation to guarantee the data is correct at the end of the conversation. !!! tip "Complete `bank_support.py` example" This example is incomplete for the sake of brevity (the definition of `DatabaseConn` is missing); you can find a complete `bank_support.py` example [here](examples/bank-support.md). @@ -133,3 +144,7 @@ support_advice="I'm sorry to hear that, John. We are temporarily blocking your c ## Next Steps To try PydanticAI yourself, follow instructions [in examples](examples/index.md). + +Read the conceptual [documentation](agents.md) to learn more about building applications with PydanticAI. + +Read the [API Reference](api/agent.md) to understand PydanticAI's interface. diff --git a/docs/install.md b/docs/install.md index 1bf921113c..2c33a32ae3 100644 --- a/docs/install.md +++ b/docs/install.md @@ -34,7 +34,7 @@ To use Logfire with PydanticAI, install PydanticAI with the `logfire` optional g uv add 'pydantic-ai[logfire]' ``` -From there, follow the [Logfire documentation](https://logfire.pydantic.dev/docs/) to configure Logfire. +From there, follow the [Logfire setup cods](logfire.md#integrating-logfire) to configure Logfire. ## Next Steps diff --git a/docs/logfire.md b/docs/logfire.md index 2781f7282d..89c52e7db4 100644 --- a/docs/logfire.md +++ b/docs/logfire.md @@ -1,9 +1,12 @@ # Monitoring and Performance Applications that use LLMs have some challenges that are well known and understood: LLMs are **slow**, **unreliable** and **expensive**. -These applications also have some challenges that most developers have encountered much less often: they're **fickle** and **non-deterministic**. Subtle changes in a prompt can completely change a model's performance, and there's no `EXPLAIN` query you can run to understand why. +These applications also have some challenges that most developers have encountered much less often: LLMs are **fickle** and **non-deterministic**. Subtle changes in a prompt can completely change a model's performance, and there's no `EXPLAIN` query you can run to understand why. -From a software engineers point of view, you can think of LLMs as the worst database you've ever heard of, but worse. +!!! danger + From a software engineers point of view, you can think of LLMs as the worst database you've ever heard of, but worse. + + If LLMs weren't so bloody useful, we'd never touch them. To build successful applications with LLMs, we need new tools to understand both model performance, and the behavior of applications that rely on them. @@ -13,14 +16,26 @@ LLM Observability tools that just let you understand how your model is performin [Pydantic Logfire](https://pydantic.dev/logfire) is an observability platform from the developers of Pydantic and PydanticAI, that aims to let you understand your entire application: Gen AI, classic predictive AI, HTTP traffic, database queries and everything else a modern application needs. -!!! note "Pydantic Logfire is a commercial product" +!!! tip "Pydantic Logfire is a commercial product" Logfire is a commercially supported, hosted platform with an extremely generous and perpetual free tier. You can sign up and start using Logfire in a couple of minutes. PydanticAI has built-in (but optional) support for Logfire via the [`logfire-api`](https://github.com/pydantic/logfire/tree/main/logfire-api) no-op package. -That means if the `logfire` package is installed, detailed information about agent runs is sent to Logfire. But if the `logfire` package is not installed, there's no overhead and nothing is sent. +That means if the `logfire` package is installed and configured, detailed information about agent runs is sent to Logfire. But if the `logfire` package is not installed, there's virtually no overhead and nothing is sent. Here's an example showing details of running the [Weather Agent](examples/weather-agent.md) in Logfire: ![Weather Agent Logfire](img/logfire-weather-agent.png) + +## Integrating Logfire + +TODO + +## Debugging + +TODO + +## Monitoring Performance + +TODO diff --git a/pydantic_ai_examples/bank_support.py b/pydantic_ai_examples/bank_support.py index 6872d01b04..b682e22039 100644 --- a/pydantic_ai_examples/bank_support.py +++ b/pydantic_ai_examples/bank_support.py @@ -16,7 +16,7 @@ class DatabaseConn: """This is a fake database for example purposes. In reality, you'd be connecting to an external database - to get information about customers. + (e.g. PostgreSQL) to get information about customers. """ @classmethod