diff --git a/docs/api/models/function.md b/docs/api/models/function.md index f83fcf4a3f..8e5af7d089 100644 --- a/docs/api/models/function.md +++ b/docs/api/models/function.md @@ -1,3 +1,10 @@ # `pydantic_ai.models.function` +A model controlled by a local function. + +[`FunctionModel`][pydantic_ai.models.function.FunctionModel] is similar to [`TestModel`][pydantic_ai.models.test.TestModel], +but allows greater control over the model's behavior. + +Its primary use case is for more advanced unit testing than is possible with `TestModel`. + ::: pydantic_ai.models.function diff --git a/docs/api/models/gemini.md b/docs/api/models/gemini.md index 5cf3315be0..24609a71fb 100644 --- a/docs/api/models/gemini.md +++ b/docs/api/models/gemini.md @@ -1,3 +1,20 @@ # `pydantic_ai.models.gemini` +Custom interface to the `generativelanguage.googleapis.com` API using +[HTTPX](https://www.python-httpx.org/) and [Pydantic](https://docs.pydantic.dev/latest/. + +The Google SDK for interacting with the `generativelanguage.googleapis.com` API +[`google-generativeai`](https://ai.google.dev/gemini-api/docs/quickstart?lang=python) reads like it was written by a +Java developer who thought they knew everything about OOP, spent 30 minutes trying to learn Python, +gave up and decided to build the library to prove how horrible Python is. It also doesn't use httpx for HTTP requests, +and tries to implement tool calling itself, but doesn't use Pydantic or equivalent for validation. + +We therefore implement support for the API directly. + +Despite these shortcomings, the Gemini model is actually quite powerful and very fast. + +## Setup + +For details on how to set up authentication with this model, see [model configuration for Gemini](../../install.md#gemini). + ::: pydantic_ai.models.gemini diff --git a/docs/api/models/groq.md b/docs/api/models/groq.md index d41a5b0acf..1eb07fcfb8 100644 --- a/docs/api/models/groq.md +++ b/docs/api/models/groq.md @@ -1,3 +1,7 @@ # `pydantic_ai.models.groq` +## Setup + +For details on how to set up authentication with this model, see [model configuration for Groq](../../install.md#groq). + ::: pydantic_ai.models.groq diff --git a/docs/api/models/openai.md b/docs/api/models/openai.md index ab3cedb646..5ae93a3eb4 100644 --- a/docs/api/models/openai.md +++ b/docs/api/models/openai.md @@ -1,3 +1,7 @@ # `pydantic_ai.models.openai` +## Setup + +For details on how to set up authentication with this model, see [model configuration for OpenAI](../../install.md#openai). + ::: pydantic_ai.models.openai diff --git a/docs/api/models/vertexai.md b/docs/api/models/vertexai.md index ace2daca4f..6e332ecc1f 100644 --- a/docs/api/models/vertexai.md +++ b/docs/api/models/vertexai.md @@ -1,3 +1,52 @@ # `pydantic_ai.models.vertexai` +Custom interface to the `*-aiplatform.googleapis.com` API for Gemini models. + +This model uses [`GeminiAgentModel`][pydantic_ai.models.gemini.GeminiAgentModel] with just the URL and auth method +changed from [`GeminiModel`][pydantic_ai.models.gemini.GeminiModel], it relies on the VertexAI +[`generateContent`](https://cloud.google.com/vertex-ai/docs/reference/rest/v1/projects.locations.endpoints/generateContent) +and +[`streamGenerateContent`](https://cloud.google.com/vertex-ai/docs/reference/rest/v1/projects.locations.endpoints/streamGenerateContent) +function endpoints +having the same schemas as the equivalent [Gemini endpoints][pydantic_ai.models.gemini.GeminiModel]. + +There are four advantages of using this API over the `generativelanguage.googleapis.com` API which +[`GeminiModel`][pydantic_ai.models.gemini.GeminiModel] uses, and one big disadvantage. + +## Setup + +For details on how to set up authentication with this model as well as a comparison with the `generativelanguage.googleapis.com` API used by [`GeminiModel`][pydantic_ai.models.gemini.GeminiModel], +see [model configuration for Gemini via VertexAI](../../install.md#gemini-via-vertexai). + +## Example Usage + +With the default google project already configured in your environment using "application default credentials": + +```py title="vertex_example_env.py" +from pydantic_ai import Agent +from pydantic_ai.models.vertexai import VertexAIModel + +model = VertexAIModel('gemini-1.5-flash') +agent = Agent(model) +result = agent.run_sync('Tell me a joke.') +print(result.data) +#> Did you hear about the toothpaste scandal? They called it Colgate. +``` + +Or using a service account JSON file: + +```py title="vertex_example_service_account.py" +from pydantic_ai import Agent +from pydantic_ai.models.vertexai import VertexAIModel + +model = VertexAIModel( + 'gemini-1.5-flash', + service_account_file='path/to/service-account.json', +) +agent = Agent(model) +result = agent.run_sync('Tell me a joke.') +print(result.data) +#> Did you hear about the toothpaste scandal? They called it Colgate. +``` + ::: pydantic_ai.models.vertexai diff --git a/docs/examples/index.md b/docs/examples/index.md index d5dcc29ab4..7c9d906615 100644 --- a/docs/examples/index.md +++ b/docs/examples/index.md @@ -20,12 +20,9 @@ If you clone the repo, you should instead use `uv sync --extra examples` to inst ### Setting model environment variables -All these examples will need you to set either: +These examples will need you to set up authentication with one or more of the LLMs, see the [model configuration](../install.md#model-configuration) docs for details on how to do this. -* `OPENAI_API_KEY` to use OpenAI models, go to [platform.openai.com](https://platform.openai.com/) and follow your nose until you find how to generate an API key -* or, `GEMINI_API_KEY` to use Google Gemini models, go to [aistudio.google.com](https://aistudio.google.com/) and do the same to generate an API key - -Then set the API key as an environment variable with: +TL;DR: in most cases you'll need to set one of the following environment variables: === "OpenAI" diff --git a/docs/install.md b/docs/install.md index 814af9f577..902be3e30d 100644 --- a/docs/install.md +++ b/docs/install.md @@ -1,19 +1,24 @@ -# Installation - -PydanticAI is available [on PyPI](https://pypi.org/project/pydantic-ai/) so installation is as simple as: +# Installation & Setup +PydanticAI is available on PyPI as [`pydantic-ai`](https://pypi.org/project/pydantic-ai/) so installation is as simple as: ```bash pip/uv-add pydantic-ai ``` -It requires Python 3.9+. +(Requires Python 3.9+) + +This installs the `pydantic_ai` package, core dependencies, and libraries required to use the following LLM APIs: + +* [OpenAI API](https://platform.openai.com/docs/overview) +* [Google VertexAI API](https://cloud.google.com/vertex-ai) for Gemini models +* [Groq API](https://console.groq.com/docs/overview) ## Use with Pydantic Logfire PydanticAI has an excellent (but completely optional) integration with [Pydantic Logfire](https://pydantic.dev/logfire) to help you view and understand agent runs. -To use Logfire with PydanticAI, install PydanticAI with the `logfire` optional group: +To use Logfire with PydanticAI, install `pydantic-ai` or `pydantic-ai-slim` with the `logfire` optional group: ```bash pip/uv-add 'pydantic-ai[logfire]' @@ -23,12 +28,298 @@ From there, follow the [Logfire setup docs](logfire.md#integrating-logfire) to c ## Running Examples -PydanticAI bundles its examples so you can run them very easily. +We distributes the [`pydantic_ai_examples`](https://github.com/pydantic/pydantic-ai/tree/main/pydantic_ai_examples) directory as a separate PyPI package ([`pydantic-ai-examples`](https://pypi.org/project/pydantic-ai-examples/)) to make examples extremely easy to customize and run. -To install extra dependencies required to run examples, install the `examples` optional group: +To install examples, use the `examples` optional group: ```bash pip/uv-add 'pydantic-ai[examples]' ``` -To run the examples, follow instructions [in the examples](examples/index.md). +To run the examples, follow instructions in the [examples docs](examples/index.md). + +## Slim Install + +If you know which model you're going to use and want to avoid installing superfluous package, you can use the [`pydantic-ai-slim`](https://pypi.org/project/pydantic-ai-slim/) package. + +If you're using just [`OpenAIModel`][pydantic_ai.models.openai.OpenAIModel], run: + +```bash +pip/uv-add 'pydantic-ai-slim[openai]' +``` + +If you're using just [`GeminiModel`][pydantic_ai.models.gemini.GeminiModel] (Gemini via the `generativelanguage.googleapis.com` API) no extra dependencies are required, run: + +```bash +pip/uv-add pydantic-ai-slim +``` + +If you're using just [`VertexAIModel`][pydantic_ai.models.vertexai.VertexAIModel], run: + +```bash +pip/uv-add 'pydantic-ai-slim[vertexai]' +``` + +To use just [`GroqModel`][pydantic_ai.models.groq.GroqModel], run: + +```bash +pip/uv-add 'pydantic-ai-slim[groq]' +``` + +You can install dependencies for multiple models and use cases, for example: + +```bash +pip/uv-add 'pydantic-ai-slim[openai,vertexai,logfire]' +``` + +## Model Configuration + +To use hosted commercial models, you need to configure your local environment with the appropriate API keys. + +### OpenAI + +To use OpenAI through their main API, go to [platform.openai.com](https://platform.openai.com/) and follow your nose until you find the place to generate an API key. + +#### Environment variable + +Once you have the API key, you can set it as an environment variable: + +```bash +export OPENAI_API_KEY='your-api-key' +``` + +You can then use [`OpenAIModel`][pydantic_ai.models.openai.OpenAIModel] by name: + +```py title="openai_model_by_name.py" +from pydantic_ai import Agent + +agent = Agent('openai:gpt-4o') +... +``` + +Or initialise the model directly with just the model name: + +```py title="openai_model_init.py" +from pydantic_ai import Agent +from pydantic_ai.models.openai import OpenAIModel + +model = OpenAIModel('gpt-4o') +agent = Agent(model) +... +``` + +#### `api_key` argument + +If you don't want to or can't set the environment variable, you can pass it at runtime via the [`api_key` argument][pydantic_ai.models.openai.OpenAIModel.__init__]: + +```py title="openai_model_api_key.py" +from pydantic_ai import Agent +from pydantic_ai.models.openai import OpenAIModel + +model = OpenAIModel('gpt-4o', api_key='your-api-key') +agent = Agent(model) +... +``` + +#### Custom OpenAI Client + +`OpenAIModel` also accepts a custom `AsyncOpenAI` client via the [`openai_client` parameter][pydantic_ai.models.openai.OpenAIModel.__init__], +so you can customise the `organization`, `project`, `base_url` etc. as defined in the [OpenAI API docs](https://platform.openai.com/docs/api-reference). + +You could also use the [`AsyncAzureOpenAI`](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/switching-endpoints) client to use the Azure OpenAI API. + +```py title="openai_azure.py" +from openai import AsyncAzureOpenAI + +from pydantic_ai import Agent +from pydantic_ai.models.openai import OpenAIModel + +client = AsyncAzureOpenAI( + azure_endpoint='...', + api_version='2024-07-01-preview', + api_key='your-api-key', +) + +model = OpenAIModel('gpt-4o', openai_client=client) +agent = Agent(model) +... +``` + +### Gemini + +[`GeminiModel`][pydantic_ai.models.gemini.GeminiModel] let's you use the Google's Gemini models through their `generativelanguage.googleapis.com` API. + +[`GeminiModelName`][pydantic_ai.models.gemini.GeminiModelName] contains a list of available Gemini models that can be used through this interface. + +!!! warning "For prototyping only" + Google themselves refer to this API as the "hobby" API, I've received 503 responses from it a number of times. + The API is easy to use and useful for prototyping and simple demos, but I would not rely on it in production. + + If you want to run Gemini models in production, you should use the [VertexAI API](#gemini-via-vertexai) described below. + +To use `GeminiModel`, go to [aistudio.google.com](https://aistudio.google.com/) and follow your nose until you find the place to generate an API key. + +#### Environment variable + +Once you have the API key, you can set it as an environment variable: + +```bash +export GEMINI_API_KEY=your-api-key +``` + +You can then use [`GeminiModel`][pydantic_ai.models.gemini.GeminiModel] by name: + +```py title="gemini_model_by_name.py" +from pydantic_ai import Agent + +agent = Agent('gemini-1.5-flash') +... +``` + +Or initialise the model directly with just the model name: + +```py title="gemini_model_init.py" +from pydantic_ai import Agent +from pydantic_ai.models.gemini import GeminiModel + +model = GeminiModel('gemini-1.5-flash') +agent = Agent(model) +... +``` + +#### `api_key` argument + +If you don't want to or can't set the environment variable, you can pass it at runtime via the [`api_key` argument][pydantic_ai.models.gemini.GeminiModel.__init__]: + +```py title="gemini_model_api_key.py" +from pydantic_ai import Agent +from pydantic_ai.models.gemini import GeminiModel + +model = GeminiModel('gemini-1.5-flash', api_key='your-api-key') +agent = Agent(model) +... +``` + +### Gemini via VertexAI + +To run Google's Gemini models in production, you should use [`VertexAIModel`][pydantic_ai.models.vertexai.VertexAIModel] which uses the `*-aiplatform.googleapis.com` API. + +[`GeminiModelName`][pydantic_ai.models.gemini.GeminiModelName] contains a list of available Gemini models that can be used through this interface. + +This interface has a number of advantages over `generativelanguage.googleapis.com` documented above: + +1. The VertexAI API is more reliably and marginally lower latency in our experience. +2. You can + [purchase provisioned throughput](https://cloud.google.com/vertex-ai/generative-ai/docs/provisioned-throughput#purchase-provisioned-throughput) + with VertexAI to guarantee capacity. +3. If you're running PydanticAI inside GCP, you don't need to set up authentication, it should "just work". +4. You can decide which region to use, which might be important from a regulatory perspective, + and might improve latency. + +The big disadvantage is that for local development you may need to create and configure a "service account", which I've found extremely painful to get right in the past. + +Whichever way you authenticate, you'll need to have VertexAI enabled in your GCP account. + +#### application default credentials + +Luckily if you're running PydanticAI inside GCP, or you have the [`gcloud` CLI](https://cloud.google.com/sdk/gcloud) installed and configured, you should be able to use `VertexAIModel` without any additional setup. + +To use `VertexAIModel`, with [application default credentials](https://cloud.google.com/docs/authentication/application-default-credentials) configured (e.g. with `gcloud`), you can simply use: + +```py title="vertexai_application_default_credentials.py" +from pydantic_ai import Agent +from pydantic_ai.models.vertexai import VertexAIModel + +model = VertexAIModel('gemini-1.5-flash') +agent = Agent(model) +... +``` + +Internally this uses [`google.auth.default()`](https://google-auth.readthedocs.io/en/master/reference/google.auth.html) from the `google-auth` package to obtain credentials. + +!!! note "Won't fail until `agent.run()`" + Because `google.auth.default()` requires network requests and can be slow, it's not run until you call `agent.run()`. Meaning any configuration or permissions error will only be raised when you try to use the model. To for this check to be run, call [`await model.agent_model({}, False, None)`][pydantic_ai.models.Model.agent_model]. + +You may also need to pass the [`project_id` argument to `VertexAIModel`][pydantic_ai.models.vertexai.VertexAIModel.__init__] if application default credentials don't set a project, if you pass `project_id` and it conflicts with the project set by application default credentials, an error is raised. + +#### service account + +If instead of application default credentials, you want to authenticate with a service account, you'll need to create a service account, add it to your GCP project (note: AFAIK this step is necessary even if you created the service account within the project), give that service account the "Vertex AI Service Agent" role, and download the service account JSON file. + +Once you have the JSON file, you can use it thus: + +```py title="vertexai_service_account.py" +from pydantic_ai import Agent +from pydantic_ai.models.vertexai import VertexAIModel + +model = VertexAIModel( + 'gemini-1.5-flash', + service_account_file='path/to/service-account.json', +) +agent = Agent(model) +... +``` + +#### Customising region + +Whichever way you authenticate, you can specify which region requests will be sent to via the [`region` argument][pydantic_ai.models.vertexai.VertexAIModel.__init__]. + +Using a region close to your application can improve latency and might be important from a regulatory perspective. + +```py title="vertexai_region.py" +from pydantic_ai import Agent +from pydantic_ai.models.vertexai import VertexAIModel + +model = VertexAIModel('gemini-1.5-flash', region='asia-east1') +agent = Agent(model) +... +``` + +[`VertexAiRegion`][pydantic_ai.models.vertexai.VertexAiRegion] contains a list of available regions. + +### Groq + +To use [Groq](https://groq.com/) through their API, go to [console.groq.com/keys](https://console.groq.com/keys) and follow your nose until you find the place to generate an API key. + +[`GroqModelName`][pydantic_ai.models.groq.GroqModelName] contains a list of available Groq models. + +#### Environment variable + +Once you have the API key, you can set it as an environment variable: + +```bash +export GROQ_API_KEY='your-api-key' +``` + +You can then use [`GroqModel`][pydantic_ai.models.groq.GroqModel] by name: + +```py title="groq_model_by_name.py" +from pydantic_ai import Agent + +agent = Agent('groq:llama-3.1-70b-versatile') +... +``` + +Or initialise the model directly with just the model name: + +```py title="groq_model_init.py" +from pydantic_ai import Agent +from pydantic_ai.models.groq import GroqModel + +model = GroqModel('llama-3.1-70b-versatile') +agent = Agent(model) +... +``` + +#### `api_key` argument + +If you don't want to or can't set the environment variable, you can pass it at runtime via the [`api_key` argument][pydantic_ai.models.groq.GroqModel.__init__]: + +```py title="groq_model_api_key.py" +from pydantic_ai import Agent +from pydantic_ai.models.groq import GroqModel + +model = GroqModel('llama-3.1-70b-versatile', api_key='your-api-key') +agent = Agent(model) +... +``` diff --git a/docs/logfire.md b/docs/logfire.md index 89c52e7db4..7cf4600b58 100644 --- a/docs/logfire.md +++ b/docs/logfire.md @@ -1,9 +1,10 @@ # Monitoring and Performance Applications that use LLMs have some challenges that are well known and understood: LLMs are **slow**, **unreliable** and **expensive**. + These applications also have some challenges that most developers have encountered much less often: LLMs are **fickle** and **non-deterministic**. Subtle changes in a prompt can completely change a model's performance, and there's no `EXPLAIN` query you can run to understand why. -!!! danger +!!! danger "Warning" From a software engineers point of view, you can think of LLMs as the worst database you've ever heard of, but worse. If LLMs weren't so bloody useful, we'd never touch them. diff --git a/pydantic_ai_examples/README.md b/pydantic_ai_examples/README.md index d1ea997ff9..dbc7e53c86 100644 --- a/pydantic_ai_examples/README.md +++ b/pydantic_ai_examples/README.md @@ -1,5 +1,11 @@ # PydanticAI Examples +[![CI](https://github.com/pydantic/pydantic-ai/actions/workflows/ci.yml/badge.svg?event=push)](https://github.com/pydantic/pydantic-ai/actions/workflows/ci.yml?query=branch%3Amain) +[![Coverage](https://coverage-badge.samuelcolvin.workers.dev/pydantic/pydantic-ai.svg)](https://coverage-badge.samuelcolvin.workers.dev/redirect/pydantic/pydantic-ai) +[![PyPI](https://img.shields.io/pypi/v/pydantic-ai.svg)](https://pypi.python.org/pypi/pydantic-ai) +[![versions](https://img.shields.io/pypi/pyversions/pydantic-ai.svg)](https://github.com/pydantic/pydantic-ai) +[![license](https://img.shields.io/github/license/pydantic/pydantic-ai.svg?v)](https://github.com/pydantic/pydantic-ai/blob/main/LICENSE) + Examples of how to use PydanticAI and what it can do. -Full documentation of these examples and how to run them is available at [ai.pydantic.dev/examples/](https://ai.pydantic.dev/examples/). +For full documentation of these examples and how to run them, see [ai.pydantic.dev/examples/](https://ai.pydantic.dev/examples/). diff --git a/pydantic_ai_examples/__main__.py b/pydantic_ai_examples/__main__.py index a694dc92c3..2233014127 100644 --- a/pydantic_ai_examples/__main__.py +++ b/pydantic_ai_examples/__main__.py @@ -1,6 +1,18 @@ -"""Very simply CLI to aid in running the examples, and for copying examples code to a new directory. +"""Very simply CLI to aid in copying examples code to a new directory. -See README.md for more information. +To run examples in place, run: + + uv run -m pydantic_ai_examples. + +For examples: + + uv run -m pydantic_ai_examples.pydantic_model + +To copy all examples to a new directory, run: + + uv run -m pydantic_ai_examples --copy-to + +See https://ai.pydantic.dev/examples/ for more information. """ import argparse diff --git a/pydantic_ai_slim/README.md b/pydantic_ai_slim/README.md index 0ba1b619f2..b70ca45647 100644 --- a/pydantic_ai_slim/README.md +++ b/pydantic_ai_slim/README.md @@ -1,7 +1,11 @@ -# Coming soon +# PydanticAI Slim [![CI](https://github.com/pydantic/pydantic-ai/actions/workflows/ci.yml/badge.svg?event=push)](https://github.com/pydantic/pydantic-ai/actions/workflows/ci.yml?query=branch%3Amain) [![Coverage](https://coverage-badge.samuelcolvin.workers.dev/pydantic/pydantic-ai.svg)](https://coverage-badge.samuelcolvin.workers.dev/redirect/pydantic/pydantic-ai) [![PyPI](https://img.shields.io/pypi/v/pydantic-ai.svg)](https://pypi.python.org/pypi/pydantic-ai) [![versions](https://img.shields.io/pypi/pyversions/pydantic-ai.svg)](https://github.com/pydantic/pydantic-ai) [![license](https://img.shields.io/github/license/pydantic/pydantic-ai.svg?v)](https://github.com/pydantic/pydantic-ai/blob/main/LICENSE) + +PydanticAI core logic with minimal required dependencies. + +For more information on how to use this package see [ai.pydantic.dev/install](https://ai.pydantic.dev/install/). diff --git a/pydantic_ai_slim/pydantic_ai/models/function.py b/pydantic_ai_slim/pydantic_ai/models/function.py index b9f4a4a811..135aa3cfe0 100644 --- a/pydantic_ai_slim/pydantic_ai/models/function.py +++ b/pydantic_ai_slim/pydantic_ai/models/function.py @@ -1,11 +1,3 @@ -"""A model controlled by a local function. - -[FunctionModel][pydantic_ai.models.function.FunctionModel] is similar to [TestModel][pydantic_ai.models.test.TestModel], -but allows greater control over the model's behavior. - -It's primary use case for more advanced unit testing than is possible with `TestModel`. -""" - from __future__ import annotations as _annotations import inspect diff --git a/pydantic_ai_slim/pydantic_ai/models/gemini.py b/pydantic_ai_slim/pydantic_ai/models/gemini.py index f0e06684b2..08241a7e5a 100644 --- a/pydantic_ai_slim/pydantic_ai/models/gemini.py +++ b/pydantic_ai_slim/pydantic_ai/models/gemini.py @@ -1,25 +1,3 @@ -"""Custom interface to the `generativelanguage.googleapis.com` API using [HTTPX] and [Pydantic]. - -The Google SDK for interacting with the `generativelanguage.googleapis.com` API -[`google-generativeai`](https://ai.google.dev/gemini-api/docs/quickstart?lang=python) reads like it was written by a -Java developer who thought they knew everything about OOP, spent 30 minutes trying to learn Python, -gave up and decided to build the library to prove how horrible Python is. It also doesn't use httpx for HTTP requests, -and tries to implement tool calling itself, but doesn't use Pydantic or equivalent for validation. - -We could also use the Google Vertex SDK, -[`google-cloud-aiplatform`](https://cloud.google.com/vertex-ai/docs/python-sdk/use-vertex-ai-python-sdk) -which uses the `*-aiplatform.googleapis.com` API, but that requires a service account for authentication -which is a faff to set up and manage. - -Both APIs claim compatibility with OpenAI's API, but that breaks down with even the simplest of requests, -hence this custom interface. - -Despite these limitations, the Gemini model is actually quite powerful and very fast. - -[HTTPX]: https://www.python-httpx.org/ -[Pydantic]: https://docs.pydantic.dev/latest/ -""" - from __future__ import annotations as _annotations import os diff --git a/pydantic_ai_slim/pydantic_ai/models/vertexai.py b/pydantic_ai_slim/pydantic_ai/models/vertexai.py index b28dbe2f67..e6d8d108c3 100644 --- a/pydantic_ai_slim/pydantic_ai/models/vertexai.py +++ b/pydantic_ai_slim/pydantic_ai/models/vertexai.py @@ -1,60 +1,3 @@ -"""Custom interface to the `*-aiplatform.googleapis.com` API for Gemini models. - -This model uses [`GeminiAgentModel`][pydantic_ai.models.gemini.GeminiAgentModel] with just the URL and auth method -changed from the default `GeminiModel`, it relies on the VertexAI -[`generateContent` function endpoint](https://cloud.google.com/vertex-ai/docs/reference/rest/v1/projects.locations.endpoints/generateContent) -and `streamGenerateContent` function endpoints -having the same schemas as the equivalent [Gemini endpoints][pydantic_ai.models.gemini.GeminiModel]. - -There are four advantages of using this API over the `generativelanguage.googleapis.com` API which -[`GeminiModel`][pydantic_ai.models.gemini.GeminiModel] uses, and one big disadvantage. - -Advantages: - -1. The VertexAI API seems to be less flakey, less likely to occasionally return a 503 response. -2. You can - [purchase provisioned throughput](https://cloud.google.com/vertex-ai/generative-ai/docs/provisioned-throughput#purchase-provisioned-throughput) - with VertexAI. -3. If you're running PydanticAI inside GCP, you don't need to set up authentication, it should "just work". -4. You can decide which region to use, which might be important from a regulatory perspective, - and might improve latency. - -Disadvantage: - -1. When authorization doesn't just work, it's much more painful to set up than an API key. - -## Example Usage - -With the default google project already configured in your environment: - -```py title="vertex_example_env.py" -from pydantic_ai import Agent -from pydantic_ai.models.vertexai import VertexAIModel - -model = VertexAIModel('gemini-1.5-flash') -agent = Agent(model) -result = agent.run_sync('Tell me a joke.') -print(result.data) -#> Did you hear about the toothpaste scandal? They called it Colgate. -``` - -Or using a service account JSON file: - -```py title="vertex_example_service_account.py" -from pydantic_ai import Agent -from pydantic_ai.models.vertexai import VertexAIModel - -model = VertexAIModel( - 'gemini-1.5-flash', - service_account_file='path/to/service-account.json', -) -agent = Agent(model) -result = agent.run_sync('Tell me a joke.') -print(result.data) -#> Did you hear about the toothpaste scandal? They called it Colgate. -``` -""" - from __future__ import annotations as _annotations from collections.abc import Mapping, Sequence diff --git a/tests/test_examples.py b/tests/test_examples.py index 43b388d856..0825ce82e2 100644 --- a/tests/test_examples.py +++ b/tests/test_examples.py @@ -27,7 +27,7 @@ from pydantic_ai.models.function import AgentInfo, DeltaToolCall, DeltaToolCalls, FunctionModel from pydantic_ai.models.test import TestModel -from .conftest import ClientWithHandler +from .conftest import ClientWithHandler, TestEnv try: from pydantic_ai.models.vertexai import VertexAIModel @@ -96,7 +96,11 @@ def find_filter_examples() -> Iterable[CodeExample]: @pytest.mark.parametrize('example', find_filter_examples(), ids=str) def test_docs_examples( - example: CodeExample, eval_example: EvalExample, mocker: MockerFixture, client_with_handler: ClientWithHandler + example: CodeExample, + eval_example: EvalExample, + mocker: MockerFixture, + client_with_handler: ClientWithHandler, + env: TestEnv, ): mocker.patch('pydantic_ai.agent.models.infer_model', side_effect=mock_infer_model) mocker.patch('pydantic_ai._utils.group_by_temporal', side_effect=mock_group_by_temporal) @@ -108,6 +112,10 @@ def test_docs_examples( mocker.patch('httpx.AsyncClient.post', side_effect=async_http_request) mocker.patch('random.randint', return_value=4) + env.set('OPENAI_API_KEY', 'testing') + env.set('GEMINI_API_KEY', 'testing') + env.set('GROQ_API_KEY', 'testing') + prefix_settings = example.prefix_settings() ruff_ignore: list[str] = ['D']