-
Notifications
You must be signed in to change notification settings - Fork 1.3k
Add Zhipu AI support with models, provider, and documentation #3086
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Please move a simplified version of this doc to a section in |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,202 @@ | ||
# Zhipu AI | ||
|
||
## Install | ||
|
||
To use Zhipu AI models, you need to either install `pydantic-ai`, or install `pydantic-ai-slim` with the `openai` optional group (since Zhipu AI provides an OpenAI-compatible API): | ||
|
||
```bash | ||
pip/uv-add "pydantic-ai-slim[openai]" | ||
``` | ||
|
||
## Configuration | ||
|
||
To use [Zhipu AI](https://bigmodel.cn/) (智谱AI) through their API, you need to: | ||
|
||
1. Visit [bigmodel.cn](https://bigmodel.cn) and create an account | ||
2. Go to [API Keys management](https://bigmodel.cn/usercenter/proj-mgmt/apikeys) | ||
3. Create a new API key | ||
|
||
## Environment variable | ||
|
||
Once you have the API key, you can set it as an environment variable: | ||
|
||
```bash | ||
export ZHIPU_API_KEY='your-api-key' | ||
``` | ||
|
||
You can then use Zhipu AI models by name: | ||
|
||
```python | ||
from pydantic_ai import Agent | ||
|
||
agent = Agent('zhipu:glm-4.5') | ||
... | ||
``` | ||
|
||
Or initialise the model directly: | ||
|
||
```python | ||
from pydantic_ai import Agent | ||
from pydantic_ai.models.openai import OpenAIChatModel | ||
|
||
model = OpenAIChatModel('glm-4.5', provider='zhipu') | ||
agent = Agent(model) | ||
... | ||
``` | ||
|
||
## `provider` argument | ||
|
||
You can provide a custom `ZhipuProvider` via the `provider` argument: | ||
|
||
```python | ||
from pydantic_ai import Agent | ||
from pydantic_ai.models.openai import OpenAIChatModel | ||
from pydantic_ai.providers.zhipu import ZhipuProvider | ||
|
||
model = OpenAIChatModel( | ||
'glm-4.5', provider=ZhipuProvider(api_key='your-api-key') | ||
) | ||
agent = Agent(model) | ||
... | ||
``` | ||
|
||
You can also customize the `ZhipuProvider` with a custom `httpx.AsyncClient`: | ||
|
||
```python | ||
from httpx import AsyncClient | ||
|
||
from pydantic_ai import Agent | ||
from pydantic_ai.models.openai import OpenAIChatModel | ||
from pydantic_ai.providers.zhipu import ZhipuProvider | ||
|
||
custom_http_client = AsyncClient(timeout=30) | ||
model = OpenAIChatModel( | ||
'glm-4.5', | ||
provider=ZhipuProvider(api_key='your-api-key', http_client=custom_http_client), | ||
) | ||
agent = Agent(model) | ||
... | ||
``` | ||
|
||
## Available Models | ||
|
||
Zhipu AI offers several models through their OpenAI-compatible API: | ||
|
||
### GLM-4 Series | ||
|
||
- **`glm-4.6`**: Latest flagship model with 205K context window | ||
- **`glm-4.5`**: High-performance model with 131K context window | ||
- **`glm-4.5-air`**: Balanced performance and cost with 131K context | ||
- **`glm-4.5-flash`**: Fast response model with 131K context | ||
|
||
### Vision Models | ||
|
||
- **`glm-4v-plus`**: Advanced vision understanding model | ||
- **`glm-4v`**: Vision model for image analysis | ||
- **`glm-4.5v`**: Vision-enabled variant with 64K context | ||
|
||
### Specialized Models | ||
|
||
- **`codegeex-4`**: Code generation and understanding model | ||
|
||
## Features | ||
|
||
### Function Calling | ||
|
||
Zhipu AI models support function calling (tool use): | ||
|
||
```python | ||
from pydantic_ai import Agent, RunContext | ||
|
||
agent = Agent( | ||
'zhipu:glm-4.5', | ||
system_prompt='You are a helpful assistant with access to tools.', | ||
) | ||
|
||
@agent.tool | ||
async def get_weather(ctx: RunContext[None], location: str) -> str: | ||
"""Get the weather for a location.""" | ||
return f"The weather in {location} is sunny, 25°C" | ||
|
||
result = await agent.run('What is the weather in Beijing?') | ||
print(result.output) | ||
``` | ||
|
||
### Streaming | ||
|
||
Zhipu AI supports streaming responses: | ||
|
||
```python | ||
from pydantic_ai import Agent | ||
|
||
agent = Agent('zhipu:glm-4.5') | ||
|
||
async with agent.run_stream('Tell me a story') as response: | ||
async for message in response.stream_text(): | ||
print(message, end='', flush=True) | ||
``` | ||
|
||
### Vision Understanding | ||
|
||
Use vision models to analyze images: | ||
|
||
```python | ||
from pydantic_ai import Agent | ||
|
||
agent = Agent('zhipu:glm-4v-plus') | ||
|
||
result = await agent.run( | ||
'What is in this image?', | ||
message_history=[ | ||
{ | ||
'role': 'user', | ||
'content': [ | ||
{'type': 'text', 'text': 'Describe this image:'}, | ||
{'type': 'image_url', 'image_url': {'url': 'https://example.com/image.jpg'}} | ||
] | ||
} | ||
] | ||
) | ||
print(result.output) | ||
``` | ||
|
||
## Important Notes | ||
|
||
### Temperature Range | ||
|
||
Unlike OpenAI, Zhipu AI requires temperature to be in the range `(0, 1)` (exclusive). Setting `temperature=0` is not supported and will cause an error. | ||
|
||
```python | ||
# This will work | ||
agent = Agent('zhipu:glm-4.5', model_settings={'temperature': 0.1}) | ||
|
||
# This will NOT work with Zhipu AIs | ||
# agent = Agent('zhipu:glm-4.5', model_settings={'temperature': 0}) | ||
``` | ||
|
||
### Strict Mode | ||
|
||
Zhipu AI does not support OpenAI's strict mode for tool definitions. The framework automatically handles this by setting `openai_supports_strict_tool_definition=False` in the model profile. | ||
|
||
## Advanced Features | ||
|
||
### Thinking Mode | ||
|
||
GLM-4.5 and GLM-4.5-Air support a "thinking" mode for complex reasoning tasks. This can be enabled using the `extra_body` parameter: | ||
|
||
```python | ||
from pydantic_ai.models.openai import OpenAIChatModel | ||
from pydantic_ai.providers.zhipu import ZhipuProvider | ||
|
||
model = OpenAIChatModel('glm-4.5', provider=ZhipuProvider(api_key='your-api-key')) | ||
|
||
# Note: Thinking mode requires using the OpenAI client directly | ||
# or passing extra_body through model_settings | ||
``` | ||
|
||
## API Reference | ||
|
||
For more details, see: | ||
|
||
- [ZhipuProvider API Reference][pydantic_ai.providers.zhipu.ZhipuProvider] | ||
- [Zhipu AI Official Documentation](https://docs.bigmodel.cn/) |
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Example file not necessary, this can just be an example in the docs |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,33 @@ | ||
"""Example of using Zhipu AI with Pydantic AI. | ||
|
||
This example demonstrates how to use Zhipu AI models with the Pydantic AI framework. | ||
|
||
To run this example, you need to: | ||
1. Install pydantic-ai with openai support: pip install "pydantic-ai-slim[openai]" | ||
2. Set your Zhipu API key: export ZHIPU_API_KEY='your-api-key' | ||
3. Run the script: python examples/zhipu_example.py | ||
""" | ||
|
||
from __future__ import annotations as _annotations | ||
|
||
import asyncio | ||
|
||
from pydantic_ai import Agent | ||
|
||
|
||
async def main(): | ||
"""Run a simple example with Zhipu AI.""" | ||
# Create an agent using Zhipu AI's GLM-4.5 model | ||
model_spec = 'zhipu:glm-4.5' | ||
agent = Agent(model_spec, system_prompt='You are a helpful assistant.') | ||
|
||
# Run a simple query | ||
result = await agent.run('What is the capital of China?') | ||
print(f'Response: {result.output}') | ||
# Access the configured model name directly from the agent's model to avoid relying on | ||
# message internals (keeps static typing happy if message schema varies by provider). | ||
print(f'Model used: {model_spec}') | ||
|
||
|
||
if __name__ == '__main__': | ||
asyncio.run(main()) |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -284,6 +284,7 @@ def __init__( | |
'together', | ||
'vercel', | ||
'litellm', | ||
'zhipu', | ||
] | ||
| Provider[AsyncOpenAI] = 'openai', | ||
profile: ModelProfileSpec | None = None, | ||
|
@@ -312,6 +313,7 @@ def __init__( | |
'together', | ||
'vercel', | ||
'litellm', | ||
'zhipu', | ||
] | ||
| Provider[AsyncOpenAI] = 'openai', | ||
profile: ModelProfileSpec | None = None, | ||
|
@@ -339,6 +341,7 @@ def __init__( | |
'together', | ||
'vercel', | ||
'litellm', | ||
'zhipu', | ||
] | ||
| Provider[AsyncOpenAI] = 'openai', | ||
profile: ModelProfileSpec | None = None, | ||
|
@@ -523,6 +526,16 @@ def _process_response(self, response: chat.ChatCompletion | str) -> ModelRespons | |
if not isinstance(response, chat.ChatCompletion): | ||
raise UnexpectedModelBehavior('Invalid response from OpenAI chat completions endpoint, expected JSON data') | ||
|
||
# Some OpenAI-compatible providers (currently only Zhipu/Z.ai, see issue #2723) omit the `object` field even | ||
# though the OpenAI schema includes it. We only patch it for that provider to avoid changing validation | ||
# error counts in tests that purposefully feed invalid OpenAI responses (which expect 4 errors, including | ||
# a missing `object`). | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't know exactly what this is referring to, but I'd rather unconditionally set |
||
if self._provider.name == 'zhipu' and not getattr(response, 'object', None): # pragma: no branch | ||
try: # defensive, in case attribute is read-only in future SDK versions | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not necessary, it's a Pydantic |
||
response.object = 'chat.completion' | ||
except Exception: # pragma: no cover | ||
pass | ||
|
||
if response.created: | ||
timestamp = number_to_datetime(response.created) | ||
else: | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
from __future__ import annotations as _annotations | ||
|
||
from .openai import OpenAIModelProfile | ||
|
||
|
||
def zhipu_model_profile(model_name: str) -> OpenAIModelProfile: | ||
"""Get the model profile for a Zhipu AI model. | ||
Zhipu AI provides OpenAI-compatible API, so we use OpenAIModelProfile. | ||
Args: | ||
model_name: The Zhipu model name (e.g., 'glm-4.6', 'glm-4.5', 'glm-4.5-air', 'glm-4.5v'). | ||
Returns: | ||
Model profile with Zhipu-specific configurations. | ||
""" | ||
# Vision models — docs show vision variants with a trailing `v` like `glm-4.5v` | ||
# Ref: https://docs.bigmodel.cn/cn/guide/develop/openai/introduction | ||
is_vision_model = model_name.startswith(('glm-4.5v', 'glm-4v')) or ( | ||
'v' in model_name and ('glm-4.5v' in model_name or 'glm-4v' in model_name) | ||
) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This check is very redundant: |
||
|
||
# Zhipu AI models support JSON schema and object output | ||
# All GLM-4 series models support function calling | ||
supports_tools = model_name.startswith(('glm-4', 'codegeex-4')) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The variable name is misleading then right? |
||
|
||
# Zhipu AI doesn't support temperature=0 (must be in range (0, 1)) | ||
# This is a known difference from OpenAI | ||
openai_unsupported_model_settings = () | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is already the default so no need to include it like this right? |
||
|
||
return OpenAIModelProfile( | ||
supports_json_schema_output=supports_tools, | ||
supports_json_object_output=supports_tools, | ||
supports_image_output=is_vision_model, | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do the vision models actually support image output, or just input? |
||
openai_supports_strict_tool_definition=False, # Zhipu doesn't support strict mode | ||
openai_supports_tool_choice_required=True, | ||
openai_unsupported_model_settings=openai_unsupported_model_settings, | ||
openai_system_prompt_role=None, # Use default 'system' role | ||
openai_chat_supports_web_search=False, | ||
openai_supports_encrypted_reasoning_content=False, | ||
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please move this doc to a section in
models/openai.md
, and move the link to the "OpenAI-compatible Providers" list below