-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Fix bug related to Azure OpenAI model streaming response
#932
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix bug related to Azure OpenAI model streaming response
#932
Conversation
sydney-runkle
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wonderful, thanks :)
|
Could you please take a look at the linting issue? Thanks! |
|
Seems the linting issue is happening because the class Choice(BaseModel):
delta: ChoiceDelta
"""A chat completion delta generated by streamed model responses."""
...I couldn't reproduce the problem for now CI pyright output: /home/runner/work/pydantic-ai/pydantic-ai/pydantic_ai_slim/pydantic_ai/models/openai.py:356:16 - error: Condition will always evaluate to True since the types "ChoiceDelta" and "None" have no overlap (reportUnnecessaryComparison) |
|
I'm trying to get some Azure AI Foundry keys so I can add a test for this. |
|
@gbe3hunna can you give me the snippet that reproduces this issue? I can't reproduce it. |
Nope. I just took the issue but already mentioned I cannot reproduce it |
|
If we can't reproduce, then I think we can close it. Thanks for the help! |
|
@Kludex Ref comment in #797 - how can I help to reproduce. As mentioned in issue - this occurs only with async content filter enabled. https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/content-filter?tabs=warning%2Cuser-prompt%2Cpython-new#asynchronous-filter |
Solves: #797
This PR fixes an issue happening with Azure OpenAI streaming Response causing the raise of an AttributeError
As stated on the issue, the error occurs in pydantic-ai's OpenAI model implementation where it assumes every delta in the streaming response contains a content field. However, with Azure OpenAI's API, some deltas (like role initialization) may not include content, resulting in choice.delta.content being None.