Skip to content

Conversation

@gbe3hunna
Copy link

Solves: #797

This PR fixes an issue happening with Azure OpenAI streaming Response causing the raise of an AttributeError

As stated on the issue, the error occurs in pydantic-ai's OpenAI model implementation where it assumes every delta in the streaming response contains a content field. However, with Azure OpenAI's API, some deltas (like role initialization) may not include content, resulting in choice.delta.content being None.

@github-actions github-actions bot temporarily deployed to deploy-preview February 15, 2025 16:12 Inactive
Copy link
Contributor

@sydney-runkle sydney-runkle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wonderful, thanks :)

@sydney-runkle
Copy link
Contributor

Could you please take a look at the linting issue? Thanks!

@gbe3hunna
Copy link
Author

gbe3hunna commented Feb 24, 2025

Seems the linting issue is happening because the Choice BaseModel defines the field delta as ChoiceDelta, and not as an Optional[ChoiceDelta] or ChoiceDelta | None, which is the case exposed at #797 (comment)

class Choice(BaseModel):
    delta: ChoiceDelta
    """A chat completion delta generated by streamed model responses."""
    ...

I couldn't reproduce the problem for now

CI pyright output: /home/runner/work/pydantic-ai/pydantic-ai/pydantic_ai_slim/pydantic_ai/models/openai.py:356:16 - error: Condition will always evaluate to True since the types "ChoiceDelta" and "None" have no overlap (reportUnnecessaryComparison)

@Kludex Kludex self-assigned this Feb 25, 2025
@Kludex
Copy link
Member

Kludex commented Feb 25, 2025

I'm trying to get some Azure AI Foundry keys so I can add a test for this.

@Kludex
Copy link
Member

Kludex commented Mar 13, 2025

@gbe3hunna can you give me the snippet that reproduces this issue? I can't reproduce it.

@gbe3hunna
Copy link
Author

@gbe3hunna can you give me the snippet that reproduces this issue? I can't reproduce it.

Nope. I just took the issue but already mentioned I cannot reproduce it

@Kludex
Copy link
Member

Kludex commented Mar 14, 2025

If we can't reproduce, then I think we can close it. Thanks for the help!

@Kludex Kludex closed this Mar 14, 2025
@martgra
Copy link

martgra commented Mar 26, 2025

@Kludex Ref comment in #797 - how can I help to reproduce. As mentioned in issue - this occurs only with async content filter enabled. https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/content-filter?tabs=warning%2Cuser-prompt%2Cpython-new#asynchronous-filter

@gbe3hunna

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants