Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Python: Partial content filtering response in Python SDK #4197

Closed
orsharab opened this issue Dec 12, 2023 · 8 comments
Closed

Python: Partial content filtering response in Python SDK #4197

orsharab opened this issue Dec 12, 2023 · 8 comments
Assignees
@moonbox3
Labels
ai connector Anything related to AI connectors bug Something isn't working python Pull requests for the Python Semantic Kernel

Comments

@orsharab
Copy link

Describe the bug
In Python SDK, when content filtering is triggered, the returned response is partial, missing "inner_error" property that indicates the reason of the content filtering:

'{"error":{"message":"The response was filtered due to the prompt triggering Azure OpenAI\'s content management policy. Please modify your prompt and retry. To learn more about our content filtering policies please read our documentation: [https://go.microsoft.com/fwlink/?linkid=2198766","type":null,"param":"prompt","code":"content_filter","status":400}](https://go.microsoft.com/fwlink/?linkid=2198766%22,%22type%22:null,%22param%22:%22prompt%22,%22code%22:%22content_filter%22,%22status%22:400%7d)}'

See OpenAI API reference for "inner_error" property

Platform
Python SDK version - 0.3.15.dev0
@shawncal shawncal added python Pull requests for the Python Semantic Kernel triage labels Dec 12, 2023
@github-actions github-actions bot changed the title Partial content filtering response in Python SDK Python: Partial content filtering response in Python SDK Dec 12, 2023
@matthewbolanos matthewbolanos added bug Something isn't working and removed triage labels Dec 12, 2023
@moonbox3
Copy link
Contributor

Thanks for filing, @orsharab. How can I repro it on my end?

@moonbox3
Copy link
Contributor

@eavanvalkenburg we should chat about plans for proper error/exception handling.

@moonbox3
Copy link
Contributor

@juliomenendez if you have any bandwidth, could you please help take a look?

@juliomenendez juliomenendez mentioned this issue Dec 26, 2023
Merged
4 tasks
@juliomenendez
Copy link
Contributor

Hi @orsharab, thank you for reporting this issue. I have a PR with a new exception to be thrown in this type of cases #4428. Feel free to take a look and share any thoughts about it.

@orsharab
Copy link
Author

orsharab commented Dec 28, 2023 via email

@juliomenendez
Copy link
Contributor

@orsharab yes, the PR doesn't restrict what type of content filter is accepted. Jailbreak is handled in the content_filter_result like the rest of the keys.

@matthewbolanos matthewbolanos added the ai connector Anything related to AI connectors label Jan 2, 2024
github-merge-queue bot pushed a commit that referenced this issue Jan 4, 2024
@juliomenendez @moonbox3
Python: Supports better exceptions when Azure OpenAI content filterin…
7f68a88 
…g is triggered (#4428)

### Motivation and Context

When an Azure OpenAI content filter error is triggered the current
`AIException` hides the details of that specific error. This makes it
hard for the consumer to see a more detailed description of the error in
their prompt or input.
Solves #4197

### Description

Detects when a content filter error is raised, then parses the
meaningful information about it and raises a new
`ContentFilterAIException` with that information ready for the consumer
to use.

### Contribution Checklist

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄

---------

Co-authored-by: Evan Mattson <35585003+moonbox3@users.noreply.github.com>
@juliomenendez
Copy link
Contributor

@orsharab the changes have been merged and should go out in the next release.

@orsharab
Copy link
Author

orsharab commented Jan 4, 2024 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@moonbox3 moonbox3

Labels
ai connector Anything related to AI connectors bug Something isn't working python Pull requests for the Python Semantic Kernel
Projects
Semantic Kernel
Archived in project
Milestone
No milestone
Development

No branches or pull requests
5 participants
@juliomenendez @matthewbolanos @moonbox3 @shawncal @orsharab