Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🎉 Add Azure Content Moderator service for image analysis #143

Merged
merged 15 commits into from
Aug 18, 2023

Conversation

teresaqhoang
Copy link
Contributor

Motivation and Context

This pull request adds content moderation and image handling features to the project. Specifically, it adds a new ContentModeratorController with endpoints for detecting sensitive image content and getting the status of content moderation. It also adds image text validation to the DocumentImportController, throwing an exception if an image does not contain text.

Additionally, this pull request adds an AzureContentModerator service and a ContentModeratorOptions class for configuring content moderation. The Azure Content Moderator service allows for the analysis of images to detect harmful content, such as hate speech, sexual content, self-harm, and violence.

The pull request also includes updates to the ChatInput and DocumentsTab components in the webapp to support the new features, including the use of the new useContentModerator hook for content moderation and the useFile hook for image upload handling.

Description

webapp

  • Add image moderation and import handling to useFile hook: Add loadImage and handleImport functions to useFile hook.
    • loadImage function loads an image from a URL and returns a promise.
    • handleImport function handles importing a file and returns a promise.
  • Add ContentModerationService for image moderation.
    • analyzeImageAsync method analyzes an image for content moderation.
    • getContentModerationStatusAsync method checks the status of the content moderation service.

webapi

  • The ChatSkill now includes an instance of AzureContentModerator which can be used to moderate user input when an image is uploaded.
  • The appsettings.json file has been updated to include configuration options for the service.
  • The ChatInput component has been updated to include a new hook useContentModerator which handles document import, including using the Azure Content Moderator API to analyze image content for offensive or unwanted elements. - The handleImport function in ChatInput has been removed and replaced with a shared function in useFile that handles file imports. Also refactored document import in DocumentsTab component to use useFile hook.

image

image
AzureContentSafety

Contribution Checklist

@github-actions github-actions bot added webapp Pull requests that update Typescript code webapi Pull requests that update .net code PR: ready for review labels Aug 9, 2023
@teresaqhoang teresaqhoang self-assigned this Aug 9, 2023
@teresaqhoang teresaqhoang marked this pull request as draft August 14, 2023 20:00
@teresaqhoang teresaqhoang added the PR: paused For PRs that have been converted to draft, are facing blockers or have no active development planned label Aug 14, 2023
@teresaqhoang teresaqhoang removed the PR: paused For PRs that have been converted to draft, are facing blockers or have no active development planned label Aug 17, 2023
@teresaqhoang teresaqhoang marked this pull request as ready for review August 17, 2023 14:46
Copy link
Contributor

@crickman crickman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DocumentImportController is within a week or two of being deleted and all of this work will need to be ported to the semantic-memory service. I wonder if just landing this in semantic-memory as a first step doesn't make more sense?

webapi/Controllers/ContentModeratorController.cs Outdated Show resolved Hide resolved
@teresaqhoang teresaqhoang marked this pull request as draft August 17, 2023 15:23
@teresaqhoang teresaqhoang added the PR: paused For PRs that have been converted to draft, are facing blockers or have no active development planned label Aug 17, 2023
@teresaqhoang
Copy link
Contributor Author

DocumentImportController is within a week or two of being deleted and all of this work will need to be ported to the semantic-memory service. I wonder if just landing this in semantic-memory as a first step doesn't make more sense?

As part of the semantic-memory work, nearly all of the image processing logic is being moved to semantic-memory, in the relatively near-term. I wonder if it makes sense to coordinate this feature within that scope?

Definitely makes sense to coordinate this feature within the semantic-memory work. Pausing this PR until then.

crickman
crickman previously approved these changes Aug 17, 2023
@teresaqhoang teresaqhoang marked this pull request as ready for review August 18, 2023 13:54
@teresaqhoang teresaqhoang removed the PR: paused For PRs that have been converted to draft, are facing blockers or have no active development planned label Aug 18, 2023
alliscode
alliscode previously approved these changes Aug 18, 2023
@teresaqhoang teresaqhoang added this pull request to the merge queue Aug 18, 2023
Merged via the queue into microsoft:main with commit 8bc548f Aug 18, 2023
7 checks passed
@teresaqhoang teresaqhoang deleted the azure-content-safety branch August 18, 2023 17:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
webapi Pull requests that update .net code webapp Pull requests that update Typescript code
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

None yet

5 participants