Skip to content

Add the sample script for the Quality Grader Evaluator#46892

Merged
salma-elshafey merged 12 commits into
mainfrom
selshafey/bugbash_qg
May 14, 2026
Merged

Add the sample script for the Quality Grader Evaluator#46892
salma-elshafey merged 12 commits into
mainfrom
selshafey/bugbash_qg

Conversation

@salma-elshafey
Copy link
Copy Markdown
Contributor

Description

Please add an informative description that covers that changes made by the pull request and link all relevant issues.

If an SDK is being regenerated based on a new API spec, a link to the pull request containing these API spec changes should be included above.

All SDK Contribution checklist:

  • The pull request does not introduce [breaking changes]
  • CHANGELOG is updated for new features, bug fixes or other significant changes.
  • I have read the contribution guidelines.

General Guidelines and Best Practices

  • Title of the pull request is clear and informative.
  • There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

  • Pull request includes test coverage for the included changes.

Copilot AI review requested due to automatic review settings May 14, 2026 16:48
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new Azure AI Projects sample that demonstrates running the Quality Grader agentic evaluator via openai.evals.* using an inline JSONL data source.

Changes:

  • Introduces sample_quality_grader.py showing how to create an evaluation and run it with multiple example items (including conversation-shaped inputs).
  • Configures a TestingCriterionAzureAIEvaluator for builtin.quality_grader and polls run status until completion.

@salma-elshafey salma-elshafey enabled auto-merge (squash) May 14, 2026 20:45
@salma-elshafey salma-elshafey merged commit 0d715b8 into main May 14, 2026
19 checks passed
@salma-elshafey salma-elshafey deleted the selshafey/bugbash_qg branch May 14, 2026 20:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants