Skip to content

Introduce Azure AI Evaluation SDK #18

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Dec 13, 2024

Conversation

YusakuNo1
Copy link
Contributor

@YusakuNo1 YusakuNo1 commented Dec 11, 2024

Introduce Azure AI Evaluation SDK. There are 3 steps in the sample code:

  • Generate evaluation data with GitHub model catalog and save it to a file.
  • Run performance and quality evaluators with GitHub model catalog.
  • For those whom want additional evaluation capability, uncomment the following code with Azure credentials, then we can run the risk and safety evaluators from Azure AI.

@YusakuNo1 YusakuNo1 changed the title Generate evaluation data and run evaluators Introduce Azure AI Evaluation SDK Dec 11, 2024
sgoedecke
sgoedecke previously approved these changes Dec 12, 2024
@sgoedecke sgoedecke merged commit 1480f19 into github:main Dec 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants