Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to new Protocol response format, use JMESPath expressions #92

Merged
merged 3 commits into from
Jun 5, 2024

Conversation

pamelafox
Copy link
Contributor

Purpose

This PR updates the evaluator to assume that the server uses the new AI Protocol, which has a slightly different JSON response format. Corresponding changes in azure-search-openai-demo are here: Azure-Samples/azure-search-openai-demo#1682

At the same time, this PR adds JMESPath expression support so that developers can easily customize the location of the answer and context in the returned JSON, so that you can use it with older protocol versions or your own custom format.

Does this introduce a breaking change?

[X] Yes - Developers should add the JMESPath parameters to match their response format if using this with a Chat App using older protocol.
[ ] No

Pull Request Type

What kind of change does this Pull Request introduce?

[ ] Bugfix
[X] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Documentation content changes
[ ] Other... Please describe:

How to Test

  • Run tests
  • Run against old server using choices[0].message.content

@pamelafox pamelafox merged commit 3aa7f95 into main Jun 5, 2024
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant