Update ai assistants evals version 2 - #673
Merged
Merged
Conversation
Revise AI Evaluations documentation with details of version 2 release
Updated the document to reflect changes in the AI Assistants creation and modification process, including new versioning details and improved structure.
Updated images in the documentation for AI Assistants integration steps, ensuring correct display and context for users.
Contributor
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Team Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
🚀 Deployed on https://deploy-preview-673--glific-docs.netlify.app |
mahajantejas
requested review from
tanuprasad530
and removed request for
SangeetaMishr
September 2, 2026 10:02
tanuprasad530
requested changes
Sep 2, 2026
Clarified instructions and added images for better understanding of the AI Assistant creation and editing process.
Refactor evaluation metrics section for clarity and consistency. Update instructions for setting up Golden Q&A sets and running evaluations.
| ### Step 3: Select an AI Assistant | ||
| Click the "Search or select an AI assistant" dropdown under AI Assistant and choose the specific assistant (and its version) you want to evaluate. | ||
| The overall score is a weighted average of three checks, each scored 0–5: | ||
| 1. Adherence to ground truth (weight 50%, how well answers match the golden answers), |
Contributor
There was a problem hiding this comment.
Here - It would be helpful to add what drives the score—whether the metric is assessing if the core answer is correct, or how closely the response aligns with the ground truth in terms of content and scope.
Contributor
Author
There was a problem hiding this comment.
updated the definition
tanuprasad530
requested changes
Sep 3, 2026
Corrected capitalization and phrasing for clarity in the AI Assistant documentation.
Updated the wording for clarity and consistency in the evaluation metrics and scoring description.
tanuprasad530
approved these changes
Sep 4, 2026
tanuprasad530
approved these changes
Sep 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.