Feature/implement the amega#1188
Merged
chakravarthik27 merged 13 commits intorelease/2.7.0from Mar 19, 2025
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull Request Overview
This pull request introduces the new AMEGA class for clinical tests, which implements methods for transforming, running, generating, and evaluating responses for sample lists.
- Introduces the AMEGA class with an alias name and a list of supported tasks.
- Implements new static methods: transform, run (async), generate_responses, and evaluate_responses.
…th sample generation and tqdm progress tracking
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR introduces a new clinical test class, AMEGA, which implements methods for transforming sample lists, running tests asynchronously, generating responses, and evaluating them.
- Introduces the AMEGA class with relevant clinical test methods.
- Renames existing BaseClincial to BaseClinical to improve clarity.
- Updates import references to use tqdm.auto for consistent behavior.
Reviewed Changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| langtest/transform/clinical.py | Added the AMEGA class with methods: transform, run, generate_responses, and evaluate_responses; fixed naming inconsistencies. |
| langtest/utils/custom_types/sample.py | Added a new AMEGASample class. |
| langtest/transform/base.py | Updated tqdm import to use tqdm.auto. |
| langtest/langtest.py | Minor formatting change with an added newline. |
Comments suppressed due to low confidence (1)
langtest/transform/clinical.py:860
- The default value for 'progress_bar' is set to False, which may not have an 'update' method; ensure that 'progress_bar' is a valid progress bar object before calling update.
progress_bar.update(1)
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Prikshit7766
approved these changes
Mar 19, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces a new class
AMEGAfor clinical tests in thelangtest/transform/clinical.pyfile. The class includes methods for transforming and running sample lists, generating responses, and evaluating those responses. Below are the key changes:Introduction of
AMEGAclass:langtest/transform/clinical.py: Added theAMEGAclass with an alias name and supported tasks.New methods in
AMEGAclass:transform: A static method to transform the sample list.run: A static method to asynchronously run the sample list through the model and generate responses.generate_responses: A static method to generate responses usingDataRetrieverandResponseGenerator.evaluate_responses: A static method to evaluate responses usingResponseEvaluatorand aggregate the results.