-
Notifications
You must be signed in to change notification settings - Fork 1.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add AnswerExactMatchEvaluator
#7050
Conversation
Pull Request Test Coverage Report for Build 8006956131Details
💛 - Coveralls |
306e225
to
53961c9
Compare
AnswerExactMatchEvaluator
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some minor changes, but LGTM otherwise! Let's also get @julian-risch to sign off.
@@ -0,0 +1,3 @@ | |||
from .answer_exact_match import AnswerExactMatchEvaluator |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Now that the metrics are (in)directly associated with specific upstream components, I don't think we need an additional statistical
submodule after all.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! 👍 The docstring describes the metric really well and the implementation of the metric is easy to read and correct. 👍
This reverts commit b4011af.
This reverts commit b4011af.
Related Issues
Proposed Changes:
Add
AnswerExactMatchEvaluator
. This Component calculates the Exact Match metrics given a list of questions, a list of expected answers for each question and the list of predicted answers for each question.How did you test it?
I added unit tests.
Notes for the reviewer
N/A
Checklist
fix:
,feat:
,build:
,chore:
,ci:
,docs:
,style:
,refactor:
,perf:
,test:
.