Adding NLP Metrics SQuADv2 and WER #368
Conversation
gagan3012
requested review from
ananyahjha93,
Borda,
justusschock,
SkafteNicki and
tchaton
as code owners
|
Hello @gagan3012! Thanks for updating this PR. There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻 Comment last updated at 2021-07-16 07:59:42 UTC |
for more information, see https://pre-commit.ci
There was a problem hiding this comment.
Mix of comments. Just wanted to make sure: Are these metrics meant to be accumulated over multiple batches or are they only intended to be evaluated for single batch/sample? Because if it is the latter then it is maybe better to implement them as functionals.
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
|
Hello, |
|
Hello, |
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
|
@gagan3012 seems it is your first PR so GH requires to allow running CI by a maintainer, so pls when you see it is hanging, pls ping me in slack so we can speed it up :] |
| @@ -0,0 +1 @@ | |||
| jiwer | |||
| @@ -0,0 +1,316 @@ | |||
| import argparse | |||
There was a problem hiding this comment.
can you confirm that this is your own code? 🐰
There was a problem hiding this comment.
I am not very convinced about having any entry point to TM, that shall be done consistently to all metrics then...
also, WEB testing is very sparse and missing any tests for squadv2
@gagan3012 we really appreciate your effort, I would just recommend splitting adding the two metrics to two PR where each will be adding only one of them which would simplify review and also your work by focussing only on one at the time 🐰
cc: @SkafteNicki
|
Hello, |
Before submitting
What does this PR do?
Fixes #52
Fixes #366 (issue).
PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
Did you have fun?
Make sure you had fun coding 🙃