Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add recognition metric (exact match) #110

Merged
merged 3 commits into from
Mar 4, 2021
Merged

feat: Add recognition metric (exact match) #110

merged 3 commits into from
Mar 4, 2021

Conversation

charlesmindee
Copy link
Collaborator

This PR implements ExactMatch metric (word-level accuracy) for recognition task
Any feedback is welcome !

@charlesmindee charlesmindee added the module: utils Related to doctr.utils label Mar 4, 2021
@charlesmindee charlesmindee added this to the 0.1.0 milestone Mar 4, 2021
@charlesmindee charlesmindee self-assigned this Mar 4, 2021
@codecov
Copy link

codecov bot commented Mar 4, 2021

Codecov Report

Merging #110 (9daed8d) into main (169fbac) will decrease coverage by 0.36%.
The diff coverage is 85.18%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #110      +/-   ##
==========================================
- Coverage   97.79%   97.43%   -0.37%     
==========================================
  Files          28       29       +1     
  Lines         908      935      +27     
==========================================
+ Hits          888      911      +23     
- Misses         20       24       +4     
Flag Coverage Δ
unittests 97.43% <85.18%> (-0.37%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
doctr/utils/metrics.py 84.61% <84.61%> (ø)
doctr/utils/__init__.py 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 169fbac...c0c1614. Read the comment docs.

Copy link
Contributor

@fg-mindee fg-mindee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR! I added a few comments. Besides, especially if we don't ignore accents or case, we could actually do this with TF and compare the output before decoding?

doctr/utils/metrics.py Outdated Show resolved Hide resolved
doctr/utils/metrics.py Outdated Show resolved Hide resolved
doctr/utils/metrics.py Outdated Show resolved Hide resolved
test/test_utils_metrics.py Show resolved Hide resolved
test/test_utils_metrics.py Outdated Show resolved Hide resolved
@fg-mindee fg-mindee changed the title feat: add recognition metric (exact macth) feat: Add recognition metric (exact match) Mar 4, 2021
@charlesmindee
Copy link
Collaborator Author

charlesmindee commented Mar 4, 2021

Thanks for the PR! I added a few comments. Besides, especially if we don't ignore accents or case, we could actually do this with TF and compare the output before decoding?

I think so, but it would require tensor manipulations to compare row-wise both tensors. What I was doing previously was translating with the dictionary to a string tensor and I concatenated the columns to have a vector of words, but I think we can compare the number sequences at a row-level without translating the outputs to save time.

Copy link
Contributor

@fg-mindee fg-mindee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might want later to convert this to a TF-only computation, but it will do for now! Thanks for the edits 🙏

@fg-mindee fg-mindee merged commit f74e46b into main Mar 4, 2021
@fg-mindee fg-mindee deleted the reco_metric branch March 4, 2021 17:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
module: utils Related to doctr.utils
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants