Fix annotation accuracy display in the UI #7652
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
SpecLad
force-pushed
the
fix-accuracy-display
branch
from
b4ee082
to
667186a
Compare
SpecLad
requested review from
bsekachev,
Marishka17 and
nmanovic
as code owners
SpecLad
requested review from
zhiltsov-max
and removed request for
bsekachev,
nmanovic and
Marishka17
SpecLad
force-pushed
the
fix-accuracy-display
branch
from
667186a
to
6e7ea50
Compare
zhiltsov-max
approved these changes
Mar 21, 2024
The current formula the UI uses to calculate accuracy is incorrect. The denominator
is supposed to be the total sample count, but ds_count + gt_count - valid_count
is not the same value. To illustrate:
**** ***. *... ****
**** ***. .*.. ****
**** ***. ..*. ****
.... ***. .... ***.
ds_count gt_count valid_count total_count (expected)
Each value is the sum of the values in the confusion matrix marked by `*`.
The current formula essentially adds values that are off the diagonal twice.
Instead of trying to derive the total count from other values, just transmit
the correct value from the server and use it in the UI.
SpecLad
force-pushed
the
fix-accuracy-display
branch
from
54726ee
to
0d195c4
Compare
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## develop #7652 +/- ##
========================================
Coverage 83.44% 83.44%
========================================
Files 373 373
Lines 39725 39726 +1
Branches 3741 3741
========================================
+ Hits 33148 33150 +2
+ Misses 6577 6576 -1
|
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation and context
The current formula the UI uses to calculate accuracy is incorrect. The denominator is supposed to be the total sample count, but ds_count + gt_count - valid_count is not the same value. To illustrate:
Each value is the sum of the values in the confusion matrix marked by
*. The current formula essentially adds values that are off the diagonal twice.Instead of trying to derive the total count from other values, just transmit the correct value from the server and use it in the UI.
How has this been tested?
Manual testing.
Checklist
developbranch[ ] I have updated the documentation accordingly[ ] I have added tests to cover my changes[ ] I have linked related issues (see GitHub docs)[ ] I have increased versions of npm packages if it is necessary(cvat-canvas,
cvat-core,
cvat-data and
cvat-ui)
License
Feel free to contact the maintainers if that's a concern.