Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor ava hook #567

Merged
merged 34 commits into from
Jan 27, 2021
Merged

Conversation

kennymckormick
Copy link
Member

@kennymckormick kennymckormick commented Jan 27, 2021

fix #549

Migration note :

You need to use --eval mAP during AVA models testing now.

AVADataset evaluation metrics modification:
Supported metrics before: proposal, bbox
Supported metrics after: mAP (the same as bbox before), the old proposal is removed.

HaodongDuan and others added 26 commits October 16, 2020 17:35
@kennymckormick kennymckormick linked an issue Jan 27, 2021 that may be closed by this pull request
@innerlee
Copy link
Contributor

Still WIP? Are there any configs need to be changed? Also give good warnings to help user migrate

@innerlee
Copy link
Contributor

It would be great to add some migration notes in the top post, to give detailed info on what has been changed, and how to change configs/cmds to the new way. Example: open-mmlab/mmpose#382

@codecov
Copy link

codecov bot commented Jan 27, 2021

Codecov Report

Merging #567 (3a4f68d) into master (b747208) will decrease coverage by 0.63%.
The diff coverage is 66.66%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #567      +/-   ##
==========================================
- Coverage   84.76%   84.12%   -0.64%     
==========================================
  Files         121      121              
  Lines        8579     8580       +1     
  Branches     1416     1416              
==========================================
- Hits         7272     7218      -54     
- Misses        953     1016      +63     
+ Partials      354      346       -8     
Flag Coverage Δ
unittests 84.11% <66.66%> (-0.64%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
mmaction/core/evaluation/ava_utils.py 68.79% <50.00%> (-9.93%) ⬇️
mmaction/datasets/ava_dataset.py 90.00% <100.00%> (+0.05%) ⬆️
mmaction/core/evaluation/recall.py 11.76% <0.00%> (-60.30%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b747208...3a4f68d. Read the comment docs.

@innerlee
Copy link
Contributor

Do you need to update modelzoo, if there are related metrics affected?

@innerlee innerlee requested a review from SuX97 January 27, 2021 08:44
Copy link
Collaborator

@SuX97 SuX97 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just changing names looks good to me. But how do we evaluate AR@AN and Recall?

Can metrics be ['mAP', 'AR@AN', 'Recall'] since those metrics have been calculated anyway?

@innerlee innerlee merged commit 944c5b1 into open-mmlab:master Jan 27, 2021
@kennymckormick kennymckormick deleted the RefactorAVAHook branch February 1, 2021 05:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Refactor evaluation of ava
3 participants