Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Add] Support HOTA evaluation for MOTChallenge. #417

Merged
merged 9 commits into from
Feb 15, 2022
Merged

[Add] Support HOTA evaluation for MOTChallenge. #417

merged 9 commits into from
Feb 15, 2022

Conversation

Seerkfang
Copy link
Collaborator

No description provided.

# Use 'track' as the default tracker
TRACKERS_TO_EVAL=['track'],
# Option values: ['pedestrian']
CLASSES_TO_EVAL=['pedestrian'],
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have self.classes, is CLASSES_TO_EVAL related to this item?

# Option values: ['pedestrian']
CLASSES_TO_EVAL=['pedestrian'],
# Option Values: 'MOT17', 'MOT16', 'MOT20', 'MOT15'
BENCHMARK=self.img_prefix.rsplit('/', 2)[-2],
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is risky, write a function to convert lower-case names, and raise errors if there are no specific benchmarks in their names.

'to manually install trackeval')

seqmap = osp.join(self.img_prefix, 'videoseq.txt')
if not osp.exists(seqmap):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add document instructions to indicate dataset preparations.

@codecov
Copy link

codecov bot commented Feb 10, 2022

Codecov Report

Merging #417 (ac9f6b6) into master (ccaaab2) will increase coverage by 0.09%.
The diff coverage is 75.00%.

❗ Current head ac9f6b6 differs from pull request most recent head 7b14bfa. Consider uploading reports for the commit 7b14bfa to get more accurate results

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #417      +/-   ##
==========================================
+ Coverage   70.89%   70.98%   +0.09%     
==========================================
  Files         117      117              
  Lines        6249     6296      +47     
  Branches     1198     1208      +10     
==========================================
+ Hits         4430     4469      +39     
- Misses       1458     1459       +1     
- Partials      361      368       +7     
Flag Coverage Δ
unittests 70.91% <75.00%> (+0.07%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
mmtrack/datasets/mot_challenge_dataset.py 73.48% <75.00%> (-0.26%) ⬇️
mmtrack/datasets/got10k_dataset.py 81.70% <0.00%> (-4.37%) ⬇️
mmtrack/datasets/pipelines/transforms.py 84.73% <0.00%> (-3.90%) ⬇️
mmtrack/datasets/trackingnet_dataset.py 78.94% <0.00%> (-1.88%) ⬇️
mmtrack/datasets/base_sot_dataset.py 86.76% <0.00%> (-0.26%) ⬇️
mmtrack/models/sot/siamrpn.py 81.56% <0.00%> (+1.11%) ⬆️
mmtrack/datasets/sot_train_dataset.py 85.85% <0.00%> (+7.07%) ⬆️
mmtrack/models/trackers/tracktor_tracker.py 79.06% <0.00%> (+15.11%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ccaaab2...7b14bfa. Read the comment docs.

@Seerkfang Seerkfang requested review from GT9505 and removed request for GT9505 February 10, 2022 04:55
'to manually install trackeval')

# get video name sequence for HOTA evaluation
seqmap = osp.join(self.img_prefix, 'videoseq.txt')
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we online generate this txt ?

@GT9505
Copy link
Collaborator

GT9505 commented Feb 14, 2022

change CI and install.md

@@ -51,7 +51,16 @@

如果不是安装预构建的包,而是从源码中构建 PyTorch,则可以使用更多的 CUDA 版本,例如 CUDA 9.0。

3. 安装 mmcv-full,我们推荐您安装以下预构建包:
3. 为VOT数据集测试评估安装额外库(可选)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typically, we need add a blank space between Chinese and English

@GT9505 GT9505 changed the title [Add] support HOTA evaluation for MOTChallenge. [Add] Support HOTA evaluation for MOTChallenge. Feb 15, 2022
@GT9505 GT9505 merged commit 52c8684 into open-mmlab:master Feb 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants