Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot reproduce the test accuracy. #27

Open
mzr1996 opened this issue Sep 6, 2022 · 0 comments
Open

Cannot reproduce the test accuracy. #27

mzr1996 opened this issue Sep 6, 2022 · 0 comments

Comments

@mzr1996
Copy link

mzr1996 commented Sep 6, 2022

I tried to run the test of HRFormer on ImageNet-1k, but the test result was strange. The top-1 accuracy is about 2.0%

Test command

bash run_eval.sh hrt/hrt_tiny ~/Downloads/hrt_tiny_imagenet_pretrained_top1_786.pth  ~/data/imagenet

Test output

[2022-09-06 15:00:15 hrt_tiny](main.py 157): INFO number of params: 8035820
All checkpoints founded in output/hrt_tiny/default: []
[2022-09-06 15:00:15 hrt_tiny](main.py 184): INFO no checkpoint found in output/hrt_tiny/default, ignoring auto resume
[2022-09-06 15:00:15 hrt_tiny](utils.py 21): INFO ==============> Resuming form /home/mzr/Downloads/hrt_tiny_imagenet_pretrained_top1_786.pth....................
[2022-09-06 15:00:15 hrt_tiny](utils.py 31): INFO <All keys matched successfully>
[2022-09-06 15:00:19 hrt_tiny](main.py 389): INFO Test: [0/391]	Time 4.122 (4.122)	Loss 8.9438 (8.9438)	Acc@1 2.344 (2.344)	Acc@5 4.688 (4.688)	Mem 2309MB
[2022-09-06 15:00:29 hrt_tiny](main.py 389): INFO Test: [10/391]	Time 1.028 (1.279)	Loss 9.0749 (9.3455)	Acc@1 5.469 (2.486)	Acc@5 12.500 (7.031)	Mem 2309MB
[2022-09-06 15:00:39 hrt_tiny](main.py 389): INFO Test: [20/391]	Time 1.027 (1.159)	Loss 9.9610 (9.3413)	Acc@1 0.781 (2.269)	Acc@5 4.688 (7.403)	Mem 2309MB
[2022-09-06 15:00:49 hrt_tiny](main.py 389): INFO Test: [30/391]	Time 0.952 (1.103)	Loss 9.1598 (9.3309)	Acc@1 1.562 (2.293)	Acc@5 7.812 (7.359)	Mem 2309MB
[2022-09-06 15:00:59 hrt_tiny](main.py 389): INFO Test: [40/391]	Time 0.951 (1.071)	Loss 9.3239 (9.3605)	Acc@1 0.781 (2.210)	Acc@5 4.688 (7.241)	Mem 2309MB
[2022-09-06 15:01:09 hrt_tiny](main.py 389): INFO Test: [50/391]	Time 0.952 (1.049)	Loss 9.7051 (9.3650)	Acc@1 0.781 (2.191)	Acc@5 3.125 (7.200)	Mem 2309MB
[2022-09-06 15:01:18 hrt_tiny](main.py 389): INFO Test: [60/391]	Time 0.951 (1.035)	Loss 9.5935 (9.3584)	Acc@1 1.562 (2.075)	Acc@5 7.812 (7.095)	Mem 2309MB
...

The environment is brand new according to the install instruction, and the checkpoint is from https://github.com/HRNet/HRFormer/releases/tag/v1.0.0 . The only change is I disabled the amp.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant