Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metrics are zero during evaluation #39

Closed
sahityabonumaddi opened this issue Mar 25, 2023 · 1 comment
Closed

Metrics are zero during evaluation #39

sahityabonumaddi opened this issue Mar 25, 2023 · 1 comment
Labels
invalid This doesn't seem right

Comments

@sahityabonumaddi
Copy link

Thanks for your excellent work. I have used pretrained weights of swin backbone and evaluated the model of ADE20k and got he following results.
03/25 23:29:06 d2.evaluation.panoptic_evaluation]: Panoptic Evaluation Results:

PQ SQ RQ #categories
All 0.000 0.000 0.000 150
Things 0.000 0.000 0.000 100
Stuff 0.000 0.000 0.000 50

[03/25 23:46:54 d2.evaluation.testing]: copypaste: Task: sem_seg
[03/25 23:46:54 d2.evaluation.testing]: copypaste: mIoU,fwIoU,mACC,pACC
[03/25 23:46:54 d2.evaluation.testing]: copypaste: 0.0029,0.0005,0.2547,0.0320
[03/25 23:46:54 d2.evaluation.testing]: copypaste: Task: panoptic_seg
[03/25 23:46:54 d2.evaluation.testing]: copypaste: PQ,SQ,RQ,PQ_th,SQ_th,RQ_th,PQ_st,SQ_st,RQ_st
[03/25 23:46:54 d2.evaluation.testing]: copypaste: 0.0000,0.0000,0.0000,0.0000,0.0000,0.0000,0.0000,0.0000,0.0000
[03/25 23:46:54 d2.evaluation.testing]: copypaste: Task: bbox
[03/25 23:46:54 d2.evaluation.testing]: copypaste: AP,AP50,AP75,APs,APm,APl
[03/25 23:46:54 d2.evaluation.testing]: copypaste: 0.0000,0.0000,0.0000,0.0000,0.0000,0.0000
[03/25 23:46:54 d2.evaluation.testing]: copypaste: Task: segm
[03/25 23:46:54 d2.evaluation.testing]: copypaste: AP,AP50,AP75,APs,APm,APl
[03/25 23:46:54 d2.evaluation.testing]: copypaste: 0.0000,0.0000,0.0000,0.0000,0.0000,0.0000

Anyhelp would be appreciated. Thanks in advance !

@praeclarumjj3
Copy link
Member

Hi @sahityabonumaddi, you evaluated OneFormer after only loading the pretrained (on classification) backbone weights. You need to load the checkpoint jointly trained for the segmentation task. The checkpoints are given here.

@praeclarumjj3 praeclarumjj3 added the invalid This doesn't seem right label Mar 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
invalid This doesn't seem right
Projects
None yet
Development

No branches or pull requests

2 participants