Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How can I do zero-shot imagnet1k test with the surgery model? #16

Closed
JianbangZ opened this issue May 22, 2023 · 1 comment
Closed

How can I do zero-shot imagnet1k test with the surgery model? #16

JianbangZ opened this issue May 22, 2023 · 1 comment

Comments

@JianbangZ
Copy link

No description provided.

@Eli-YiLi
Copy link
Contributor

This issue has been discussed in #13 .

The answer is that "Single label like imagenet is not influenced, because architecture surgery is designed for explainability task, and feature surgery compute the redundant feature as a common bias for each class. Thus, giving a same bias doesn’t change the rank and accuracy, instead it influences scores across images and benefits mAP for multi-label".

If you want to test classification mAP, the first step is to record cosine similarity from the original path with feature surgery, and then eval with gt using package like torchmetrics.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants