Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jaccard Distance values look off - over 90% are 1s #31

Open
010akv opened this issue Sep 20, 2022 · 0 comments
Open

Jaccard Distance values look off - over 90% are 1s #31

010akv opened this issue Sep 20, 2022 · 0 comments

Comments

@010akv
Copy link

010akv commented Sep 20, 2022

With no reranking, distance values are distributed evenly across the whole 0-1 range. But with reranking, although the accuracy improves ~2%, the distance histogram gets compressed into narrow range (0.7-0.9) with several distance values > 1. This looks incorrect. I studied the problem further and found that this is due to Jaccard distance that has over 90% values as 1s.

How I got this issue:

Use custom dataset (unable to share the dataset, 4k images, 200 identities) with reranking (k1=54, k2=6, lambda=0.3. - this gave the best overall fscr). Plot histograms of

  • original distance - looks right - distributed evenly across the whole range 0-1
  • Jaccard distance - looks way off - over 85% values are 1s.
  • final distance - looks off due to Jaccard distance. Distributed in narrow window - 0.7-0.9. Many values are > 1!

Expected behavior:

Final distance must be evenly distributed across the whole 0-1 range.

Questions:

  1. What is 2-temp_min here? Is it same as the max operation - eq. 10 in the paper?
  2. Does k1 depend on the number of images in the dataset? V is of size NxN (total num images). When k1 << N V has many 0s even after qe. Could this be why my Jaccard distances are all 1s?
  3. Is this expected of the distance distribution? If yes, why and if not, any suggestions to fix?
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant