-
Notifications
You must be signed in to change notification settings - Fork 17
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Will you provide the evaluation on the out-of-domain datasets and the enriched datastore? #6
Comments
Hi @dongwhfdyer, glad to hear you've been able to reproduce the COCO-index results. @RitaRamo and I will follow up on your main request shortly. For now I can clarify that |
That's great! And will you release the augmented datastore in the future? |
@dongwhfdyer the easiest way to reproduce the results from the table above would be for us to share the retrieved neighbors with you (the equivalent to If the trouble you are facing is just with the actual evaluation script (i.e. with adapting the coco-captions metric computation to the other datasets), let us know and we'll share the relevant files with you. |
Thank you! I think I need both. 😄 |
Just to make sure, of the three things I mentioned: nearest neighbors, indexes, and evaluation files, which ones do you need? |
I need nearst neighbors, indexes, and evaluation files. |
Sure, we'll try to provide as soon as possible, probably tomorrow :) |
@RitaRamo Thank you for your fantastic work! |
Hi, Those ids correspond to the ids of the images. As in your example, id 67 can be found in
All the validation images and test images are in the |
Hi~ Thanks for sharing data! |
Hey! We are doing research following your work. We have reproduced your work over coco dataset which performs very well, even better than your provided data! But when it comes to other datasets you referred in your paper, like Vizwiz, MSR-VTT, we find it very complicated to get the corresponding metrics results. So I wonder if you could provide the code for these experiments?
In addition, is the
datastore/coco_index_captions.json
the augmented datastore you mentioned in 5.2 Augmenting the datastore?The text was updated successfully, but these errors were encountered: