New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cannot reproduce the author's results with the pre-trained models #9
Comments
I will update the code to support latest pytorch version. The result seems a little weird. Could you provide more details to reimplement the result? |
Could you get the same result as our provided notebook? |
Hi @dddzg , Thanks for the reply, much appreciated. It's working now. ;) It was a mistake on my side. The source-code is not that different, I just refactored it a bit and added unit tests. The problem was that for the checkpoint loader I had a separate unit test. So, the order is not guaranteed and the checkpoint was not loaded in time for the inference to happen. To fix it, I moved the model building and checkpoint loading phase to a Custom image with 10 hand-engineered patches Kittens image with 10 hand-engineered patches Kittens image with 10 random generated patches Custom image with 10 random generated patches As you can see, it works. However, we need some improvements when it comes to random-patches. The boxes are not good. The authors of this paper claimed to have improved it using region proposal via selective search: https://arxiv.org/pdf/2106.04550.pdf. No pre-trained models yet, though. Thanks again and congrats on the good work. |
Hi, @wilderrodrigues . The boxes are not good with random cropped patches. Because the task is just the pre-text task. We don't really care about the accuracy of the pre-text task. |
Hi @dddzg , Thanks for the extra info. We will probably try to fine-tune it and see how it behaves. Quick question: during fine-tuning I can change the number of queries / patches, right? Because now it's upper bounded to 100 and 10, respectively. For instance, when trying the fine-tuned COCO checkpoint you made available, on my custom image, I got this: I used 10 random patches. So, the result is reasonably good. I would expect that with more patches we could find more objects. Will keep you posted on my experiments / changes to the code. Thanks again. |
Yeah, just checked and I will fine tune and change the queries and patches. :)
Need to convert my dataset to the COCO format first. |
Hi there,
I'm currently experimenting with some Few/One/Zero-Shot for object detection and classification. For one of the tasks, your paper has been experimented with.
Unfortunately, I haven't been able to reproduce your results with the pre-trained models you have made available. I also noticed that the inference code you made available does not work out of the box. To support my points, here some details:
Results:
Patches
Detections
Patches
Detections
Hardware used
Yeah, I tried with both CPU and a CUDA compliant device.
Are you sure you have uploaded the rights checkpoint files?
Thanks in advance and looking to hear from you.
The text was updated successfully, but these errors were encountered: