How to use the pretrained models for Pascal-VOC mentioned in the Training stage of READMED

[RockJim2001]

After reading the whole paper, what i know is that the whole training process of the model combines with two stages.In the first stage, the end iteration of the model is 12000.In the second stage, the end iteration of the model is 18000.

However, i got a ValueError that is 'loaded state dict contains a parameter group that doesn't match the size of optimizer's group', when i train the model with the file named model_final.pth provided by the file named README.md.

The biggest puzzle is that why from the provided 'model_final.pth' file, a saved iteration i got is 17999. Is the provided file saved from the whole training stage or only the pretrained models for Pascal-VOC?

I'm looking forward to your reply!

[Went-Liang]

The model_final.pth provided by Google Drive is the final training file. This file contains an unused operator, "roi_heads.similarity_feat.{bias, weight}," which was not used during training and was accidentally left in the model. I suspect that this may be causing the error you're encountering. You can remove this operator's state_dict using Torch.

Additionally, when testing the model_final.pth, you'll need to create a file named last_checkpoint in the directory "UnSniffer/detection/data/VOC-Detection/faster-rcnn/UnSniffer/random_seed_0" with the content "model_final.pth"

[RockJim2001]

@Went-Liang
I'glad to receive your reply!

The question of how to use the provided weight has been addressed by myself!

Evaluating the model only with the model_final.pth provided by Google Drive,the result i got is showed as follow:

The parameter of ncut_threshold and energy_threshold is 0.8, 4.504916269302372 respectively. This result is got after charging some code as the following:

Is it nesscessary?
By the way, is it has a important influence on the evaluated result only with one gpu on the evaluated stage with the provided pretraining model?

[Went-Liang]

The iteration step should be 0.01 instead of 0.1. I apologize for the typo in the paper. When testing with the code and model from the repository, an ncut_threshold value of 0.71 was obtained, and the test results match the paper at this threshold. Because grid_traverse.py has a low computational efficiency, the code is written as np.arange(0.68, 0.72, 0.01).

Additionally, the number of GPUs should not affect the results.

[RockJim2001]

I'm glad to receive your reply timely! Your interpretation is so clear and patient. I will try the advice you provided. Now I can't respond to your advice, the re-evaluating model will take a little longer. The most important is that I believe the method is an imaginative breakthrough. I can't repeat the result because I just used an incorrect training method! I believe I will repeat the result under the author's guidance! The only one I want to say is '666'!

[Went-Liang]

OVO

[RockJim2001]

@Went-Liang
Oh, the best terrible is that I haven't gotten a similar result with the paper. Can you help to analyze the log files speeding up your valuable time? If you agree, I would be most grateful! For your convenience, I can provide you with my Sunflower software account by email to help me remotely! If so, all the better. I would be very grateful!

[RockJim2001]

I have repeated the result compared with the paper！

[beiyan1911]

I have repeated the result compared with the paper！

@RockJim2001 Hello, friend. The results I obtained are quite similar to the ones you initially obtained, that is, they differ significantly from the results in the paper. This question has been bothering me for several days. Have you finally obtained results similar to those in the paper? Can you help me point out where my problem lies? If I could receive your reply, I would be very grateful. My test results on COCO-OOD are as follows:

Ap: 0.17466372511974906
precision: 0.01768584195231697
recall: 0.6831623415811708
f1: 0.034479082588367876

the thresholds:
energy threshold: 4.472734236717228
ncut threshold: 0.7100000000000001

[RockJim2001]

I suggest that you could test the model with the pretraind weight provided by the paper！If you need my trained weight， i will share it to you by Email！ @beiyan1911

[beiyan1911]

Thank you very much for your reply. The result I got was using the pre-trained model provided in the paper. My email address is beiyan1911@gmail.com. If possible, please send me your trained model for testing. This will help me find where my problem is. @RockJim2001

[RockJim2001]

I'm sorry for the delay in recovering your email. Attached are the results of using the pre-trained weights provided by the paper and the results of evaluating the weights trained by me. I hope it will be useful to you! 发件人：beiyan1911 ***@***.***> 发送日期：2023-12-07 13:49:56 收件人：Went-Liang/UnSniffer ***@***.***> 抄送人：RockJim2001 ***@***.***>,Mention ***@***.***> 主题：Re: [Went-Liang/UnSniffer] How to use the pretrained models for Pascal-VOC mentioned in the Training stage of READMED (Issue #14) Thank you very much for your reply. The result I got was using the pre-trained model provided in the paper. My email address is ***@***.*** If possible, please send me your trained model for testing. This will help me find where my problem is. @RockJim2001 — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***> 从网易企业邮箱发来的云附件 新建文件夹.rar 588.11M | 过期时间：2023年12月27日 14:57 下载

[beiyan1911]

@RockJim2001

Thank you for your reply. Perhaps there are some problems with the reply function of GitHub, and I cannot see the file download link you shared. If it's convenient, could you please send it directly to my email. Alternatively, you can briefly tell me if the test results of the model you trained are close to the results reported in the paper. If the answer is yes, it indicates that there may be some issues with my local code, possibly due to different software environments. I will investigate and find the cause. Thank you very much.