Question in Evaluate

[muzhaohui]

Hello there!
First of all, thank you for your outstanding work! I have a problem when reproducing your work.

I'm use the following command to evaluate.
python evaluate.py --config configs/mm-distillnet.cfg --checkpoint trained_models/mm-distillnet.0.pth.tar

But get bad performance.Can you help me how to improve?

Thanks!

[sunwell1994]

same problem here. Do you use the given version of python, Pytorch, and Cuda? My version is a bit different, but it should not degrade the performance so much during inference.

[muzhaohui]

同样的问题在这里。您是否使用给定版本的 python、Pytorch 和 Cuda？我的版本有点不同，但它不应该在推理过程中降低性能。

I found that the cause of the problem was the thermal teacher, because there was basically no output during the test. And the provided model cannot be fully imported when importing. I feel that the author’s thermal teacher model may not be EfficientDet_D2 in the code.
#15

[sunwell1994]

同样的问题在这里。您是否使用给定版本的 python、Pytorch 和 Cuda？我的版本有点不同，但它不应该在推理过程中降低性能。

I found that the cause of the problem was the thermal teacher, because there was basically no output during the test. And the provided model cannot be fully imported when importing. I feel that the author’s thermal teacher model may not be EfficientDet_D2 in the code.
#15

Are teacher models called during inference? I thought that teacher models are used during training to generate labels and only the audio regression model will be used for inference.

[catherine-qian]

hi, can I know your dataset structure?

Because from the readme, it should contains 50 directories with “ We split the data using a 60 (train) /20 (validation)/20 (test) scheme, which can be found under the SPLITS directory”:
http://multimodal-distill.cs.uni-freiburg.de/static/dist/MAVD_dataset/README.txt?

When I download the dataset, I only get 12 directories with the following structures:
432K drive_day_2020_03_18_15_42_58
53M drive_day_2020_04_14_14_33_17
314M drive_day_2020_04_14_15_10_13
173M drive_day_2020_04_14_15_20_14
202M drive_day_2020_04_14_16_16_27
862M drive_day_2020_04_21_15_48_20
488M drive_day_2020_04_21_15_58_22
2.6G drive_day_2020_05_21_20_55_36
5.6G drive_night_2020_05_21_21_05_38
7.1G drive_night_2020_05_21_22_22_50
11G drive_night_2020_05_31_22_00_00
104M drive_night_2020_05_31_22_23_23

[eyeris]

@muzhaohui May I know how did you make it runnable?
I used the same method but end up getting this.

Also, I have the following issue as well. Tried to reach the authors in many ways but didn't have any reply.
#18

[muzhaohui]

@muzhaohui May I know how did you make it runnable? I used the same method but end up getting this.

Also, I have the following issue as well. Tried to reach the authors in many ways but didn't have any reply. #18

1. Check if there is any problem with the path
2. If it doesn't work, delete the offending data

[liushibei]

Have you solved this problem?
The performance of the model I trained is very poor

The performance of the model provided by article are also very poor.