Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Questions about training tests #9

Closed
wlc-git opened this issue Nov 30, 2021 · 10 comments
Closed

Questions about training tests #9

wlc-git opened this issue Nov 30, 2021 · 10 comments

Comments

@wlc-git
Copy link

wlc-git commented Nov 30, 2021

Use the code provided by the author for training, which is obtained
MR_all: 98.65
MR_day: 99.52
MR_night: 96.37
Recall_all: 6.53
May I ask what does MR mean? Why is it that recall is getting lower and lower when I retrain? What do these mean respectively and why are they different from the data displayed in your paper
MR_all: 7.58
MR_day: 7.96
MR_night: 6.95
Recall_all: 96.70
Different, Is there something I didn't adjust，a little confused, looking forward to your answer, thank you
@socome
Copy link
Member

socome commented Nov 30, 2021 •
edited
Loading

image

Thank you for your interest in our work again! We verified the code again by implementing it in a new docker container. However, there was no error found. It usually takes 20~30 epochs to achieves the similar performance to the result in our paper. Did you run the code in a docker container ? Please, make sure to follow the same environments we provided. Also, MR means the log-averaged miss-rate sampled against FPPI. Please refer to our paper for more detailed information. link

@wlc-git
Copy link
Author

wlc-git commented Nov 30, 2021 via email

@wlc-git
Copy link
Author

wlc-git commented Nov 30, 2021 via email

@xown3197
Copy link
Member

Could you show me the terminal log?

@wlc-git
Copy link
Author

wlc-git commented Nov 30, 2021 via email

@xown3197
Copy link
Member

I can't see anything.

The training data in the KAIST dataset consists of 25k RGB-Thermal pairs.
But you said you used 7541 images for your training data.
If you are right about wanting to use the KAIST dataset, I think you should check the data set.

@wlc-git
Copy link
Author

wlc-git commented Nov 30, 2021 via email

@rgw117
Copy link
Member

rgw117 commented Nov 30, 2021 •
edited
Loading

I am not sure what you mean by " too many samples of the data set". The KAIST dataset consists of 12 subsets ranging from 0 to 11, and 0 ~ 5 subsets are used for training whereas 6~11 subsets are used for inference. The number of training dataset cannot be 7,595 but should be a lot more. You can refer to this website for more information of the dataset.

@unizard
Copy link
Member

unizard commented Nov 30, 2021

@wlc-git Please, follow the evaluation protocol [1] and solve your problem by yourself.

[1] Multispectral Pedestrian Detection: Benchmark Dataset and Baselines, CVPR 2015.

@wlc-git
Copy link
Author

wlc-git commented Nov 30, 2021 via email

@socome socome closed this as completed Nov 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@unizard @socome @xown3197 @wlc-git @rgw117