New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

validation accuracy at 50% instead of 60%? #50

Closed
polo5 opened this Issue May 4, 2018 · 2 comments

Comments

Projects
None yet
3 participants
@polo5

polo5 commented May 4, 2018

I get 50% accuracy on the ILSVRC2012 validation set without training, which falls short of the 60% boasted in the AlexNet paper (i.e 40.7% top-1 error rate).

Any idea what could be the problem? I'm thinking this could be due to the parameters of the local_response_normalisation layer: yours seem different than the caffe AlexNet here. For instance you have alpha=1e-5 instead of 1e-4. I've been playing with those but have yet to find a configuration that gets me to 60%.

@polo5 polo5 changed the title from lrn settings inconsistent to validation accuracy at 50% instead of 60%? May 4, 2018

kratzert added a commit that referenced this issue May 7, 2018

@kratzert

This comment has been minimized.

Owner

kratzert commented May 7, 2018

thanks for the hint with the normalization factor. I think I remember that I put it for some reason to 1e-5 but it is so long ago and I coudn't find a commit where this was discussed. So I changed it to the publication value again. Regarding the ImageNet validation: I never tried it actually. One thing, while accuracy could probably not match is the difference in image loading ( See this issue). Another reason might be, that you test on single images. In the original AlexNet paper, they worked with 5 random crops from the original 256x256 images and averaged the predictions.
If you find other reasons, why the accuracy may be different, let me know.

@JoLivermorium

This comment has been minimized.

JoLivermorium commented Jun 4, 2018

Great thanks to @polo5 and, of course, to @kratzert too. The model with 1e-5 alpha gives 23% error rate on validation set of ILSVRC2012, which is close enough. Other tricks mentioned by @kratzert might improve a couple percentage as well.

@kratzert kratzert closed this Jun 4, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment