Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

validation accuracy at 50% instead of 60%? #50

Closed
polo5 opened this issue May 4, 2018 · 2 comments
Closed

validation accuracy at 50% instead of 60%? #50

polo5 opened this issue May 4, 2018 · 2 comments

Comments

@polo5
Copy link

@polo5 polo5 commented May 4, 2018

I get 50% accuracy on the ILSVRC2012 validation set without training, which falls short of the 60% boasted in the AlexNet paper (i.e 40.7% top-1 error rate).

Any idea what could be the problem? I'm thinking this could be due to the parameters of the local_response_normalisation layer: yours seem different than the caffe AlexNet here. For instance you have alpha=1e-5 instead of 1e-4. I've been playing with those but have yet to find a configuration that gets me to 60%.

@polo5 polo5 changed the title lrn settings inconsistent validation accuracy at 50% instead of 60%? May 4, 2018
kratzert added a commit that referenced this issue May 7, 2018
@kratzert
Copy link
Owner

@kratzert kratzert commented May 7, 2018

thanks for the hint with the normalization factor. I think I remember that I put it for some reason to 1e-5 but it is so long ago and I coudn't find a commit where this was discussed. So I changed it to the publication value again. Regarding the ImageNet validation: I never tried it actually. One thing, while accuracy could probably not match is the difference in image loading ( See this issue). Another reason might be, that you test on single images. In the original AlexNet paper, they worked with 5 random crops from the original 256x256 images and averaged the predictions.
If you find other reasons, why the accuracy may be different, let me know.

@zhaoyanlyu
Copy link

@zhaoyanlyu zhaoyanlyu commented Jun 4, 2018

Great thanks to @polo5 and, of course, to @kratzert too. The model with 1e-5 alpha gives 23% error rate on validation set of ILSVRC2012, which is close enough. Other tricks mentioned by @kratzert might improve a couple percentage as well.

@kratzert kratzert closed this Jun 4, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants