-
Notifications
You must be signed in to change notification settings - Fork 4.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
BatchNorm Layer: it seems to be safer to add a epsilon to the standard deviation #303
Comments
epsilon value is handled in model converter that is pre-added to variance |
Why there is not a parameter for epsilon in the layer BatchNorm like in the layers such as InstanceNorm and MVN, I've found |
It is for some historical reason, and it is fine to add eps parameter with default value of zero for consistency. |
cook
added a commit
to cook/ncnn
that referenced
this issue
Mar 27, 2018
ghimiredhikura
pushed a commit
to ghimiredhikura/NCNNSFD
that referenced
this issue
Jul 26, 2020
…encent#311) * Add the epsilon parameter to the BatchNorm layer. (fix Tencent#303) * Move the eps into the sqrt.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
In the following codes cited from the file src/layer/batchnorm, if the
sqrt_var
is zero, there will be a zero-division error. So I advice to add a epsilon to the standard deviation or add some protection codes.(虽然可能性不大, 但加上去毕竟更安全些)
The text was updated successfully, but these errors were encountered: