Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

为什么 CNN's conv bias 设为 False,如 ResNet、SEnet等 #17

Open
jayboxyz opened this issue Nov 15, 2019 · 0 comments
Open

为什么 CNN's conv bias 设为 False,如 ResNet、SEnet等 #17

jayboxyz opened this issue Nov 15, 2019 · 0 comments
Labels
CNN-卷积神经网络 卷积神经网络

Comments

@jayboxyz
Copy link
Owner

jayboxyz commented Nov 15, 2019

参考:kuangliu/pytorch-cifar#52

Any channel bias added would only affect the channel mean. Since BatchNorm2d is applied after Conv2d and will remove the channel mean, there's no point of adding bias to Conv2d.
To put it in another way, the gradient w.r.t. the conv bias will be zero anyway.

添加的任何通道偏置只会影响通道平均值。 由于BatchNorm2d在Conv2d之后应用,并且将删除通道均值,因此没有必要向Conv2d添加偏差。
换句话说,梯度w.r.t. 无论如何,转化偏倚将为零。

@jayboxyz jayboxyz added the CNN-卷积神经网络 卷积神经网络 label Jan 7, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CNN-卷积神经网络 卷积神经网络
Projects
None yet
Development

No branches or pull requests

1 participant