Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using "kernel" for weight matrix in Dense layer seems confusing #10010

Closed
amueller opened this issue Apr 23, 2018 · 2 comments
Closed

Using "kernel" for weight matrix in Dense layer seems confusing #10010

amueller opened this issue Apr 23, 2018 · 2 comments

Comments

@amueller
Copy link
Contributor

The "Dense" layer has parameters "kernel_initializer", "kernel_regularizer" and "kernel_constraint" which in my opinion are quite confusing names. I have not seen the weights in a dense layer being referred to as "kernel". The docs say "Initializer for the kernel weights matrix". I assume these parameters are named for consistency with the convolutional kernels and changing them might be tricky (though I'm not sure if consistency with the convolutional layers is really a good criterion here).

But I think the documentation could be a bit more clear. If the docs just said "weight matrix" instead of "kernel weight matrix" I feel that would be easier to understand.

@fchollet
Copy link
Member

fchollet commented Apr 23, 2018 via email

@amueller
Copy link
Contributor Author

amueller commented Apr 23, 2018

Thanks for your reply. If it's used in keras and tensorflow I guess it is now canonical (I have not seen it in a deep learning paper though). Having done deep learning before either existed, it's really foreign to me. In the language I am used to, weights did not include the biases. I guess tensorflow and keras decided a different nomenclature. "Kernel" is a very overloaded term in ML and I feel this usage is not helping.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants