New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use nn.softmax in keras.activations.softmax #47572
Conversation
nn.softmax is used for tensors with all dimensions added test_softmax_2d_axis0 covering fixed bug
Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). 📝 Please visit https://cla.developers.google.com/ to sign. Once you've signed (or fixed any issues), please reply here with What to do if you already signed the CLAIndividual signers
Corporate signers
ℹ️ Googlers: Go here for more info. |
@googlebot I signed it! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thanks
There are two issues with tf.keras.activations.softmax :
First:
it implements softmax for 3D+ tensors in not optimal way due to old workarounds:
For 3D tensor, the softmax gradient is getting really complicated which seriously hurts performance. example:
Second:
It returns wrong results for 2D tensors, axis=0. I also prepared unit test for this scenario.