Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add algorithmic optimizer to convert Log(Softmax(x)) to LogSoftmax(x) #25455

Merged
merged 1 commit into from Mar 7, 2019
Merged

Add algorithmic optimizer to convert Log(Softmax(x)) to LogSoftmax(x) #25455

merged 1 commit into from Mar 7, 2019

Commits on Feb 28, 2019

  1. Add algorithmic optimizer to convert Log(Softmax(x)) to LogSoftmax(x) 

    This PR adds an algorithmic optimizer which converts `Log(Softmax(x))` to `LogSoftmax(x)`. [`LogSoftmax`](https://www.tensorflow.org/api_docs/cc/class/tensorflow/ops/log-softmax) is numerically more stable and may be a bit faster in some cases.

This could be expanded in the future to also optimize `log(softmax(x) * y) = logsoftmax(x) + log(y)` and `log(softmax(x) / y) = logsoftmax(x) - log(y)`.
    @lgeiger
    lgeiger committed Feb 28, 2019
    Copy the full SHA
    597f02c View commit details
    Browse the repository at this point in the history