Output of softmax GRU and LSTM layers does not add up to 1

[louisabraham]

import numpy as np
from keras.models import *
from keras.layers import *
m = Sequential()
m.add(LSTM(3, input_shape=(3,2), activation='softmax'))
print(m.predict(np.random.rand(5,3,2)).sum(axis=-1))

[ 0.56759441  0.59162366  0.57279199  0.52342385  0.54326206]

Works fine with Dense or SimpleRNN.
If this is normal, it should be specified in the doc, because it is NOT logical…

[joelthchao]

Yes, this is somehow misleading. Activation here is directly apply on each hidden unit. However, we won't use LSTM in this way. Usually, we do it by:

m.add(LSTM(hidden_unit, input_shape=(3,2)))
m.add(Dense(3, activation='softmax')) 

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 30 days if no further activity occurs, but feel free to re-open a closed issue if needed.