Initialization in Stacked LSTM

[varunkumar3618]

I am trying to build a stacked LSTM model and have having trouble initializing the weights of the intermediate transformations. My code looks like this:
cells = [LSTM(dim=self._hidden_size, weights_init=initialization.Identity())]
stack = RecurrentStack(transitions=transitions, weights_init=wi, biases_init=bi)

It seems that the only initializer accepted by RecurrentStack is Constant. If I pass in Identity or IsotropicGaussian, the code allocating weights throws a ValueError when it notices that the incoming state is a hidden state-cell state pair and not a matrix.

It would be extremely helpful if you could modify the interface to allow arbitrary initializations or let me know how to work around this restriction. Thanks!

[rizar]

Sorry for the late reply. If you are still interested, can you please post an code example that reproduces the issue?

…

On Thu, 1 Dec 2016 at 17:56 varunkumar3618 ***@***.***> wrote: I am trying to build a stacked LSTM model and have having trouble initializing the weights of the intermediate transformations. My code looks like this: cells = [LSTM(dim=self._hidden_size, weights_init=initialization.Identity())] stack = RecurrentStack(transitions=transitions, weights_init=wi, biases_init=bi) It seems that the only initializer accepted by RecurrentStack is Constant. If I pass in Identity or IsotropicGaussian, the code allocating weights throws a ValueError when it notices that the incoming state is a hidden state-cell state pair and not a matrix. It would be extremely helpful if you could modify the interface to allow arbitrary initializations or let me know how to work around this restriction. Thanks! — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1169>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAn8YuZc-zHNytPlbeOWMzp8EIQBu6Rmks5rD1C6gaJpZM4LB_i9> .