Join GitHub today
bi-directional rnn alignment when masked #2536
basically: (0 is mask)
The desired output should be aligned
To have a proper aligned bi-directional RNN. And actually.. being able to set the offset would also be useful.. If one wanted.
Tensorflow does not have this problem. They have a custom implementation which addresses it.
I've **scoured theano, lasagne, blocks, pylearn issues and code, but I can't find any implementations that would work.
At the moment, I have the following possible solution:
**footnote. though, the internets are vast. I probably missed things.
Sooo. I assumed something earlier and thought this was the original BRNN architecture style (aligning in this way). I don't think it is. But, It's something I am going to be using and it may come up for someone else, so I'll just leave the issue up for now.
For example [1, 2, 3, 4, 0, 0], say both W and U are ones.
If [4, 7, 9, 10, 0, 0] is what you want, then you can also do something like:
Hi @xingdi-eric-yuan ,
thanks =). Ya, I think I got confused while reading a paper on something, got really focused on figuring out how to solve it, and didn't realize my mistake until after I posted.
also, I didn't know about alloc!
side note: There is an issue with nonzero in how it indexes (for example, it gives you a tuple, so you've have to do nonzero()). I usually use it to do accuracy with masks.
The issue here was that each row in your matrix will have different size paddings. And if you want the first element of a forward RNN to align with the first element of a backward RNN, it's a bit wonky.