Gated Orthogonal Recurrent Unit
This model combines gating mechanism and orthogonal RNN approach. It solves forgetting problem and long-term dependency.
If you find this work useful, please cite arXiv:1706.02761.
requires TensorFlow 1.2.0
To use GORU in your model, simply copy goru.py.
Then you can use GORU in the same way you use built-in LSTM:
from goru import GORUCell cell = GORUCell(hidden_size, capacity, fft)
Integer. Only works for tunable style.
True, GORU is set to FFT style. Default is
Example tasks for GORU
We put two examples toy tasks: copying task and denoise task
Copying Memory Task
python copying_task.py --model GORU
python denoise_task.py --model GORU