Skip to content
Gated Orthogonal Recurrent Unit implementation in tensorflow
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.gitignore
LICENSE
README.md
copying_task.py
denoise_task.py
goru.py

README.md

GORU-tensorflow

Gated Orthogonal Recurrent Unit

This model combines gating mechanism and orthogonal RNN approach. It solves forgetting problem and long-term dependency.

If you find this work useful, please cite arXiv:1706.02761.

Installation

requires TensorFlow 1.2.0

Usage

To use GORU in your model, simply copy goru.py.

Then you can use GORU in the same way you use built-in LSTM:

from goru import GORUCell
cell = GORUCell(hidden_size, capacity, fft)

Args:

  • hidden_size: Integer.
  • capacity: Optional. Integer. Only works for tunable style.
  • fft: Optional. Bool. If True, GORU is set to FFT style. Default is True.

Example tasks for GORU

We put two examples toy tasks: copying task and denoise task

Copying Memory Task

python copying_task.py --model GORU

Denoise Task

python denoise_task.py --model GORU
You can’t perform that action at this time.