Skip to content

amsukdu/SimplePythonRNN

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

4 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

SimplePythonRNN

This is RNN, LSTM & GRU with Bidirectional only in python & numpy. It is simple and slow but will get the job done 👍

Specification

Type : Vanilla, LSTM, GRU + Bidrectional

Weight Initialization : HE Normal

Weight Update Policy : ADAM, ADAGRAD

Regulization : Droupout

Prerequisites

numpy (+ mkl for intel processors. recommend anaconda)

Example

seq_length = 128
learning_rate = 1e-3
rnn = RNN(
    [
        {'type': 'lstm', 'hidden_size': 256},
        {'type': 'lstm', 'hidden_size': 256},
        {'type': 'lstm', 'hidden_size': 256},
        # {'type': 'lstm', 'hidden_size': vocab_size, 'dropout': 0.5, 'bi': True, 'u_type': 'adagrad'},
    ],
    vocab_size, learning_rate, bi=False)

Example RNN is trained by Tolstoy's famous novel "Anna Karenina".

she timed in anna, and coulderent a government manner, stiva fields, scowling him pertuable to vrinss myself, and quieted it out in any melancholorilly his passing feeling of the intensery which she would forget him, and they his brother's with tcharge upon him in her eyes "for you who this name," the son in amiable of his shohy lots.

Generated string over 65000 iterations.

API Reference

Parameter Description
type 'lstm', 'gru', 'vanilla'
hidden_size hiddden dimension size
lr learning rate
u_type 'adam', 'adagrad'
bi Bidirectional. True, False

License

MIT