Skip to content
/ lstm Public
forked from nicodjimenez/lstm

Minimal, clean example of lstm neural network training in python, for learning purposes.

Notifications You must be signed in to change notification settings

sysgeass/lstm

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

34 Commits
 
 
 
 
 
 

Repository files navigation

lstm

A basic lstm network can be written from scratch in a few hundred lines of python, yet most of us have a hard time figuring out how lstm's actually work. The original Neural Computation paper is too technical for non experts. Most blogs online on the topic seem to be written by people who have never implemented lstm's for people who will not implement them either. Other blogs are written by experts (like this blog post) and lack simplified illustrative source code that actually does something. The Apollo library built on top of caffe is terrific and features a fast lstm implementation. However, the downside of efficient implementations is that the source code is hard to follow.

This repo features a minimal lstm implementation for people that are curious about lstms to the point of wanting to know how lstm's might be implemented. The code here follows notational conventions set forth in this well written tutorial introduction. This article should be read before trying to understand this code (at least the part about lstm's). By running python test.py you will have a minimal example of an lstm network learning to predict an output sequence of numbers in [-1,1] by using a Euclidean loss on the first element of each node's hidden layer.

Play with code, add functionality, and try it on different datasets. Pull requests welcome.

Please read my blog article if you want details on the backprop part of the code.

This sample code has been ported to the D programming language by Mathias Baumann: https://github.com/Marenz/lstm as well as Julia by @hyperdo https://github.com/hyperdo/julia-lstm, Alfiuman in C++ (with CUDA) https://github.com/Alfiuman/WhydahGally-LSTM-MLP and Ascari in JavaScript (for nodejs) https://github.com/carlosascari/lstm.

About

Minimal, clean example of lstm neural network training in python, for learning purposes.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 100.0%