subLSTMs for pytorch from Cortical microcircuits as gated-recurrent neural networks
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
subLSTM
tasks/word_language_model
test
.gitignore
.travis.yml
LICENSE
README.md
release.sh
setup.cfg
setup.py

README.md

subtractive LSTM (subLSTM), for Pytorch

Build Status PyPI version

This is an implementation of subLSTM described in the paper Cortical microcircuits as gated-recurrent neural networks, Rui Ponte Costa et al.

Install

pip install pytorch-sublstm

Usage

Parameters:

Following are the constructor parameters:

Argument Default Description
input_size None Size of the input vectors
hidden_size None Size of hidden units
num_layers 1 Number of layers in the network
bias True Bias
batch_first False Whether data is fed batch first
dropout 0 Dropout between layers in the network
bidirectional False If the network is bidirectional

Example usage:

nn Interface

import torch
from torch.autograd import Variable
from subLSTM.nn import SubLSTM

hidden_size = 20
input_size = 10
seq_len = 5
batch_size = 7
hidden = None

input = Variable(torch.randn(batch_size, seq_len, input_size))

rnn = SubLSTM(input_size, hidden_size, num_layers=2, bias=True, batch_first=True)

# forward pass
output, hidden = rnn(input, hidden)

Cell Interface

import torch
from torch.autograd import Variable
from subLSTM.nn import SubLSTMCell

hidden_size = 20
input_size = 10
seq_len = 5
batch_size = 7
hidden = None

hx = Variable(torch.randn(batch_size, hidden_size))
cx = Variable(torch.randn(batch_size, hidden_size))

input = Variable(torch.randn(batch_size, input_size))

cell = SubLSTMCell(input_size, hidden_size, bias=True)
(hx, cx) = cell(input, (hx, cx))

Tasks:

A language modeling task is included here. Refer to its README for more info.

Attributions:

A lot of the code is recycled from pytorch