No description, website, or topics provided.
Switch branches/tags
Nothing to show
Clone or download
Pull request Compare This branch is even with DipLernin:master.
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
code
datasets
expriments
plots
static/js
trained_models
.gitignore
LICENSE.txt
README.md
index.html
optimzations.txt

README.md

Text Generation

The goal of this project is to generate text, accordingly to what our system has learned from its training, analyzing the text of certain datasets. Therefore the main idea is to predict the next characters given an input text. An example is presented below:

Architecture

The architecture built is described by this figure:

  1. The input used is sequences formed by 40 one-hot encoding characters. There are 59 possible characters.
  2. An RNN(Recurrent Neural Network) layer to take into account the temporal information of the data.
  3. A softmax, which for each possible character gives the corresponding probability of being the next.
  4. The output is chosen by predicting the character with the largest probability.

Different models were tried for this task, which their differences lie in which RNN is implemented:

  • One layer LSTM (Long Short-Term Memory) with 128 hidden units.
  • One layer GRU (Gated Recurrent Unit) with 128 hidden units.
  • One layer PLSTM (Phased LSTM)
  • Two layer LSTM with 256 and 128 hidden units respectively.

Here we present the implementation of the one layer LSTM model implemented with Keras:

model = Sequential()
model.add(LSTM(128, input_shape=(maxlen, len(chars))))
model.add(Dense(len(chars)))
model.add(Activation('softmax'))

Data

The datasets used for this purpose are:

References

Slides for our project can be found here

Webpage for the project is here