No description, website, or topics provided.
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.

Unrolling Recurrent Neural Networks

This repository contains all the code & documents for the presentation held within Iași AI community.


The experiment

The experiment part of the project was to see whether the vanilla RNN can be trained to generate lyrics for manele. These lyrics are low quality and very simplistic.

The results

The vanilla RNN from the presentation was trained with 9 hyperparameter configurations as described in table below. Each training cycle ended at 1.000.000 iterations over a sequence of 25 characters.

Run Id Hidden size Learning rate Loss
1 100 0.1 N/A
2 100 0.01 44.355101
3 100 0.001 59.384402
4 250 0.1 48.762288
5 250 0.01 37.229775
6 250 0.001 55.494079
7 500 0.1 52.587987
8 500 0.01 27.412387
9 500 0.001 53.266636

As can be seen from results, the model got stuck in a local minimum, the best results being achieved by run 8 with hidden_size=500 and learning_rate=0.01.

If you're still curious you can browse the files with generated samples for each run: