Skip to content
This developed system is the implementation of my Master Thesis to visualize Recurrent Neural Networks with text prediction as a testbed. It contributes with user trust, trust for developers of model's generalization and insights how to improve neural network architecture to optimize performance.
Python
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.idea
LSTM
Vanilla RNN
.gitattributes
.gitignore
AllCharacters-Fouriermap.png
AllCharacters-Heatmap.png
AllCharacters-Terminal.PNG
Hidden_States.png
LICENSE
LSTMDots-HeatmapSeed0.png
LSTMDots-TerminalSeed0.PNG
LSTMSpaces-FouriermapSeed1Hypotheses2RelevanceDetected.png
LSTMSpaces-HeatmapSeed1Hypotheses2.png
LSTMSpaces-TerminalSeed1Hypotheses2.PNG
Output_Gate.png
README.md
package.json
test.py

README.md

Visualization of Recurrent Neural Networks

This developed system is the implementation of my Master Thesis to visualize Recurrent Neural Networks with text prediction as a testbed. It contributes with user trust, trust for developers of model's generalization and insights how to improve neural network architecture to optimize performance.

Word prediction with LSTM

Sample of terminal output

Predicted text highligted based on neural activity of the LSTM hidden state output, marked in figure below. LSTM Hidden States to be visualized

Neuron 295 and 376 are automatically hypothesized to be relevant for spaces independently through the heatmap and fourier heat surfance. They seem to have an action potential that goes from negative up to zero when triggered by the spaces. Sample of terminal output

Corresponding heatmap with relevance scores

Heatmap of extracted feature of interest " " (space) (left) and automatic suggested relevant neurons 295 and 376 through relevance heatmap (right).

Corresponding heatmap

Corresponding Fourier transform heat surface

Fourier transform of detected neuron 295 and 376 comfirmig the sensitivity to spaces by showing significant frequency components of 0.5 (every second word).

Corresponding Fourier transform heat surface

Another sample of terminal output

Predicted text highligted based on neural activity of the LSTM output gate (marked below). LSTM Output Gate to be visualized

Neuron 455 and 561 are automatically hypothesized to be significantly active (red) during sentences and get deactivated by dots and question marks.

Sample of terminal output

Corresponding heatmap with relevance scores

Heatmap of extracted feature of interest ".", "?" and "!" (left) and automatically suggested relevant neuron 455 and 561 through relevance heatmap (right).

Corresponding heatmap

Character prediction with Vanilla RNN

Sample of terminal output

Predicted text highligted based on neural activity of the LSTM hidden state output. Neuron 18 is detected to have many frequency components. The neuron seems to have a pattern to get deactivated (blue) when triggered by the second letter of synthesized words. Sample of terminal output

Corresponding heatmap with relevance scores

Heatmap extracting all features. Thus, the relevance is zero. Corresponding heatmap

Corresponding Fourier transform heat surface

Fourier transform and automatically detected peaks for neuron 18, the neuron has multiple significant frequency components.
Corresponding Fourier transform heat surface

You can’t perform that action at this time.