LSTM examples?

[jrobinson01]

Following on the other issue I created #108 , I'm trying to teach an LSTM network to write a simple children's book. I'm getting odd behavior but really don't know what I'm doing to begin with. I'd love to get this example working and added to the readme for others to follow but am hitting lots of little roadblocks. Here's my code:

const brain = require('brain.js');

const words = new Map();
words.set('00001', 'Jane');
words.set('00010', 'Spot');
words.set('00100', 'Doug');
words.set('01000', 'saw');
words.set('10000', '.');

const trainingData = [
  {input: [0,0,0,0,1], output: [0,1,0,0,0]},// Jane -> saw
  {input: [0,0,0,1,0], output: [0,1,0,0,0]},// Spot -> saw
  {input: [0,0,1,0,0], output: [0,1,0,0,0]},// Doug -> saw
  {input: [0,1,0,0,0], output: [0,0,1,0,0]},// saw -> Doug
  {input: [0,1,0,0,0], output: [0,0,0,1,0]},// saw -> Spot
  {input: [0,1,0,0,0], output: [0,0,0,0,1]},// saw -> Jane
  {input: [0,0,0,1,0], output: [1,0,0,0,0]},// Spot -> .
  {input: [0,0,0,0,1], output: [1,0,0,0,0]},// Jane -> .
  {input: [0,0,1,0,0], output: [1,0,0,0,0]},// Doug -> .
  {input: [1,0,0,0,0], output: [0,0,0,0,1]},// . -> Jane
  {input: [1,0,0,0,0], output: [0,0,0,1,0]},// . -> Spot
  {input: [1,0,0,0,0], output: [0,0,1,0,0]},// . -> Doug
];

const lstm = new brain.recurrent.LSTM();
const result = lstm.train(trainingData);
const run1 = lstm.run([0,0,0,0,1]);// Jane
const run2 = lstm.run(run1);
const run3 = lstm.run(run2);
const run4 = lstm.run(run3);

console.log(words.get('00001'));// start with 'Jane'
console.log(words.get(run1));// saw
console.log(words.get(run2));// Jane, Doug or Spot
console.log(words.get(run3));// .
console.log(words.get(run4));// Jane, Doug or Spot

console.log('run 1:', run1, typeof run1);
console.log('run 2:', run2, typeof run2);
console.log('run 3:', run3, typeof run3);
console.log('run 4:', run4, typeof run4);

The results in the console:

Jane
.
Jane
.
Jane
run 1: 10000 string
run 2: 00001 string
run 3: 10000 string
run 4: 00001 string

some observations:

• the output of run is a string. I was expecting an array

It obviously isn't working as I expected. Since I can't find a good example I really have no idea what I'm doing wrong. I'm wondering if I'm training it wrong? I'm starting to think I should be giving it example sequences as input... like {input: "Jane saw Spot", output: "."} but I can't wrap my head around how to express that as valid input.

[robertleeplummerjr]

const trainingData = [
  'Jane saw Doug.',
  'Spot saw himself.',
  'Doug saw Jane.'
];

const lstm = new brain.recurrent.LSTM();
const result = lstm.train(trainingData, { iterations: 1000 });
const run1 = lstm.run('Jane');
const run2 = lstm.run('Spot');
const run3 = lstm.run('Doug');

console.log('run 1: Jane' + run1);
console.log('run 2: Spot' + run2);
console.log('run 3: Doug' + run3);

https://jsfiddle.net/robertleeplummerjr/x3cna8rn/2/

I actually just discovered a bug, using the .toFunction doesn't output the character converter, so we'll need to fix that before v1, but the net seems to run just fine. I got:

run 1: Jane saw Doug.
run 2: Spot saw himself.
run 3: Doug saw Jane.

😎

[robertleeplummerjr]

Notice too, after each name, there are spaces. That isn't magic nor was the net specifically told to put spaces, it is just math. The net just saw them and added them after training.

[robertleeplummerjr]

Now I'm starting to have fun with this: https://jsfiddle.net/robertleeplummerjr/x3cna8rn/4/

run 1: Jane saw Doug.
run 2: Spot saw Doug and Jane looking at each other.
run 3: Doug saw Jane.

[robertleeplummerjr]

lol, this is actually addictive: https://jsfiddle.net/robertleeplummerjr/x3cna8rn/5/

run 1: Jane saw Doug.
run 2: Spot saw Doug and Jane looking at each other.
run 3: Doug saw Jane.
run 4: It was love at first sight, and Spot had a frontrow seat. It was a very special moment for all.

[robertleeplummerjr]

https://twitter.com/robertlplummer/status/947275642365730816

Now we just need a publisher...

[jrobinson01]

Amazing!! So I was completely wrong about how LSTM's expect their training data it seems? I was thinking the time series was each item in the array of training data, but it seems that it's not. In this case, it's using each character of each item?

This is really fun! https://codepen.io/anon/pen/xpdmdN?editors=1010

I'm seeing some weird behavior sometimes, where instead of text, it gives output like '19,19,19,19...19' instead of words. Not sure if that is the bug you mentioned on the other issue.

So now that I can feed it text, and think I understand how it expects text input, I'm lost as to how to feed it anything else. Suppose we wanted try to predict the weather. Our input data might look like this:

{
   date:'10-12-17',
   temperature: 45,
   humidity: 30
},
{
   date:'10-13-17',
   temperature: 40,
   humidity: 10
},

How would we convert that into something the LSTM can try to make sense of, both as training data, and then once trained, as input?

[robertleeplummerjr]

Actually you are more correct than you think. The net does provide some syntactical sugar in some scenarios. Objects have bit yet been worked out, unless you convert them to string or number.

[robertleeplummerjr]

To be clearer. things like numbers aren't so good with recurrent nets as they have been defined to me and I've seen them implemented. Part of the reason is that every character in the net is the input, so numbers are seen as more like characters (honestly more like buttons that are pushed of a sequence, that is translated to another sequence when predicting). This is borrowed mentality from recurrentjs, and is part of the reason I started working on v2. I want the ability to feed data into a recurrent net, not just "push buttons".

[robertleeplummerjr]

I found the issue related when playing around with the toFunction method here #110. Will be pushing a pr soon, locally I have it fixed.

[robertleeplummerjr]

released: https://www.npmjs.com/package/brain.js

[robertleeplummerjr]

I'd like to use this children's book example in the release of v1 on the brain.js.org homepage. Are you ok with that?

[jrobinson01]

Ok yeah, that makes sense (the pushed buttons analogy)! That's what I've been observing. The LSTM currently throws an error for example, if you try to run it with a word it's never seen before.

I'd love to see the example in the readme! That was a big motivator for this exercise, to help others make sense of it. I got the idea for the children's book here btw: https://www.youtube.com/watch?v=WCUNPb-5EYI

[robertleeplummerjr]

are you cool with using it as a demo? It is just simple enough and fun enough I feel like it'd be great as a primer into recurrent.

[jrobinson01]

Yeah go for it! I think it does a good job giving newcomers and idea how the different types of networks work and what types of projects they’re a good fit for.