Generate possible results from training

[stephanbogner]

Is there a way to train the brain and generate possible results instead of letting the brain analyse a certain input?

For example:
"Give me 4 colors which match 'orange' with at least 80% accuracy".

As pseudo-code:

var net = new NeuralNetwork(); //Create neural network

net.train([{input: {r:1, g:0.65, b:0},  output: {orange: 1}}, //This is orange
           {input: {r:0, g:0.54, b:0},  output: {green: 1}}, //This is green
           {input: {r:0.6, g:1, b:0.5}, output: {green: 1}}, //This is also green
           {input: {r:0.67, g:0, b:1},  output: {purple: 1}}]); //This is purple

var output = net.run({"orange": ">0.8", "results": 4}); //return 4 colors which match 'orange' with at least 80% accuracy

Pseudo-output:

[{r:1,    g:0.65, b:0},
 {r:0.98, g:0.55, b:0},
 {r:1,    g:0.55, b:0.2},
 {r:0.85, g:0.55, b:0}]

A real world example would be:
"Tell me the first thing which comes to your mind when I say Internet". You might say something like Tim Berners-Lee or Wikipedia.

Is something like that possible with BrainJS?
Note: I also posted this question on StackOverflow.

[robertleeplummerjr]

Very cool idea! I will first state one of my favorite quotes: "anything is possible, even if it isn't". I will invest some time this weekend thinking about this, and see if I (or if anyone else wants to chime in) have a solution for you.

[robertleeplummerjr]

Snap! Just thought about what you are actually trying to do and essentially you are running the network backwards. Not only is this possible, I think it should be easy. Will see if I can get a POC later today.

Some notes:

• Probably use a more js-ly defined query, similar to mongo for example, so that it is all json and free of extra parsers
• Since this would just be in the context of js specifically, you could loose the quotes for brevity
• Since you eloquently ordered your query in english, I'll try and do the same in js.
• run seems like the wrong method name, since we are (hypothetically at this point) reversing the procedure, from input -> output, to output -> input, may I suggest a verb that describes it in english? A quick (seemingly ironic) reverse word lookup reveals "mull" may be a good find (http://www.onelook.com/?loc=rescb&refclue=think&w=mull)

Thoughts on:
var output = net.mull({ results: 4, orange: { gte: 0.8 } });

[robertleeplummerjr]

Posting research as found. To find a result, one must apparently un-sigmoid or flatten the curved output. In the source, we have a very elegant sigmoid of: (1 / (1 + Math.exp(-sum)));, of which the opposite is apparently (as found when playing around with console) -(Math.log(sum) / Math.log(Math.E));.

[robertleeplummerjr]

The es6 version of the no lib neural function is:

function run(input, net) {
  let i = 1
    , layer
    , output
    , id
    , iid
    , node
    , sum
    ;

  for (; i < net.layers.length; i++) {
    layer = net.layers[i];
    output = {};

    for (id in layer) {
      node = layer[id];
      sum = node.bias;

      for (iid in node.weights) {
        sum += node.weights[iid] * input[iid];
      }
      output[id] = (1 / (1 + Math.exp(-sum)));
    }
    input = output;
  }

  return output;
}

If my calculations are correct (and there is a good reason they are not), the inverse would look something like:

function mull(output, net) {
  let i = net.layers.length
    , layer
    , input
    , id
    , iid
    , node
    , sum
    ;

  while (i-- > 1) {
    layer = net.layers[i];
    input = {};

    for (id in layer) {
      node = layer[id];
      sum = node.bias;

      for (iid in node.weights) {
        sum -= node.weights[iid] / output[iid];
      }
      input[id] = -(Math.log(sum) / Math.log(Math.E));
    }
    output = input;
  }

  return input;
}

[robertleeplummerjr]

If a neural network could be described as a mathematical funnel, what you have devised is a mathematical explosion.

[robertleeplummerjr]

It doesn't stop there, if you can receive an input to get an output and receive an output to get an input and if you had an intermediate function put them back and forth, you would have neural reflection.

[robertleeplummerjr]

I'm really close here: https://jsfiddle.net/robertleeplummerjr/jn0n4xn4/7/

[robertleeplummerjr]

This is a very rough proof of concept: https://jsfiddle.net/robertleeplummerjr/jn0n4xn4/11/

[robertleeplummerjr]

Brain.js, what's on your mind?

[robertleeplummerjr]

I did get further with this when tinkering last night, but it occurred to me that without knowing what the inputs were, it gets a bit hairy.

[stephanbogner]

Unfortunately I'm too bad of a coder to help you with your code 😞
But I am happy that you are curious with what I wanted to achieve!
If I can help in any way, let me know.

[robertleeplummerjr]

The more I look up about this, the more interesting it gets. I found this: https://www.coursera.org/course/neuralnets where he does this very thing with his network. And the question asked on stack exchange: http://stackoverflow.com/questions/27673883/using-a-learned-artificial-neural-network-to-solve-inputs

[robertleeplummerjr]

This reply is what I'm following: http://stackoverflow.com/a/27675589

[robertleeplummerjr]

Closer proof of concept: https://jsfiddle.net/jn0n4xn4/12/

[robertleeplummerjr]

I believe I'm onto something. I now am debugging the values that come from each step of both run and mull (I really don't like that name, lol, even though it may be accurate) and comparing them from both run and mull. I noticed that the values from mull were huge! It hit me that (and I don't know why this never hit me prior) that the values that are sent through the network are sigmoid-ed.
You see, at first I was affectively un-sigmoid-ing the values (logit) and feeding that math back through, I logit the output to obtain the input, but it never occurred to me that the value that needs to be reversed back to the previous node are all sigmoid-ed.
Onwards and upwards. I will have an updated fiddle soon.