Training with Raw Image data as inputs

[mubaidr]

What is wrong?

Training error does not go below 1, and I think this is because my inputs strings have different lengths.

(currently i am using base64 encoded image data as input to the network, is this OK?)

Where does it happen?

In the a new brain.recurrent.RNN() when trying to run a net in node.js on my Windows 10

How do we replicate the issue?

Git Source here: https://github.com/mubaidr/image-parsing/blob/master/src/index.js#L45

How important is this (1-5)?

3

Expected behavior (i.e. solution)

Any suggestions to make input data to be same length for all images or maybe i am doing something wrong?

Other Comments

Thanks for the awesome work!

[jusso-dev]

I don't have an answer to this but + 1 for the gif

[mubaidr]

UPDATE:
I am now using raw data from the image (which is same size across all images) But the error is still in the range 400-500.

Also when using simple neural network error is NAN

[jusso-dev]

Neural network will never work with data that isn't numerical. What happens if you one-hot encode your input before feeding it into the network? What does your error rate look like?

[robertleeplummerjr]

You could try something like this: https://jsfiddle.net/8kpq58bu/117/
Here we encode the 256 colors like: color / 255

[mubaidr]

Neural network will never work with data that isn't numerical. What happens if you one-hot encode your input before feeding it into the network? What does your error rate look like?

Well, I tried using RAW data from the image, which is essentially an array of integer values of each pixel(gray scaled and threshold applied, so values are either 0 or 1). Still error rate fluctuates between 200-400.

You could try something like this: https://jsfiddle.net/8kpq58bu/117/ Here we encode the 256 colors like: color / 255

Thanks I will try this technique and see if it improves anything.

UPDATE:
What if I try to compress image data to some string format by representing series of pixel using Huffman style encoding? For example, if there are 31 zeros in a sequence the i replace them with 0-31 in my resulting string and similarly for 1.

e.g. For this data: 00011000000001111000001 result should be: 0-3,1-2,0-8,1-4,0-5,1-1

Sample Images I am trying to train:
https://github.com/mubaidr/image-parsing/tree/master/data/sample

Thanks everyone, I am learning new thing each time you guide me.

[mubaidr]

UPDATE:

What i am trying to do:

Ok So I have tried using following input/output sets for training so far and nothing works, training error approaches almost 1:

• Image data as base64

trainingData={input : ['base64-image-characterset-genrated-using-fs-module'],
output: ['a']}
// and
trainingData={input : ['base64-image-characterset-genrated-using-fs-module'],
output: [{option: 'a'}]}
// Above two training sample generate different training error! wot!

• Image Raw pixel data

trainingData={input : [[/*integer array of image raw data*/]], // Also tried {input : [...[/*integer array of image raw data*/]],
output: ['a']} // also tried output: [{option: 'a'}]}

• Image data as monochrome/grayscale with threshold (training samples tried same as above)

Observations

• In all cases Training error never goes below.
• I have tried using 3 types of neural networks and different activation techniques too.

Questions:

• Why does changing output format effects error rate?
• What would the best format for image data to be trained?

Source
All the source for this node.js application is available here:
https://github.com/mubaidr/image-parsing

[robertleeplummerjr]

@jusso-dev You can feed representations of data into a neural network, specifically the recurrent neural network, like the example of the children's book

Also, I see that is what is happening here, neat!

@mubaidr brain.recurrent.RNN will likely not give you good results, because it is not sophisticated enough to retain what data you are trying to send in. It is a vanilla recurrent neural network and thus suffers from exploding or disappearing gradients. (here is a link to a fantastic book I love refering too that explains it in practical terms: https://leonardoaraujosantos.gitbooks.io/artificial-inteligence/content/recurrent_neural_networks.html#long-short-term-memory-networkslstm, if you like the book, be sure to share it and star it here: https://www.gitbook.com/book/leonardoaraujosantos/artificial-inteligence/details)

There are many ways to break down this problem, but you could try and use the brain.recurrent.LSTM class, that will likely be able to help.

If you were feeling a little adventurous, and would like to help out with the new layer composition and gpu performance features, you could venture into using the branch and try out the new FeedForward class and perhaps use a convolution layer which is better at compressing the inputs into patters that can more easily be classified. More on convolutions here.

[mubaidr]

I have already tried LSTM but no result. One of my main concern here is that if my input/output format is correct in this application? @robertleeplummerjr

Thanks for book reference, I will definitely try to read and learn. Also, gonna try new branch.

[mubaidr]

UPDATE:

• NN

Preparing training data...
[ { input: [ 1, 0, 0, 0 ], output: [ 'a' ] },
  { input: [ 0, 1, 0, 0 ], output: [ 'b' ] },
  { input: [ 0, 0, 1, 0 ], output: [ 'c' ] },
  { input: [ 0, 0, 0, 1 ], output: [ 'd' ] } ]

Training started...


Training finished in 0.014s with error: NaN

• RNN

Preparing training data...
[ { input: [ 1, 0, 0, 0 ], output: [ 'a' ] },
  { input: [ 0, 1, 0, 0 ], output: [ 'b' ] },
  { input: [ 0, 0, 1, 0 ], output: [ 'c' ] },
  { input: [ 0, 0, 0, 1 ], output: [ 'd' ] } ]

Training started...


Training finished in 0.674s with error: 1.3745829649011243

• LSTM

Preparing training data...
[ { input: [ 1, 0, 0, 0 ], output: [ 'a' ] },
  { input: [ 0, 1, 0, 0 ], output: [ 'b' ] },
  { input: [ 0, 0, 1, 0 ], output: [ 'c' ] },
  { input: [ 0, 0, 0, 1 ], output: [ 'd' ] } ]

Training started...


Training finished in 2.623s with error: 1.2690826830737323

[mubaidr]

UPDATE:

Thanks everybody for the guidance, I have been able to solve this problem using simple Neural-networks and using One Dimensional data fro the image.

I issue was related to (mainly) input/output format. Now the training data sample looks like this:

[
{
input: [/*Array from monochrome Image (0s and 1s only)*/],
output: {
a:1
// OR b: 1 depending on the data
}
},
....
]

So in short, encoding my image data into simpler integer array solved this problem.

[jusso-dev]

Feel free to share the finished code

[mubaidr]

Everything is opensource under this repository, including sample images, test script: https://github.com/mubaidr/image-parsing

👍

Preparing Image data: https://github.com/mubaidr/image-parsing/blob/master/src/index.js#L30
Input to NN: https://github.com/mubaidr/image-parsing/blob/master/src/index.js#L115

[jusso-dev]

Awesome thanks 👌😉