Cross validation

[agamm]

I hope you like it!

Notes & clarifications,

• I used your idea of merging this feature directly into the Trainer.train object, in order to do that I craeted _trainSet so the code will be more DRY.

• I created two new tests that try to learn the sin function so I could test the cross validation - if you have a better idea for a test I would be glad to hear it.

• I noticed a few unit tests that were inconsistent (probable race conditions):

  • I got this one every 3rd test in average (looks like it happens when the tests take too long to run):

  [03:00:38] TypeError in plugin 'gulp-mocha'
  Message:
  Cannot read property 'input' of undefined
  Stack:
  TypeError: Cannot read property 'input' of undefined
  at getSamples (D:\Development\synaptic\src\trainer.js:643:21)
  at Object.Trainer.timingTask (D:\Development\synaptic\src\trainer.js:668:19)
  at Suite.<anonymous> (D:\Development\synaptic\test\synaptic.js:385:32)
  at context.describe.context.context (D:\Development\synaptic\node_modules\mocha\lib\interfaces\bdd.js:47:10)
  at Object.<anonymous> (D:\Development\synaptic\test\synaptic.js:383:1)
  at Module._compile (module.js:456:26)
  at Object.Module._extensions..js (module.js:474:10)
  at Module.load (module.js:356:32)
  at Function.Module._load (module.js:312:12)
  at Module.require (module.js:364:17)
  at require (module.js:380:17)
  [03:00:38] Finished 'min' after 6.95 s
  

  • I got these weird failed tests about 2 times in all of the times I ran gulp (probably once in 50 runs):

  Optimized and Unoptimized Networks Equivalency
    1) should produce the same output for both networks after 100 iterations
    2) should produce the same output for both networks after 200 iterations
    3) should produce the same output for both networks after 300 iterations
    4) should produce the same output for both networks after 400 iterations
    5) should produce the same output for both networks after 500 iterations
    6) should produce the same output for both networks after 600 iterations
    7) should produce the same output for both networks after 700 iterations
    8) should produce the same output for both networks after 800 iterations
    9) should produce the same output for both networks after 900 iterations
    10) should produce the same output for both networks after 1000 iterations
  

[cazala]

Looks good, I'm merging it (: about the Cannot read property 'input' of undefined, yea I noticed it before, I don't know yet what's causing it but it's something related to the Timing Task.. I have to set some time aside to check what's wrong with it, but it's difficult to replicate cause as you said, it doesn't happen every time.

[cazala]

just one thing, as soon as I merged, one of the SIN tests on the CI failed, I re-ran it and it passed, so maybe we should train the network a lil longer to make sure it will pass all the tests, or change the desired error

[agamm]

Yay! I am happy you found it useful.

SIN -Tests -> do you mind if it makes the whole tests run longer? I tried to minimize it because I personally hate when tests take too long to complete. I'll try to edit it to make it more consistent.

BTW, I just noticed that there is a bug in the code, the options I declared make it so you will always have cross validation on - should I add a fix in the PR for the SIN test?
I don't mind if you fix it yourself so no one will have a buggy version:
In src/trainer.js line 15~ change to:

this.crossValidate = options.crossValidate || null;

Instead of:

this.crossValidate = {
    testSize: .3,
    testError: .01
}

[lucasBertola]

Like I writed in #75 , i did a pull Request for fix the

I got this one every 3rd test in average