Update Train_LM with minimum error step criterium

[mc10011]

The proposed changes help in two ways:

1. Prevent training from running forever when error improves slightly
2. Stop a training early before k_max is reached if error doesn't improve any more

[yabata]

Thanks for your contribtion. Nice approach!
But would we not want to reset "early" to zero when a step is successfull?

[yabata]

I think yo can write this with just one "if verbose" statement

[mc10011]

I agree.

[mc10011]

My approach assumes that if a step doesn't improve the Error at least by the min_E_step, then the step is not successful "line 734 if Enew<E and abs(E-Enew)>=min_E_step: ". This way we have the chance to re-adjust the learning rate and get a successful step right after a min_E_step step.

[yabata]

My approach assumes that if a step doesn't improve the Error at least by the min_E_step, then the step is not successful "line 734 if Enew<E and abs(E-Enew)>=min_E_step: ". This way we have the chance to re-adjust the learning rate and get a successful step right after a min_E_step step.

Sure, but should the counter "early" not be reset to 0 if there is a successfull step again? Such that it only brekas if this happens 5 times in a row.

[mc10011]

Changes done.

PS: Since I introduced this early criterium I have never encountered the "infinite-training" problem again

[yabata]

Thank you for your contribution :)

[mc10011]

Hello Dennis, thank you for accepting my request! Regarding my application, I am using your toolbox for my master thesis in Marine engineering with the target to simulate an internal combustion engine as an RNN and compare it with its linear and first principle thermodynamic MATLAB/Simulink models. Eventually, if all work out well, we plan to add the RNN model as the reference for a Controls strategy. Thank you for your work and for making it available online. Best regards, Pantelis

…

On Mon, 22 Apr 2019 at 20:47, Dennis Atabay ***@***.***> wrote: Thank you for your contribution :) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#8 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGHAD6KNDHUWY2D5FFGB6DDPRX2ZVANCNFSM4GYGSMUA> .

[yabata]

Hi Pantelis, Very interesting topic! I hope it works out! Good luck with your thesis. Cheers Dennis Von: mc10011 <notifications@github.com> Gesendet: Dienstag, 23. April 2019 08:46 An: yabata/pyrenn <pyrenn@noreply.github.com> Cc: Dennis Atabay <dennis.atabay@gmail.com>; State change <state_change@noreply.github.com> Betreff: Re: [yabata/pyrenn] Update Train_LM with minimum error step criterium (#8) Hello Dennis, thank you for accepting my request! Regarding my application, I am using your toolbox for my master thesis in Marine engineering with the target to simulate an internal combustion engine as an RNN and compare it with its linear and first principle thermodynamic MATLAB/Simulink models. Eventually, if all work out well, we plan to add the RNN model as the reference for a Controls strategy. Thank you for your work and for making it available online. Best regards, Pantelis

On Mon, 22 Apr 2019 at 20:47, Dennis Atabay ***@***.***> wrote: Thank you for your contribution :) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#8 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGHAD6KNDHUWY2D5FFGB6DDPRX2ZVANCNFSM4GYGSMUA> .

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#8 (comment)> , or mute the thread <https://github.com/notifications/unsubscribe-auth/ACGZMJR4OYXXKJLDQEA24NLPR2WB3ANCNFSM4GYGSMUA> .