Share your artificial intelligence source code example with everyone

[joaopauloschuler]

Have you got cool examples? Feel free to pull and share your example with everyone!!!

[maxkleiner]

Hi I
I just tested the supersimple.lpr and compared the results, after 3000 epochs each I got very different results, a few are in the expected range others made no sense for me:
XOR AND OR
3000 x 0 Output: 0.10 0.10 0.10
3000 x 1 Output: 0.80 0.10 0.80
3000 x 2 Output: 0.80 0.10 0.80
3000 x 3 Output: 0.10 0.80 0.80
others very different
3000 x 0 Output: 0.37 0.16 0.56
3000 x 1 Output: 0.77 0.10 0.82
3000 x 2 Output: 0.59 0.00 0.55
3000 x 3 Output: 0.11 0.72 0.57

[joaopauloschuler]

Hello @maxkleiner, your experiment is interesting and I'm glad that you did it.

If you assume that:
0.1 = False.
0.8 = True.
(0.1+0.8) = 0.45 = Threshold.
y < 0.45 = False.
y > 0.45 = True.

With above assumptions, you'll find that your "others very different" results are precise results in boolean terms.

This API implements Stochastic Gradient Descent. As the name implies, it's not deterministic. Most probably, if you increment the number of epochs, results will look more stable.

Please feel free to bring to my attention anything that you believe isn't correct or sufficient in this reply.

[maxkleiner]

Another Example is the //Application.Title:='CIFAR-10 SELU Classification Example';
It never gets better score than at the beginning, so the loss doesnt really change till epoch 50, what could be wrong(config at bottom)?:
epoch training accuracy training loss training error validation accuracy validation loss
1 0.0955 2.3005 1.7996 0.0977 2.3028
2 0.0924 2.3026 1.8 0.0977 2.3028
3 0.0919 2.3027 1.8 0.0977 2.3028
4 0.0981 2.3063 1.8007 0.098 2.3027

47 0.0938 2.3036 1.8002 0.0977
48 0.0997 2.3018 1.7998 0.0977
49 0.0903 2.303 1.8001 0.0977
50 0.0995 2.3036 1.8002 0.0977

NeuralFit:= TNeuralImageFit.Create;
NeuralFit.FileNameBase:= 'ImageClassifierSELU_Tutor89_5';
NeuralFit.InitialLearningRate:= 0.0004;
// SELU seems to work better with smaller learning rates.
NeuralFit.LearningRateDecay:= 0.03;
NeuralFit.StaircaseEpochs:= 10;
NeuralFit.Inertia:= 0.9;
NeuralFit.L2Decay:= 0.00001;
NeuralFit.verbose:= true;

[maxkleiner]

Hello @maxkleiner, your experiment is interesting and I'm glad that you did it.

If you assume that: 0.1 = False. 0.8 = True. (0.1+0.8) = 0.45 = Threshold. y < 0.45 = False. y > 0.45 = True.

With above assumptions, you'll find that your "others very different" results are precise results in boolean terms.

This API implements Stochastic Gradient Descent. As the name implies, it's not deterministic. Most probably, if you increment the number of epochs, results will look more stable.

Please feel free to bring to my attention anything that you believe isn't correct or sufficient in this reply.

Thanks for the answer, you know that Im experimenting with a script in maXbox, in comparison with FPC could be the answer, Il'l keep you posted, thanks for your great work!

[maxkleiner]

Hi Joao Theres the article in the 100th. Edition of BPM of CAI: http://xv9li.mjt.lu/nl2/xv9li/sh08.html?m=AUsAAED2zaoAAAAOHLYAAARCM1IAAAAAbUYAAN_4ABNaLgBhiOAa2YEQ7HvnSZeVXlPfMoEYzQATSVs&b=75405464&e=f6f8de75&x=H1TrR12BUVYkwY-A0LB_Hw Greetings, Max

--- http://www.softwareschule.ch/ https://maxbox.codeplex.com/ *******************************************

On 2021-11-06 12:31, joaopauloschuler wrote: Hello @maxkleiner [1], your experiment is interesting and I'm glad that you did it. If you assume that: 0.1 = False. 0.8 = True. (0.1+0.8) = 0.45 = Threshold. y < 0.45 = False. y > 0.45 = True. With above assumptions, you'll find that your "others very different" results are precise results in boolean terms. This API implements Stochastic Gradient Descent. As the name implies, it's not deterministic. Most probably, if you increment the number of epochs, results will look more stable. Please feel free to bring to my attention anything that you believe isn't correct or sufficient in this reply. -- You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub [2], or unsubscribe [3]. Triage notifications on the go with GitHub Mobile for iOS [4] or Android [5].

Links: ------ [1] https://github.com/maxkleiner [2] #8 (comment) [3] https://github.com/notifications/unsubscribe-auth/AAZ4MYPDWERSSPTJGTKWNH3UKUGYJANCNFSM4I7SFKQA [4] https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&amp;mt=8&amp;pt=524675 [5] https://play.google.com/store/apps/details?id=com.github.android&amp;referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub

[joaopauloschuler]

@maxkleiner, congrats for your 100th edition!

I'll test "Another Example is the //Application.Title:='CIFAR-10 SELU Classification Example';" in other environments than I usually test and let you know. I missed this bug report.

[joaopauloschuler]

From where should I start to test it with maXbox?

[maxkleiner]

Hi Joao Start with the download and just call the exe: maXbox4.exe https://sourceforge.net/projects/maxbox/files/latest/download then you load the script and press F9: https://sourceforge.net/projects/maxbox/files/Examples/CAI/1065__CAI_2_SimpleImageClassifier21_EKON_blogstarter.txt/download takes a lot more time than a compiler, runs only with 5 epochs gives about 62% acc. Greetings, Max

--- http://www.softwareschule.ch/ https://maxbox.codeplex.com/ *******************************************

On 2021-11-13 21:16, joaopauloschuler wrote: From where should I start to test it with maXbox? -- You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub [1], or unsubscribe [2]. Triage notifications on the go with GitHub Mobile for iOS [3] or Android [4].

Links: ------ [1] #8 (comment) [2] https://github.com/notifications/unsubscribe-auth/AAZ4MYJCI5NHLVEMTGPYVH3UL3BQVANCNFSM4I7SFKQA [3] https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&amp;mt=8&amp;pt=524675 [4] https://play.google.com/store/apps/details?id=com.github.android&amp;referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub

[maxkleiner]

Works fine now, did also test MNIST Benchmark, 7 times slower in training but thats scripting. Greetings, Max

--- http://www.softwareschule.ch/ https://maxbox.codeplex.com/ *******************************************

On 2021-11-06 12:31, joaopauloschuler wrote: Hello @maxkleiner [1], your experiment is interesting and I'm glad that you did it. If you assume that: 0.1 = False. 0.8 = True. (0.1+0.8) = 0.45 = Threshold. y < 0.45 = False. y > 0.45 = True. With above assumptions, you'll find that your "others very different" results are precise results in boolean terms. This API implements Stochastic Gradient Descent. As the name implies, it's not deterministic. Most probably, if you increment the number of epochs, results will look more stable. Please feel free to bring to my attention anything that you believe isn't correct or sufficient in this reply. -- You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub [2], or unsubscribe [3]. Triage notifications on the go with GitHub Mobile for iOS [4] or Android [5].

Links: ------ [1] https://github.com/maxkleiner [2] #8 (comment) [3] https://github.com/notifications/unsubscribe-auth/AAZ4MYPDWERSSPTJGTKWNH3UKUGYJANCNFSM4I7SFKQA [4] https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&amp;mt=8&amp;pt=524675 [5] https://play.google.com/store/apps/details?id=com.github.android&amp;referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub