N2D2 versions conflict

[nassim-abderrahmane]

Il y a une anomalie avec N2D2 car j'obtiens des résultats différents avec deux versions de N2D2.

Version actuelle "21/03/2018": accuracy = 77.92 %
Version antérieure "16/03/2017": accuracy = 95.37 %

Ce problème est présent seulement pour les réseaux à spikes.

Je utilise la commande "n2d2 mnist28_300_10_Spike.ini -test" pour les deux versions.

[olivierbichler-cea]

Bonjour,
Pouvez-vous nous joindre le fichier mnist28_300_10_Spike.ini qui pose problème ?

[nassim-abderrahmane]

Bonjour, Vous trouverez ci-joint le fichier mnist28_300_10_Spike.ini . Merci! Nassim Le 2018-03-22 17:39, Olivier a écrit :

Bonjour, Pouvez-vous nous joindre le fichier mnist28_300_10_Spike.ini qui pose problème ? -- You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub [1], or mute the thread [2]. * Links: ------ [1] #15 (comment) [2] https://github.com/notifications/unsubscribe-auth/AjIvMzFKAgbvWp2Em60SQ8pHMiRuWiQSks5tg9PPgaJpZM4S0-sO

; Lyes Khacef ; N2D2 Spiking MultiLayer Perceptron ; LEAT - 31.05.2017 ; This network reaches 95.37% recognition rate on MNIST test dataset ; This file runs the network in both formal and spike coding on the MNIST test dataset ; Command: ; n2d2 "mnist28_300_10_Spike.ini" -learn 600000 -log 60000 DefaultModel=Transcode_CUDA CheckWeightRange=0 ; Database [database] Type=MNIST_IDX_Database Validation=0.2 ; Environment [env] SizeX=28 SizeY=28 BatchSize=1 ConfigSection=env.config [env.config] ; Spike-based computing StimulusType=Poissonian PeriodMin=1,000,000 PeriodMeanMin=10,000,000 PeriodMeanMax=100,000,000,000 PeriodRelStdDev=0.0 ; First layer (fully connected) [fc1] Input=env Type=Fc NbOutputs=300 ActivationFunction=Rectifier WeightsFiller=XavierFiller ConfigSection=common.config ; Second layer (fully connected) [fc2] Input=fc1 Type=Fc NbOutputs=10 ActivationFunction=Rectifier WeightsFiller=XavierFiller ConfigSection=common.config,fc2.config [fc2.Target] [fc2.config] ; Spike-based computing TerminateDelta=4 BipolarThreshold=1 ;BipolarThreshold=1 ??? ; Common config for static model [common.config] NoBias=0 WeightsSolver.LearningRate=0.01 WeightsSolver.Momentum=0.0 WeightsSolver.Decay=0.0 Solvers.LearningRatePolicy=None Solvers.LearningRateStepSize=1 Solvers.LearningRateDecay=0.1 ; Spike-based computing Threshold=1.0 BipolarThreshold=0 IncomingDelay=0

[olivierbichler-cea]

J'arrive à reproduire le problème.
Nous allons investiguer !

[olivierbichler-cea]

The problem arises because of weights normalization that is done after the test in frame mode, which was likely changed between the two versions of N2D2.
The following lines are executed at the end of a test:

deepNet->normalizeFreeParameters();
deepNet->exportNetworkFreeParameters("weights_normalized");

deepNet->normalizeOutputsRange(outputsRange, 0.25);
deepNet->exportNetworkFreeParameters("weights_range_normalized");

Then, when the spike test follows, the weights that are used are the one normalized by the second normalize command.
Long story short, to obtain the same results than for the previous N2D2 version, just re-run N2D2 without the "-test" argument after the learning.
You can also load specific weights for the test with the "-w <weights_folder>" argument.

Now, the rational behind the automatic normalization after the test in frame mode is that the outputs range normalization should improve the spike coding. In your case, it is not and I suspect that this is just luck, but on bigger networks the score would likely fall apart...
There are indeed several issues with your INI simulation:

• You are using ReLU activation on the output layer. The chances are high that this may cause some neurons to become silent. You should be using a softmax (with loss) during the learning and drop it for the test. In all cases, the output layer should not have an activation function (or Linear activation).
• Bias it not supported in spike mode, so you should be using NoBias=1.

Considering that, I achieve for the same network a test score of 98.20% in frame mode and 97.98% in spike mode using the "-w weights_range_normalized" weights and a terminate delta of 20.

[olivierbichler-cea]

The problem appears to be solved, closing the issue.
Please don't hesitate to reopen it if needed!