Stochastic neurons #1424

rowleya · 2023-12-19T16:06:31Z

Adds a couple of stochastic neuron models that might make WTA and Travelling Salesman problems work better

coveralls · 2023-12-19T16:20:22Z

coverage: 69.462% (-0.07%) from 69.531%
when pulling b7be66a on stochastic_neurons
into 0a9574e on master.

…hester/sPyNNaker.git into stochastic_neurons

rowleya · 2024-03-15T12:01:53Z

I think this is ready now - the models work as they are

Christian-B

Some things to address before looking at the actual changes

Christian-B · 2024-04-22T06:24:16Z

neural_modelling/makefiles/neuron/IF_curr_delta_fixed_prob/Makefile

@@ -0,0 +1,24 @@
+# Copyright (c) 2017 The University of Manchester


New but dated 2017?

Not built

Christian-B · 2024-04-22T06:24:45Z

neural_modelling/makefiles/neuron/IF_trunc_delta/Makefile

@@ -0,0 +1,24 @@
+# Copyright (c) 2014 The University of Manchester


new but dated 2014

Not built

Christian-B · 2024-04-22T06:35:33Z

spynnaker/pyNN/models/neuron/builds/if_curr_delta_fixed_prob.py

+        threshold_type = ThresholdTypeFixedProb(v_thresh, p_thresh, seed)
+        super().__init__(
+            model_name="IF_curr_delta_fixed_prob",
+            binary="IF_curr_delta_fixed_prob.aplx",


Never built!

Fixed and test added

Christian-B · 2024-04-22T06:37:02Z

spynnaker/pyNN/models/neuron/builds/if_trunc_delta.py

@@ -0,0 +1,73 @@
+# Copyright (c) 2017 The University of Manchester


1, new but dated 2017
2. Never tested

Fixed and test added

Christian-B · 2024-04-22T06:37:27Z

spynnaker/pyNN/models/neuron/builds/if_trunc_delta.py

+        threshold_type = ThresholdTypeStatic(v_thresh)
+
+        super().__init__(
+            model_name="IF_trunc_delta", binary="IF_trunc_delta.aplx",


Does not exists!

Added build and test added

Christian-B · 2024-04-22T06:38:11Z

spynnaker/pyNN/models/neuron/builds/stoc_exp.py

@@ -0,0 +1,27 @@
+# Copyright (c) 2015 The University of Manchester


Christian-B · 2024-04-22T06:40:42Z

spynnaker/pyNN/models/neuron/builds/stoc_exp_stable.py

@@ -0,0 +1,31 @@
+# Copyright (c) 2015 The University of Manchester


Christian-B · 2024-04-22T06:42:36Z

spynnaker/pyNN/models/neuron/threshold_types/threshold_type_fixed_prob.py

@@ -0,0 +1,88 @@
+# Copyright (c) 2015 The University of Manchester


Christian-B · 2024-04-22T06:42:50Z

spynnaker_integration_tests/test_various/test_stochastic_models.py

@@ -0,0 +1,73 @@
+# Copyright (c) 2017 The University of Manchester


Christian-B

some more comments.
Especially double recording yet only sdram space for 1
possible double spike

Christian-B · 2024-04-22T08:29:10Z

neural_modelling/src/common/maths-util.h

+	return ukbits(accumulator);
+}
+
+static const uint64_t MAX_REAL = 0xFFFFFFFF0000L;


Christian-B · 2024-04-22T08:29:20Z

neural_modelling/src/common/maths-util.h

+
+static const uint64_t MAX_REAL = 0xFFFFFFFF0000L;
+
+static const uint64_t MIN_REAL = 0x000000010000L;


also unused

Christian-B · 2024-04-22T08:41:12Z

neural_modelling/src/neuron/implementations/neuron_impl_stoc_exp.h

+SOMETIMES_UNUSED // Marked unused as only used sometimes
+static bool neuron_impl_initialise(uint32_t n_neurons) {
+    // Allocate DTCM for neuron array
+    if (sizeof(neuron_impl_t) != 0) {


why do we need this check?
neuron_impl_t is defined locally so will never have size 0

Christian-B · 2024-04-22T09:24:40Z

neural_modelling/src/neuron/implementations/neuron_impl_stoc_exp.h

+//! Array of neuron states
+static neuron_impl_t *neuron_array;
+
+SOMETIMES_UNUSED // Marked unused as only used sometimes


Unusual having all these SOMETIMES_UNUSED in an h file only included in one make file

Removed from this file in general

Christian-B · 2024-04-22T10:14:00Z

neural_modelling/src/neuron/implementations/neuron_impl_stoc_exp.h

+
+			// Record things
+			neuron_recording_record_int32(PROB_INDEX, neuron_index, 0);
+			neuron_recording_record_accum(V_RECORDING_INDEX, neuron_index, ZERO);


Does it make sense to record these with the same index and time before and after spike?
Or better to use a different index

It doesn't record twice - there was a continue but now updated to use else, and separated code into separate functions

Christian-B · 2024-04-22T10:17:32Z

neural_modelling/src/neuron/implementations/neuron_impl_stoc_exp.h

+		uint32_t random = mars_kiss64_seed(neuron->random_seed);
+
+		// If the random number is less than the probability value, spike
+		if (random < prob) {


This looks like a neuron could spike twice in one timestep.

Christian-B · 2024-04-22T10:18:43Z

neural_modelling/src/neuron/implementations/neuron_impl_stoc_exp_stable.h

+}
+
+SOMETIMES_UNUSED // Marked unused as only used sometimes
+static void neuron_impl_do_timestep_update(


same comments about index used twice, sdram and possible double spike

Christian-B · 2024-04-22T10:19:32Z

neural_modelling/src/neuron/implementations/neuron_impl_stoc_sigma.h

+}
+
+SOMETIMES_UNUSED // Marked unused as only used sometimes
+static void neuron_impl_do_timestep_update(


again indexes reused only sdram for 1 recording and two chances to spike

Christian-B · 2024-04-22T10:37:35Z

neural_modelling/src/neuron/implementations/neuron_impl_stoc_exp.h

+        REAL v_membrane = (neuron->bias + neuron->inputs[0]) - neuron->inputs[1];
+
+        // Record things
+        neuron_recording_record_accum(V_RECORDING_INDEX, neuron_index, v_membrane);


some timesteps will record twice.
yet neuron recorder sampling rates are 1
this will cause you to run out of sdram on a long run!

…hester/sPyNNaker.git into stochastic_neurons

Christian-B

looks ok pending Jenkins being green

rowleya · 2024-04-23T06:39:09Z

Jenkins is green so merging!

rowleya added 4 commits December 15, 2023 12:51

Add stochastic neuron model

e186ae8

Rats

cbd2bda

Add sigma and fix maths

1578246

Make these too

9be9612

Christian-B and others added 25 commits January 5, 2024 08:13

Merge branch 'master' into stochastic_neurons

e67c94d

Fixes

85aff99

Merge branch 'stochastic_neurons' of https://github.com/SpiNNakerManc…

4daaa82

…hester/sPyNNaker.git into stochastic_neurons

Bias is negative!

6f19925

Remove log

fba7b2a

Make sure maths is right

f9d76d8

Work with division without same degree of overflow as separate

d8396a8

Remove empty lines

07520d0

Add IF trunc delta model (not stochastic)

e3302e9

Merge branch 'master' into stochastic_neurons

5ad0d3f

Add types

df3f701

More type fixes

7a90ee8

More type fixes

be16c78

Add "stable" version of stochastic neuron

220d46a

Avoid neuron being stuck low

3670372

Oops

c3166cd

Add a fixed probability threshold model

48a277e

Fixes to allow compilation to work again

ce555bf

Also include the header!

debaf90

Add TSP evaluator vertex

c49c1fb

Allow TSP

139a079

Change name of key and function

01a6467

Add poisson control option

11e75d9

Get Poisson config

e531be6

Fix import

617324b

rowleya added 5 commits March 14, 2024 16:33

Adding poisson rates and times to config

7204ad6

Remove this code from here as it is really part of TSP

97059b1

Merge branch 'master' into stochastic_neurons

2d3c1e7

pylint

71fd6b7

Add a test of the new models

6ed7995

rowleya marked this pull request as ready for review March 15, 2024 11:11

rowleya requested a review from Christian-B March 15, 2024 11:11

Merge branch 'master' into stochastic_neurons

1ef5909

Christian-B requested changes Apr 22, 2024

View reviewed changes

Christian-B and others added 4 commits April 22, 2024 14:51

Merge branch 'master' into stochastic_neurons

0482ccd

Fix dates and add more tests

57d3db7

Merge branch 'stochastic_neurons' of https://github.com/SpiNNakerManc…

bf4b97e

…hester/sPyNNaker.git into stochastic_neurons

Improve the code

b7be66a

Christian-B approved these changes Apr 22, 2024

View reviewed changes

rowleya merged commit df7ce07 into master Apr 23, 2024
10 checks passed

rowleya deleted the stochastic_neurons branch April 23, 2024 06:40

		@@ -0,0 +1,24 @@
		# Copyright (c) 2017 The University of Manchester

		@@ -0,0 +1,24 @@
		# Copyright (c) 2014 The University of Manchester

		@@ -0,0 +1,73 @@
		# Copyright (c) 2017 The University of Manchester

		@@ -0,0 +1,27 @@
		# Copyright (c) 2015 The University of Manchester

		@@ -0,0 +1,31 @@
		# Copyright (c) 2015 The University of Manchester

		@@ -0,0 +1,88 @@
		# Copyright (c) 2015 The University of Manchester


		static const uint64_t MAX_REAL = 0xFFFFFFFF0000L;

		static const uint64_t MIN_REAL = 0x000000010000L;

Stochastic neurons #1424

Stochastic neurons #1424

Conversation

rowleya commented Dec 19, 2023

coveralls commented Dec 19, 2023 • edited Loading

rowleya commented Mar 15, 2024

Christian-B left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Christian-B left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Christian-B left a comment

Choose a reason for hiding this comment

rowleya commented Apr 23, 2024

coveralls commented Dec 19, 2023 •

edited

Loading