Interneuron improvements - #132
Conversation
|
I don't want this to derail this PR, but I did want to note that the first time someone mentioned "interneurons" to me I was somewhat confused because the fact that they are used for NEF decoding wasn't clear in the name, so I assumed that they were inhibitory interneurons as we've used in some other contexts (e.g. respecting Dale's principle). Yes I know that not all biological interneurons are inhibitory, but I also haven't seen the term interneuron used before in the sense of decoding values. Since this PR adds several interneuron-related elements to the API, it might be a good time to explicitly decide if we're ok with the name interneuron or if we can come up with something more clear. A few alternative names come to mind:
My vote would be for decode neurons as it's more clear what their purpose is (though obviously it's never going to be perfectly clear with one word), and isn't a term already used in neuroscience. |
+1 to that suggestion! |
tbekolay
left a comment
tbekolay
left a comment
There was a problem hiding this comment.
Made a bunch of inline comments, mostly pretty nitpicky, but some have more substance. It's a fair bit, so I'd be happy to make the changes as it's fresh in my mind if you have other things you're working on @hunse.
Also, we pretty much have consensus on switching to "decode neurons" though we should make sure there are no dissenting opinions, so I would do that in this PR, and I think we should remove the #148 and #151 commits to see if it causes test failures (and prioritize them if it does).
| self._inter_rate = None | ||
|
|
||
| # number of inter neurons | ||
| self.n = n |
There was a problem hiding this comment.
Personally I would prefer this be n_neurons, it's impossible to grep through files for n if I want to know where this is used. Also means you wouldn't need an explanatory comment, as it would be clear from the name.
There was a problem hiding this comment.
So n isn't actually the total number of neurons. It's the number of neurons per dimension per on/off pair (at least in all the interneuron types we have so far). This is because all our interneurons use a system of on/off pairs and separate interneurons for each dimension. Theoretically, you could have interneurons where it's just one big group of neurons, and each responds to multiple dimensions (just like an ensemble with multiple dimensions).
So we could call it neurons_per_dimension, but even that wouldn't be quite accurate because the actual neurons per dimension would be twice that (because we always have on/off pairs).
There was a problem hiding this comment.
Maybe n_pairs or pairs_per_dimension? I'm not okay with the one-character n.
There was a problem hiding this comment.
I'd also be okay with size I think
|
|
||
| def build_interencoders(model, ens): | ||
| """Build encoders accepting on/off interneuron input.""" | ||
| def build_inter_encoders(model, ens, kind='inter_encoders'): |
There was a problem hiding this comment.
Why not pass in the Interneurons instance here rather than a string? With the class hierarchy, it should be fine to pass in any instance.
There was a problem hiding this comment.
Because the string also gets used for naming the synapse, which is then used to find the synapse by name when we're making connections.
| target = sim.data[p_stim_a] + sim.data[p_stim_b] | ||
|
|
||
| error = np.abs(sim.data[p_c][tmask] - target[tmask]).mean() | ||
| print(error) |
There was a problem hiding this comment.
I'm guessing this was here to get good tolerance values initially? I would probably remove it in lieu of maybe something like plt.title("error=%g" % error)
There was a problem hiding this comment.
Partly to get good initial values, but also because this is somewhat a benchmark in addition to a test. I use this test to make changes to different interneuron classes and see how it affects the performance.
My plan was to use the logger fixture once the PR where I add that gets merged, but I still have some problems with the logger (i.e. it would be nice to be able to just print everything that it logs to the screen, with a command line option like --log-to-screen).
There was a problem hiding this comment.
Yeah the plan is to get rid of logger because pytest has its own logging stuff that works better than what we had (minus logging to individual files, but we rarely used that).
| nengo.Connection(b, c) | ||
|
|
||
| out_synapse = nengo.Alpha(0.03) | ||
| stim_synapse = out_synapse.combine( |
There was a problem hiding this comment.
This feels weird. Why is stim_synapse like this?
There was a problem hiding this comment.
Each time there's a connection in the network, there's two Lowpass(0.005) filters applied (one from the ensemble and one from the interneurons). Two identical lowpass filters make an alpha filter, and I combine one of them for each connection. This replicates the filtering and delay that happens in the network, so that the stim signal and recorded output signal are comparable.
|
Ok, I've made almost all the requested changes (except one, where removing a function precluded doing it). Two things left to do are make the name change to DecoderNeurons, and test this on the chip (some of the tolerances on the new tests might have to be adjusted). |
|
There are two changes to lines that aren't covered currently. One is in the builder function for probes: The other is a |
|
I pushed a fixup (8acfa65) to change the class hierarchy around a little bit (simplified to remove |
|
The problem with basing the I think it would be possible to re-write all of them to use the same |
|
Ok, it was actually pretty easy to have them use the same |
|
I've fixed up I'm happy with this, now! |
Ditto, the classes look great. I'll get started testing the diff lines not yet covered. |
|
Added in 7d8e879. Will rebase once all CI passes and merge if there are no issues in the rebase. |
Sure, I'll change to |
0f71682 to
06d7b96
Compare
|
The rebase was not difficult, but did require some changes, and I wrote a changelog entry and a consolidated commit message. Will wait for @hunse to sign off on the changelog entry and commit message before merging. |
|
The changelog entry looks good. The only error in the commit message is that NoisyDecodeNeurons was actually the default behaviour before this commit for Ensemble->Ensemble connections. The OnOffDecodeNeurons are only used to generate input spikes, both before and after this commit. |
|
I added a docstring for |
|
There's one test that I've found failing on the chip so far, which is What's actually asserted, though, is that the neurons with positive encoders only fire for the positive part of the sine wave, and neurons with negative encoders only fire for the negative part. It seems like in this branch, on the chip, there are two stray spikes that happen in positive-encoder neurons during the negative part of the sine wave. Otherwise, the behaviour is identical between emulator and chip, and exactly as expected. Really, I don't think this is a big cause for concern, and we could just have slightly looser tolerances on the test, since everything is behaving pretty-much as it should. But, I am annoyed that the emulator is not matching the chip perfectly here, and I'd like to look a bit more into that. |
|
Ok, so what seems to be happening here is that when we do spike probes with If I run the test with So this is definitely unrelated to this PR, and I don't think we should try to fix it here. I would suggest adjusting the tolerances of the test to allow these couple of stray spikes, and make an issue for this. Or we could keep the test the same and just switch to |
|
I made issue #170 to track this problem, and changed the test to use |
|
I ran all the I also ran the |
I think if you install from pip, you don't get the examples, so that might explain why it has fewer tests. 70 passing tests sounds about right. All else sounds good, I'll do a quick double check of this then merge. |
The biggest change is to be able to customize how many decode neurons are used and how they are organized. Additionally, the name "interneurons" was changed to "decode neurons" to avoid confusion with the concept of inhibitory interneurons in neuroscience. This commit also removes sharing encoders between node inputs and ensemble inputs. This greatly simplifies things, because these no longer have to be identical. Several decode neuron implementations are provided: - OnOffDecodeNeurons uses identical pairs of on/off neurons for each dimension. This is the default for generating input spikes (for nodes). - NoisyDecodeNeurons uses pairs of on/off neurons for each dimension with injected noise so that each on/off pair is different. This was the default behavior prior to this commit for decoded connections between two ensembles. - Preset5DecodeNeurons uses five heterogeneous on/off pairs with pre-set values for each dimension. - Preset10DecodeNeurons uses ten heterogeneous on/off pairs with pre-set values for each dimension. This is now the default behavior for decoded connections between two ensembles. This commit uncovered a bug in test_communication.test_neurons2node in which a magic number on the chip was being treated as a spike. For now, we set `precompute=True` to avoid the bug. Partially addresses #129.
4 participants
This helps to address #129.
Rather than using homogeneous interneurons with noise, this uses heterogeneous interneurons. However, they are designed so that they all use the same decoder, which means that the size of encoders in target populations can remain small (otherwise it would scale with the number of interneurons).
Based off #124.