Neurons to ensemble connections with interneurons #156
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
hunse
force-pushed
the
neuron-ens-interneuron
branch
from
1cef169 to
d8ba13e
Compare
tbekolay
force-pushed
the
neuron-ens-interneuron
branch
from
d8ba13e to
7f09e88
Compare
Member
|
Rebased this to current master. |
drasmuss
force-pushed
the
neuron-ens-interneuron
branch
from
7f09e88 to
68a1709
Compare
These connections use decode neurons and work well for typical neuron->ensemble connections that map from high-dimensional neuron space to low-dimensional ensemble space. - Added neurons->ensemble test. This test intentionally maps from high-dimensional (neuron) space to low-dimensional (ensemble) space to limit the number of decode neuron groups. Having a transform that maps to high dimensions (e.g. identity transform) does not work well, because we need an decode neuron population for each output dimension of the transform, and if each population is only receiving input from one pre neuron this is very noisy.
drasmuss
force-pushed
the
neuron-ens-interneuron
branch
from
a40226a to
b995a31
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
An alternative to #71.
In #71, I feel like I cheated, and specifically implemented neurons->ensemble connections that only have a scalar transform. In that case, the dimensionality of the post-ensemble has to be the same as the pre neurons.
In this PR, I do it in a more general way, by including interneurons in the case when we're connecting from neurons to ensemble. This allows us to put arbitrary transforms in there.
HOWEVER, I did find that for the case where we have neurons->ensemble connections of the first type, with scalar transforms, this new method does not work well. I think that's because each group of interneurons (i.e. each dimension in the interneurons population) is being driven only by one input neuron, so its input is very noisy and it just doesn't perform well. Furthermore, this would only work for small numbers of input neurons (less than 50 or so), since we use 20 interneurons to represent each dimension so for greater numbers of pre neurons we'd just run out of neurons on a single core for the interneurons.
Anyway, I feel like this is not a big limitation, since if you have the encoders for the post population, you can just do a neurons->neurons connection with the encoders as the transform to get behaviour (without interneurons) like I had implemented before. It can be a bit annoying to get the encoders, since essentially it means you have to generate them yourself at model creation time, but this isn't a huge burden.
Also, this is much simpler than #71 was, since there I had to allow for different encoders depending on whether we were connecting from interneurons (i.e. another ensemble) or neurons.