Added amplitude to LIF

[hunse]

Motivation and context:

Allow scaling on the output of LIF neurons. This can be useful when working with learning, to allow the output of the neuron to be in a more normalized range.

How has this been tested?

I added a unit test that checks that the scaled output of a normal neuron (amplitude == 1) matches that that of the neuron with amplitude set, both for the static rates and the dynamic spikes.

How long should this take to review?

• Quick (less than 40 lines changed or changes are straightforward)

Types of changes:

• New feature (non-breaking change which adds functionality)

Checklist:

• I have read the CONTRIBUTING.rst document.
• I have updated the documentation accordingly.
• I have included a changelog entry.
• I have added tests to cover my changes.
• All new and existing tests passed.

[drasmuss]

I usually do this by changing max_rates instead. For example, instead of doing

ens = nengo.Ensemble(1, 1, max_rates=[100], neuron_type=nengo.LIFRate(amplitude=0.1))

I'd do

ens = nengo.Ensemble(1, 1, max_rates=[10], neuron_type=nengo.LIFRate())

Obviously not exactly the same (the former is scaling the output of the nonlinearity, the latter is scaling the input), but gets you a similar result.

Another option is to incorporate the amplitude into the connection weights on any efferent connections (e.g., transform=amplitude), or the gains of postsynaptic neurons (these would be exactly the same, but slightly more complicated).

So even though I would actually find the amplitude parameter helpful for training/learning, as you point out, I am kind of inclined against it because

1. It is somewhat redundant with existing functionality
2. It adds an extra parameter
3. It kind of muddies the waters of max_rates (would we say that nengo.Ensemble(1, 1, max_rates=[100], neuron_type=nengo.LIFRate(amplitude=0.1)) has a maximum firing rate of 100Hz, or 10Hz?)

That being said, I'm not strongly opposed, so if other people like it I'm totally fine with adding amplitude in.

[arvoelke]

Could this be made a parameter / feature of the learning rule(s) instead?

[hunse]

The problem with max_rates is that it actually changes the firing rates. This affects how the network operates. In my networks, I want the neurons to be firing at ~100 Hz, to help minimize the noise caused by spikes.

Incorporating amplitude into postsynaptic connection weights would certainly be the "normal" way to do it. In fact, if I wanted to describe my model to anybody either in neuroscience or machine learning, I would tell them the amplitude just gets folded in to the efferent connection weights, such that my model has the same structure as traditional models. Really the reason that it helps me is that I can scale my neurons to be more like sigmoids or ReLUs, so if I train a network using one of those neuron types with backprop, then it's easy for me to swap in a LIF neuron type instead and still have things work with similar learning rates and initial weight magnitudes.

This is also kind of the reason that I wouldn't want to have it be part of the learning rule, because whether you want this scaling or not really depends on the neuron model that you're using.

[jgosmann]

How does this affect different backends?

[hunse]

How does this affect different backends?

Like pretty much anything in the Nengo API, they could decide whether to support it or not. For the most part, I don't think it should be too hard to support, because in essence it can be folded into the connection weights when you're actually computing it (so all the computations involved would be things current hardware can do).

[jgosmann]

Some minor things, otherwise LGTM.

[jgosmann]

I wonder if this should be low_open=True? I suppose one could set the amplitude to 0 without crashing Nengo, but it seems like it wouldn't make much sense.

Also, this parameter needs to be added to the docstring.

[hunse]

Done.

[jgosmann]

Technically this breaks backwards compatibility if anyone provided min_voltage as a positional argument. But I'm fine with that, I think min_voltage was rarely used if at all.

[hunse]

Yeah, people shouldn't rely on the position of keyword arguments. But there's also no need to have amplitude in front of min_voltage, so I'll switch them.

[hunse]

I've implemented @jgosmann's recommended changes.

[jgosmann]

Thank you, @hunse! :)