Training doesn't work with synapses

[hunse]

One issue I've run into a number of times is that I've forgotten to make synapses=None on my connections for training, and the training does not converge.

I realize that in some types of networks (recurrent networks?) synapses might be desirable, but I'm wondering if there could be some kind of warning that reminds users if they have synapses in their network, that training might not work.

[drasmuss]

Yeah I think a warning for the special case where you're training over one timestep would make sense (since in that case we can be pretty sure that synapse!=None is a mistake). For training over any steps >1 I'd think no warning though, since it would be not unreasonable for someone to want synaptic filtering there.

[hunse]

Makes sense to me.

If you wanted to be fancy you could check the number of steps versus the time constant of the filter. But that could get messy for more complex filters. Probably better to just have a note about that in some notebook on training networks over time (maybe there already is one).