Fast and safe Siegert implementation

[AlexVanMeegen]

Is your feature request related to a problem? Please describe.
The siegert_neuron requires a numerically tricky integration where an exp(s**2) and and 1 + erf(s) interact in the integrand. Currently, this problem is solved using a substitution that leads to an infinite integral which is better behaved but still tricky. The problem and the currently implemented solution is documented here.

Describe alternatives you've considered
Instead of the above substitution, one can use the scaled complementary error function to boil down the integrals to a single, well-behaved one. Here are the details: siegert_numerics.pdf. In all my tests so far this works pretty well.

Additional context
This idea is approve by @mhelias; I also asked @tomtetzlaff for his opinion but his answer is pending. I can provide a fully Python based implementation in case someone wants to test it.

[clinssen]

@AlexVanMeegen: this is a very in-depth description, and I'm not sure I understand the implications without further discussing this in detail. Could we ask you to take care of filing a pull request for this, if you (and @tomtetzlaff) indeed decide that this confers sufficient advantages?

[AlexVanMeegen]

@clinssen, I am not sure I see why a full PR is needed to understand the implications. Could you be a bit more specific what you are missing? Then I can try to answer before creating a full blown PR.

Nonetheless, I am of course most willing to make the PR if desired!

[clinssen]

I don't understand the domain well enough, to see the complete implications. If you are proposing to implement a mathematical re-formulation that yields the same results, but more accurately or with more numerical stability, this would I think be very welcome! And could you also say how it might impact runtime?

[AlexVanMeegen]

Yes, precisely: This is all about the numerical evaluation of a tricky integral (Eq. 2 in my notes). My proposal is simply to use a different mathematical reformulation of the integral than the one that is currently used. The new reformulation is, to my best current knowledge, better behaved.

Naively, I think this will also improve runtime without affecting the accuracy, but this of course needs a careful benchmark.

[AlexVanMeegen]

@tomtetzlaff also approves this, so I guess it is indeed sensible. I'll prepare a PR, please bug me in case I forget about this!

[clinssen]

Great, thank you! Could you please also check whether there is a unit test covering the behaviour of the neuron near the point of numerical instability?

[AlexVanMeegen]

Sure, I'll try to keep this in mind!

[jougs]

Shouldn't this issue have the word "safe" in the title, rather than "secure"? The same applies for #1849. Thanks!

[AlexVanMeegen]

Agreed, my bad!

[jougs]

If it were a really aggressive optimization, I'd of course also have accepted furious 😈

[AlexVanMeegen]

"These optimizations were performed by trained professionals, don't try this at home."