Numerically stable sigmoid function

[lukelowry]

Description

Updates CommonMath smooth step function behavior and documentation, and removes a stale expected-failure marker for the New England contingency-analysis example now that it passes.

Proposed changes

• Replace the logistic sigmoid with the stable tanh form.
• Define the sigmoid scale MU in one place and use it consistently in sigmoid, ramp.
• Add plots for the CommonMath documentation.
• Remove dsigmoid as its unused.
• Update CommonMath.md equations for the tanh sigmoid and smooth ramp form.
• Remove WILL_FAIL TRUE from newengland_ca, since ContingencyAnalysis now completes successfully.

Checklist

• All tests pass for the focused coverage listed below.
• Code compiles cleanly with flags -Wall -Wpedantic -Wconversion -Wextra.
• The new code follows GridKit™ style guidelines.
• There are unit tests for the new code.
• The new code is documented.
• The feature branch is rebased with respect to the target branch.

Changelog changes N/A

Further comments

The ContingencyAnalysis fix was completely unexpected. I needed to fix this function for my feature branch of the REECB implementation to run correctly. I was happy to see that this fixed another issue.

@PhilipFackler How often did you encounter this when implementing ContingencyAnalysis? I am curious, that would be useful for related commentary in the paper.

I'd also like to note here that this likely means we can fix/remove the inconsistent scaling that exists in IEEET1 and TGOV1 and some other models, I think. The failures were originally thought to be a solver issue but it was a NaN/inf issue. Before this change I was unable to make MU much larger than 240 but now I can increase it to be very large, so I can make the piecewise approximations as very sharp which is helpful for validation purposes

[nkoukpaizan]

A few minor comments.

The sensitivity of tanh to mu is different from exp (and to the abs forms), so it makes sense that NaNs occur at different values of mu.

@lukelowry please make sure to document (plot) the different forms we are exploring somewhere (PR, technical document or paper) for future reference.

[nkoukpaizan]

Missed this in a previous merge, but the explanation of the "softplus" form for $\rho$ is no longer in the documentation. I'm now noticing this with the tanh versus exp . Could $\rho$ also be written with the same elemental functions as $\sigma$?

[lukelowry]

The implementation of $\sigma$ before and after the proposed change is mathematically equivalent. it's just the numerical computation that's different. (i.e. the exp definition at very large negative values of $x$ computes like $\dfrac{1}{\infty}$ whereas the tanh version is identical result but stable)

Regarding $\rho$, to my knowledge, no, this can not use the same elemental functions and still be numerically stable.

I added justifications to the doc for clarity so this tracked information

[lukelowry]

@nkoukpaizan I went ahead and added plots since I already had some available. Let me know what you think.

[nkoukpaizan]

There are a couple of rendering issues for CommonMath.md on GH. After those are fixed, this should be good to merge.

[lukelowry]

There are a couple of rendering issues for CommonMath.md on GH. After those are fixed, this should be good to merge.

@nkoukpaizan Fixed, thank you for catching those rendering issues!