Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rule for tanh has catastrophic cancellation for |x| > 20 #102

Open
tpapp opened this issue Dec 13, 2023 · 2 comments
Open

rule for tanh has catastrophic cancellation for |x| > 20 #102

tpapp opened this issue Dec 13, 2023 · 2 comments

Comments

@tpapp
Copy link
Contributor

tpapp commented Dec 13, 2023

One could simply revert #30 to fix this, but if one wants CSE then it can be calculated from first principles. I started a discussion which has numerical examples.

@devmotion
Copy link
Member

ChainRules should suffer from the same problem, possibly the rule was just copied to it from DiffRules: https://github.com/JuliaDiff/ChainRules.jl/blob/40b9058c1a6798a4f72bda1227786d584b44c822/src/rulesets/Base/fastmath_able.jl#L41

@stevengj
Copy link
Contributor

Note that the catastrophic cancellation happens way sooner than |x|=20 … by x = 4 the 1 - tanh(x)^2 formula has already lost 2 digits, by x = 6 it has lost 4 digits, and by x = 14 it has lost 10 digits.

See also @tpapp's discourse thread on this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants