You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The flatness epsilon is currently hardcoded to 1e-13. For single precision computation, we should set this to a much larger value like 1e-5 or something so that we produce more accurate results.
Will need to do some testing to evaluate options. Perhaps we can make this epsilon configurable as an input option.
The text was updated successfully, but these errors were encountered:
The flatness epsilon is currently hardcoded to 1e-13. For single precision computation, we should set this to a much larger value like 1e-5 or something so that we produce more accurate results.
Will need to do some testing to evaluate options. Perhaps we can make this epsilon configurable as an input option.
The text was updated successfully, but these errors were encountered: