Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make the relative tolerance in expect_near_rel continuous #1656

Closed
martinmodrak opened this issue Jan 29, 2020 · 1 comment · Fixed by #1657
Closed

Make the relative tolerance in expect_near_rel continuous #1656

martinmodrak opened this issue Jan 29, 2020 · 1 comment · Fixed by #1657
Labels
Milestone

Comments

@martinmodrak
Copy link
Contributor

Description

As discussed on Discourse, expect_near_rel behaves weirdly for values around zero. Most importantly, the relative tolerance jumps up at the tolerance value - here is the relative tolerance vs. average of the tested values in the current implementation, assuming tol = 1e-8:

image

This makes it hard to write tests that are strict for values near zero while not being too strict for large values.

Proposed solution

I propose that expect_near_rel has two tolerances relative (tol_rel) and minimal (tol_min). The final tolerance is then computed as

auto avg = 0.5 * (fabs(x1) + fabs(x2));
tol_final = max(tol_min, tol_rel * avg);
EXPECT_NEAR(x1, x2, tol_final);

where x1 and x2 are the values we are comparing.

The relative tolerance now increases smoothly when approaching zero as seen in this plot made with tol_rel = 1e-8, tol_min = 1e-16.
image

It is slightly unclear what should be the default value for tol_min. Using tol_min = tol_rel means the tolerance stays the same or is increased (relative to the current implementation). Using tol_min = tol_rel ^ 2 means the tolerance stays the same or decreases. Values in between would make the tolerance sometimes increase sometimes decrease.
Currently the default tolerance is 1e-8. Having tol_min = 1e-8 is quite lax, while having tol_min = 1e-16 is very strict (I can't make almost any slightly strong test pass with tol_min = 1e-16).

I would currently prefer somehing in between, possibly tol_min = max(tol_rel ^ 2,1e-14) (as 1e-14 is IMHO achievable for many functions).

Current Version:

v3.1.0

@mcol
Copy link
Contributor

mcol commented Feb 7, 2020

Fixed by #1657.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants