Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consistency checker verbose output is inconsistent with the actual tolerances evaluated #425

Closed
snl-bconder opened this issue Jun 2, 2023 · 2 comments

Comments

@snl-bconder
Copy link

Per the numpy documentation for np.isclose, the total tolerance of that function is equal to atol + rtol * abs(b)

SarPy calls the np.isclose method when doing approximations.

When the CPHD validator provides verbose output, the printed tolerances are incorrectly shown per the calculation in consistency.py:438.

This syndrome I observe is that tolerances are under-represented, and the representation shown for negative values will not account for rtol at all.

@pressler-vsc
Copy link
Collaborator

Good catch snl-bconder and thanks for the nice write-up. np.isclose has a weird history and we got bit by its odd definition here. We're working through a fix that makes the printout consistent with the checked arithmetic and will link the PR to this issue once it is posted.

@Darder
Copy link
Contributor

Darder commented Jun 9, 2023

Closing issue, PR 427 addresses issue.

@Darder Darder closed this as completed Jun 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants