interp: fix GE/LE operators to handle double-precision comparison#2525
Merged
rene-dev merged 1 commit intoLinuxCNC:2.9from Jun 18, 2023
Merged
interp: fix GE/LE operators to handle double-precision comparison#2525rene-dev merged 1 commit intoLinuxCNC:2.9from
rene-dev merged 1 commit intoLinuxCNC:2.9from
Conversation
This commit does four things: 1. Changes the LE/GE operators to better handle double-precision floating-point comparisons. When comparing two values as equal (or not equal) the EQ and NE operators considered the floating-point precision "issue" by comparing the difference between the two values to TOLERANCE_EQUAL, GE and LE did not. 2. Tidy the difference by using fabs() 3. Modify the TOLERANCE_EQUAL threshold to 1e-6. Previously this was set to .0001 which is very close to a reasonable difference between two gcode coordinate values. 4. Update the documentation to reflect these change.
snowgoer540
changed the title
Member
|
the linuxcnc norway meeting approves this PR. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This commit does four things: