5.7 bug 81399 test replace number round+gis precision differences #72
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This rounds the numbers in the output where there is a
decimal to the number of decimal places specified.
Numbers > 1e10 or < -1e10 are expressed as components.
Trailing 0s aren't added to existing numbers
e.g.
--replace_numberic_round 5
10 -> 10
10. -> 10.
t1.col -> t1.col
t1elephant -> t1elephant
0x0FE11DE -> 0x0FE11DE
10.12345 -> 10.12345
10.123456 -> 10.12346
10.1234567 -> 10.12346
10.123451 -> 10.12345
10.00000 -> 10
10.00100 -> 10.001
10.000004 -> 10
10.000005 -> 10.00001
…tance_sphere test
…or cross platform results
|
Hi, thank you for your contribution. Please confirm this code is submitted under the terms of the OCA (Oracle's Contribution Agreement) you have previously signed by cutting and pasting the following text as a comment: |
|
I confirm the code being submitted is offered under the terms of the OCA, and that I am authorized to contribute it. |
|
Hi, thank you for your contribution. Your code has been assigned to an internal queue. Please follow |
grooverdan
changed the title
vmg
pushed a commit
to planetscale/mysql-server
that referenced
this pull request
Nov 21, 2023
buildkite: use private-whopper queue
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
While testing gis functions on ppc64le I discovered that the double precision frequently caused 1.9999 -> 2 discrepancies that weren't corrected by the regex.
I've added the replace_numberic_round mysqltest function that round decimal output to the prescribed precision. Excessive 0s at the end of output are removed.
The inbuilt rounding of printf has caused rest results to be rounded (sign always assumed to be positive). The rounded results here are as a result of running on an x86 machine.
There are still some test failures on ppc64le after these commits due to slightly different geometries returned. I have some been given some gis advice as to which of these are incorrect and will solve the cause of these in a future pull request.