use reduced precision float base class#150
Merged
asfgit merged 1 commit intoapache:masterfrom Jun 19, 2015
laimis:controlfp_s
Merged
use reduced precision float base class#150asfgit merged 1 commit intoapache:masterfrom laimis:controlfp_s
asfgit merged 1 commit intoapache:masterfrom
laimis:controlfp_s
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
We noticed that running tests in 32 bit CPUs was introducing floating point rounding issues and tests were failing as the same calculation could produce different result if executed multiple times in a single test. After some discussions on the mailing list it was determined that on 32 bit machines different instructions are used from 64 bit machines to process floating point numbers. Intermediate results when transferred with higher precision to the next operations eventually lead to rounding errors once a final calculation is returned as float.
From all the reading and trying several approaches so far, the only way I was able to consistently to get floating point conversion issues to go away is to pinvoke _controlfp_s to set floating point precision to 24 bits.
I got the idea from https://randomascii.wordpress.com/2013/07/16/floating-point-determinism/, see section "Floating-point settings (runtime)", last paragraph. controlfp seems to come up in other suggestions around this issue as well.
Here is the information on _controlfp_s: https://msdn.microsoft.com/en-us/library/c9676k6h.aspx. And it seems like there is an equivalent function in *nix environments although I haven't tried them or researched them in more depth yet.
For tests that rely on floating point calculations to be consistent, a variant of LuceneTestCase is used that sets the precision mask to 24 bits before each test run. I put this in the test as I don't think this code belongs to the core itself.