Increase make_literal precision #598
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
boost::compute::detail::make_literal is losing some precision when making literal floating point values because it uses std::numeric_limits::digits10, but that only gives us the number of decimal digits that will survive a decimal->native>decimal conversion; what we are doing is a native->decimal->native conversion, which can require up to ::max_digits10 (which is larger than ::digits10 by 2 (double) or 3 (float)).
However, max_digits10 is a c++11 feature, so this commit uses digits10 + 3 when the c++11 numeric_limits isn't available. That should be enough for an IEEE-style float, though is slightly more than necessary for some (e.g. double).
I also added a test for this (which fails with the current conversion for both float and double, and passes with the proposed commits).