Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

explicitly truncate float precision in format_float #301

Merged
merged 2 commits into from Aug 26, 2019

Conversation

@jpoehnelt
Copy link
Contributor

commented Aug 26, 2019

Python is defaulting to 6 decimal places. As #277 shows, using 7 decimal places will lead to different results. 8 decimal places is probably more than we need.

https://en.wikipedia.org/wiki/Decimal_degrees#Precision

I'll wait to fix tests depending on 7 or 8 decimal places.

closes #277

@jpoehnelt jpoehnelt merged commit 5439986 into master Aug 26, 2019
3 checks passed
3 checks passed
cla/google All necessary CLAs are signed
continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
@jpoehnelt jpoehnelt deleted the fix/truncated_floats branch Aug 26, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.