-
Notifications
You must be signed in to change notification settings - Fork 31
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Convert duration with full precision #476
Conversation
Best solution I came up with is to abuse |
qupulse/utils/types.py
Outdated
absolute_error == 0: Return the exact value of the float. 0.8 == 3602879701896397 / 4503599627370496 | ||
else: Use absolute error to limit the denominator | ||
|
||
str(value) guarantees that all floats have a different result with sensible rounding.this was chosen as a |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
typo
…loat deprecation and some cleanup
Benchmarks indicate a 10% performance hit in "combined" benchmark. For raw computations there is a factor of 2 overhead over gmpy2. The new code needs ~10µs per numeric operation and ~20µs to convert a float (on my machine). |
Before:
After:
|
# Conflicts: # tests/utils/time_type_tests.py + Make random usage deterministic
Pull Request Test Coverage Report for Build 1866
💛 - Coveralls |
# Conflicts: # tests/_program/loop_tests.py
This pull request fixes a duration value change (#475 ) when converting it from
float
toTimeType
(arbitrary precision via fractions) by abusingfloat
tostr
to get a consistent rounding.This might lead to unexpected consequences due to float rounding errors with fractional times.
TODO:
I think the factor 2 worse performance (with gmpy) is worth the improved ergonomics.
@eendebakpt