Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid IEEE754 converting fraction of seconds #487

Merged
merged 1 commit into from
Jul 19, 2016

Conversation

timonwong
Copy link
Contributor

@timonwong timonwong commented Jul 18, 2016

Due to the limitations of IEEE754, we may lost precision when
converting string back to datetime objects, for example:

float('0.511581') * 1e6, gives us 0.511580, which is unexpected.

In order to address that issue, use of float is avoided.

@timonwong timonwong force-pushed the fix-datetime-fraction-of-seconds branch from b94c23e to c438e4f Compare July 18, 2016 09:13
@coveralls
Copy link

coveralls commented Jul 18, 2016

Coverage Status

Coverage increased (+0.4%) to 88.737% when pulling c438e4f on timonwong:fix-datetime-fraction-of-seconds into 2525d0a on PyMySQL:master.

@coveralls
Copy link

coveralls commented Jul 18, 2016

Coverage Status

Coverage increased (+0.5%) to 88.858% when pulling c438e4f on timonwong:fix-datetime-fraction-of-seconds into 2525d0a on PyMySQL:master.

@timonwong timonwong force-pushed the fix-datetime-fraction-of-seconds branch from c438e4f to 4229e39 Compare July 18, 2016 13:37
@coveralls
Copy link

coveralls commented Jul 18, 2016

Coverage Status

Coverage increased (+0.4%) to 88.742% when pulling 4229e39 on timonwong:fix-datetime-fraction-of-seconds into 2525d0a on PyMySQL:master.

@timonwong timonwong closed this Jul 19, 2016
@timonwong timonwong reopened this Jul 19, 2016
@coveralls
Copy link

coveralls commented Jul 19, 2016

Coverage Status

Coverage increased (+0.4%) to 88.742% when pulling 4229e39 on timonwong:fix-datetime-fraction-of-seconds into 2525d0a on PyMySQL:master.

@methane
Copy link
Member

methane commented Jul 19, 2016

Thanks. But I feel parsing code is large enough to use regular expression.
Could you rewrite to use this?

r"(\d{1,4})-(\d{1,2})-(\d{1,2})[T ](\d{1,2}):(\d{1,2}):(\d{1,2})(?:.(\d{1,6}))?"

Due to the limitations of IEEE754, we may lost precision when
converting string back to datetime objects, for example:

`float('0.511581') * 1e6`, gives us `0.511580`, which is unexpected.

In order to address that issue, use of `float` is avoided
@timonwong timonwong force-pushed the fix-datetime-fraction-of-seconds branch from 4229e39 to 1df94d3 Compare July 19, 2016 08:11
@coveralls
Copy link

coveralls commented Jul 19, 2016

Coverage Status

Coverage increased (+0.3%) to 88.662% when pulling 1df94d3 on timonwong:fix-datetime-fraction-of-seconds into 2525d0a on PyMySQL:master.

@timonwong
Copy link
Contributor Author

Thanks. But I feel parsing code is large enough to use regular expression.
Could you rewrite to use this?

r"(\d{1,4})-(\d{1,2})-(\d{1,2})T :(\d{1,2}):(\d{1,2})(?:.(\d{1,6}))?"

Yep, I rewrited both timedelta and time converters as well.

@methane
Copy link
Member

methane commented Jul 19, 2016

Thank you.

@methane methane merged commit 3397c23 into PyMySQL:master Jul 19, 2016
@timonwong timonwong deleted the fix-datetime-fraction-of-seconds branch September 26, 2016 03:28
openstack-gerrit pushed a commit to openstack/openstack that referenced this pull request Nov 15, 2016
Project: openstack/requirements  de1b936cec39a73c205041b50ebebbaaefb56a17

Bump PyMySQL version to 0.7.6

Current min version has a microsecond rounding bug that makes it
impossible to do conditional updates (compare and swap) since the
datetime read by PyMySQL and therefore by SQLAlchemy does not match the
one in the DB.

This is affecting operations in Cinder now that we have a new cleanup
mechanism.

PyMySQL PR #487: PyMySQL/PyMySQL#487

Change-Id: I44ebfd593c78412bb6ce9c9ea3104c5c7c8f0af5
Closes-Bug: #1641312
openstack-gerrit pushed a commit to openstack/requirements that referenced this pull request Nov 15, 2016
Current min version has a microsecond rounding bug that makes it
impossible to do conditional updates (compare and swap) since the
datetime read by PyMySQL and therefore by SQLAlchemy does not match the
one in the DB.

This is affecting operations in Cinder now that we have a new cleanup
mechanism.

PyMySQL PR #487: PyMySQL/PyMySQL#487

Change-Id: I44ebfd593c78412bb6ce9c9ea3104c5c7c8f0af5
Closes-Bug: #1641312
openstack-gerrit pushed a commit to openstack/openstack that referenced this pull request Nov 15, 2016
Project: openstack/requirements  de1b936cec39a73c205041b50ebebbaaefb56a17

Bump PyMySQL version to 0.7.6

Current min version has a microsecond rounding bug that makes it
impossible to do conditional updates (compare and swap) since the
datetime read by PyMySQL and therefore by SQLAlchemy does not match the
one in the DB.

This is affecting operations in Cinder now that we have a new cleanup
mechanism.

PyMySQL PR #487: PyMySQL/PyMySQL#487

Change-Id: I44ebfd593c78412bb6ce9c9ea3104c5c7c8f0af5
Closes-Bug: #1641312
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 3, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants