pytime.c loses precision under Windows

[pitrou]

BPO	19738
Nosy	@tim-one, @loewis, @pitrou, @vstinner, @larryhastings, @bitdancer

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2015-04-09.22:02:58.642>
created_at = <Date 2013-11-23.17:27:12.145>
labels = ['interpreter-core', 'type-feature']
title = 'pytime.c loses precision under Windows'
updated_at = <Date 2015-04-09.22:10:51.830>
user = 'https://github.com/pitrou'

bugs.python.org fields:

activity = <Date 2015-04-09.22:10:51.830>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2015-04-09.22:02:58.642>
closer = 'vstinner'
components = ['Interpreter Core']
creation = <Date 2013-11-23.17:27:12.145>
creator = 'pitrou'
dependencies = []
files = []
hgrepos = []
issue_num = 19738
keywords = []
message_count = 10.0
messages = ['204063', '204112', '204114', '204117', '204118', '204121', '204132', '240392', '240397', '240399']
nosy_count = 6.0
nosy_names = ['tim.peters', 'loewis', 'pitrou', 'vstinner', 'larry', 'r.david.murray']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue19738'
versions = ['Python 3.5']

[pitrou]

Under Windows, GetSystemTimeAsFileTime has a 100 ns resolution, but pygettimeofday stores it in a timeval which only has microsecond resolution. As a consequence, some precision is lost under Windows (which shows e.g. in time.time() results).

[tim-one]

Just noting for the record that a C double (time.time() result) isn't quite enough to hold a full-precision Windows time regardless:

>>> from datetime import date
>>> d = date.today() - date(1970, 1, 1)
>>> s = int(d.total_seconds())  # seconds into "the epoch"
>>> s *= 10**7  # number of 100ns into the epoch
>>> s.bit_length()
54
>>> 54 > 53  # QED ;-)
True

[pitrou]

Perhaps we need a time_ns() method? (returning an integer timestamp in nanoseconds)

[loewis]

I think this issue can be resolved by reducing the loss to the maximum available precision; it's about time.time(), after all.

I don't think pygettimeofday can change; gettimeofday traditionally has only µs. So the issue really is that it is used in implementing time.time().

As for whether an integer-returning current-time function in Python is needed: -1 (but then, I voiced the same concern when the ns-filestamp APIs where added). Adding an API will "force" people to rewrite their code, with no real improvement for practical improvement. The "force" comes from the mere availability of the API, and any emerging claims that using the time_ns() function is "more correct".

I really wish Python would have a 128-bit floating point type that could be used to represent a time stamp. Until such a type is available (perhaps in 2025), I propose that we live with limitations of 64-bit floating point. Anybody *really* needing the Windows system time can use ctypes (or pywin32) already.

[tim-one]

I agree overall with Martin, although time.time() could be made a little better on Windows by getting the Windows time directly (instead of "needlessly" losing info by going thru pygettimeofday).

[larryhastings]

Martin: I think the best choice would be a decimal object--which, now that we have decimal in C, is probably sufficiently performant for serious consideration.

[vstinner]

I think the best choice would be a decimal object--which, now that we have decimal in C, is probably sufficiently performant for serious consideration.

This idea was rejected: see the PEP-410 :-)

[bitdancer]

Victor, is this issue still relevant given your recent work on time?

[vstinner]

Victor, is this issue still relevant given your recent work on time?

In Python 3.3, I modified time.time() to use GetSystemTimeAsFileTime() instead of gettimeofday().

In Python 3.5, I replaced _PyTime_gettimeofday(_PyTime_timeval *tp) (resolution of 1 us) with _PyTime_t _PyTime_GetSystemClock(void) (resolution of 1 ns).

Moreover, most Python functions now use internally a monotonic clock (_PyTime_GetMonotonicClock) instead of the system clock (_PyTime_GetSystemClock), especially to compute timeout. Python 3.5 now ensures that a monotonic clock is available, it checks once at runtime (during Python startup).

The initial issue "pytime.c loses precision under Windows" has been fixed in the issue bpo-22117 which introduced the _PyTime_t type. The resolution of _PyTime_t is no more hardcoded in the API. Currently, it uses a resolution of 1 nanosecond, but it may be worse if we get compilation issues (no 64 bits signed integer type, too bad) or better (ex: if a compiler supports 128 bits signed integer type, I didn't try __int128 of GCC). The resolution is only handled in pytime.c, not in the users of the API.

Martin: I think the best choice would be a decimal object--which, now that we have decimal in C, is probably sufficiently performant for serious consideration.

This change has been discussed in the PEP-410 which has been rejected.

datetime.datetime.now() now uses _PyTime_GetSystemClock() and so indirectly GetSystemTimeAsFileTime() on Windows. Since the datetime type may get nanosecond resolution (issue bpo-15443), it will be possible to get the current time with nanosecond resolution ;-)

I now close the issue.

[vstinner]

Perhaps we need a time_ns() method? (returning an integer timestamp in nanoseconds)

Oh, I missed this message. The current trend in Python (os.stat) is to use a number of nanoseconds to store a timestamp. If we continue this trend in the datetime module, maybe with a new total_nanoseconds() method for example, it may make sense to add a new time.time_ns() function.