New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

get-internal-real-time should return monotonic time #20

Open
eugeneia opened this Issue Mar 3, 2017 · 5 comments

Comments

Projects
None yet
2 participants
@eugeneia
Contributor

eugeneia commented Mar 3, 2017

I think get-internal-real-time should return a monotonic/non-decreasing value. Even though the function is not mentioned in the manual (also an issue), its docstring says

Return the real time in the internal time format. (See INTERNAL-TIME-UNITS-PER-SECOND.) This is useful for finding elapsed time.

which a non-monotonic clock is not. Proposed solution: have get-internal-real-time derive its return value from clock_gettime(3) instead of gettimeofday(2).


Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

@xrme

This comment has been minimized.

Show comment
Hide comment
@xrme

xrme Mar 3, 2017

Member

There's ccl:current-time-in-nanoseconds, which was implemented in response to http://trac.clozure.com/ccl/ticket/924.

I am not against having get-internal-real-time use the same mechanism. However, I understand that some of these monotonic clocks may stop increasing during system sleep.

Linux, FreeBSD, and Solaris-ish systems have clock_gettime.

The Macintosh has mach_absolute_time: https://developer.apple.com/library/content/qa/qa1398/_index.html

For Windows, there's QueryPerformanceCounter:
https://msdn.microsoft.com/en-us/library/windows/desktop/dn553408(v=vs.85).aspx

https://www.python.org/dev/peps/pep-0418/ collects a lot information about various systems.

Member

xrme commented Mar 3, 2017

There's ccl:current-time-in-nanoseconds, which was implemented in response to http://trac.clozure.com/ccl/ticket/924.

I am not against having get-internal-real-time use the same mechanism. However, I understand that some of these monotonic clocks may stop increasing during system sleep.

Linux, FreeBSD, and Solaris-ish systems have clock_gettime.

The Macintosh has mach_absolute_time: https://developer.apple.com/library/content/qa/qa1398/_index.html

For Windows, there's QueryPerformanceCounter:
https://msdn.microsoft.com/en-us/library/windows/desktop/dn553408(v=vs.85).aspx

https://www.python.org/dev/peps/pep-0418/ collects a lot information about various systems.

@eugeneia

This comment has been minimized.

Show comment
Hide comment
@eugeneia

eugeneia Mar 4, 2017

Contributor

According to PEP 418 / Operating system time functions / Monotonic Clocks the options you mentioned all do include sleep (but only the proposed windows API accounts for time spent suspended—not sure if this is desired). Its looks though, as if current-time-in-nanoseconds does not deliver on windows where it uses GetSystemTimeAsFileTime, which according to MSDN documentation (also see “System Time”) is non-monotonic:

Retrieves the current system date and time.

System time is the current date and time of day. […] Note that the system can periodically refresh the time by synchronizing with a time source. Because the system time can be adjusted either forward or backward, do not compare system time readings to determine elapsed time.

Disclaimer: I am absolutely clueless of windows APIs.

get-internal-real-time could be something like this

(defun get-internal-real-time ()
  (nth-value 0 (floor (current-time-in-nanoseconds)
                      (load-time-value
                       (/ 1000000000 internal-time-units-per-second)))))

but I am afraid I would be no help at all with the windows implementation of current-time-in-nanoseconds.

Contributor

eugeneia commented Mar 4, 2017

According to PEP 418 / Operating system time functions / Monotonic Clocks the options you mentioned all do include sleep (but only the proposed windows API accounts for time spent suspended—not sure if this is desired). Its looks though, as if current-time-in-nanoseconds does not deliver on windows where it uses GetSystemTimeAsFileTime, which according to MSDN documentation (also see “System Time”) is non-monotonic:

Retrieves the current system date and time.

System time is the current date and time of day. […] Note that the system can periodically refresh the time by synchronizing with a time source. Because the system time can be adjusted either forward or backward, do not compare system time readings to determine elapsed time.

Disclaimer: I am absolutely clueless of windows APIs.

get-internal-real-time could be something like this

(defun get-internal-real-time ()
  (nth-value 0 (floor (current-time-in-nanoseconds)
                      (load-time-value
                       (/ 1000000000 internal-time-units-per-second)))))

but I am afraid I would be no help at all with the windows implementation of current-time-in-nanoseconds.

@eugeneia

This comment has been minimized.

Show comment
Hide comment
@eugeneia

eugeneia Jun 7, 2017

Contributor

I made it sound like get-internal-real-time was a CCL extension, but I remembered that its not. So in face of this it shouldn’t be documented in the CCL manual I suppose.

The standard is also kind of vague about requirements for internal time

get-internal-real-time returns as an integer the current time in internal time units, relative to an arbitrary time base. The difference between the values of two calls to this function is the amount of elapsed real time (i.e., clock time) between the two calls.

While I find the reference to clock time confusing, I tend to interpret “elapsed real time” to mean it should be monotonic.

So let’s say I want to fix this: I noticed hat get-internal-real-time lives in level-1/ while ccl:current-time-in-nanoseconds lives in lib/. Are there any dependencies I have to watch out for here? I.e. is lib allowed to depend on level-1? How about vice versa?

Contributor

eugeneia commented Jun 7, 2017

I made it sound like get-internal-real-time was a CCL extension, but I remembered that its not. So in face of this it shouldn’t be documented in the CCL manual I suppose.

The standard is also kind of vague about requirements for internal time

get-internal-real-time returns as an integer the current time in internal time units, relative to an arbitrary time base. The difference between the values of two calls to this function is the amount of elapsed real time (i.e., clock time) between the two calls.

While I find the reference to clock time confusing, I tend to interpret “elapsed real time” to mean it should be monotonic.

So let’s say I want to fix this: I noticed hat get-internal-real-time lives in level-1/ while ccl:current-time-in-nanoseconds lives in lib/. Are there any dependencies I have to watch out for here? I.e. is lib allowed to depend on level-1? How about vice versa?

@xrme

This comment has been minimized.

Show comment
Hide comment
@xrme

xrme Jun 7, 2017

Member

Certainly stuff in lib/ can depend on level-1.

There are or were certain restrictions about using #_ in level-1. I don't exactly remember the details. That's why ccl::gettimeofday is implemented the way it is (by calling a kernel import function, i.e., a C function in the lisp kernel whose address is stashed in a special location that lisp knows how to find).

I'd try moving the definition of current-time-in-nanoseconds into level-1/l1-lisp-threads.lisp and then see if it works to do (rebuild-ccl :clean t). If you get away with that, then see if it works to reimplement get-internal-real-time in terms of current-time-in-nanoseconds. If that fails, then I'd consider adding a new kernel import function to access the system's monotonic clock.

Member

xrme commented Jun 7, 2017

Certainly stuff in lib/ can depend on level-1.

There are or were certain restrictions about using #_ in level-1. I don't exactly remember the details. That's why ccl::gettimeofday is implemented the way it is (by calling a kernel import function, i.e., a C function in the lisp kernel whose address is stashed in a special location that lisp knows how to find).

I'd try moving the definition of current-time-in-nanoseconds into level-1/l1-lisp-threads.lisp and then see if it works to do (rebuild-ccl :clean t). If you get away with that, then see if it works to reimplement get-internal-real-time in terms of current-time-in-nanoseconds. If that fails, then I'd consider adding a new kernel import function to access the system's monotonic clock.

@eugeneia

This comment has been minimized.

Show comment
Hide comment
@eugeneia

eugeneia Jun 9, 2017

Contributor

Let me take a step back and question my own agenda here. I’ve looked at the sources of two other open source implementations (SBCL and ECL) and neither implement get-internal-real-time as monotonic. So it seems to me at least the current consensus is that monotonicity is not implied by the standard.

While I think that my interpretation of get-internal-real-time would be more useful, there are other ways to get a monotonic clock. See for instance @death’s monotonic-clock library. While this library only supports linux for the moment, it could be argued that it represents a more efficient alternative to bring the feature to all implementations in a de facto standard way via (ql:quickload "monotonic-clock").

Most importantly it would relieve implementations from bothering about it. I.e. if I had my way and CCL’s get-internal-real-time were to become monotonic, the added subroutines would have to be maintained, and it wouldn’t be portable to depend on it, unless all implementations agreed on this interpretation of the standard.

Whether or not that overhead is worth it depends on the CCL users. Does anyone except me want this behavior? It might be a net-positive for CCL to have this feature, regardless of what other implementations do. It might also be that everyone is better off if I try to contribute to @death’s monotonic-clock library instead.

Contributor

eugeneia commented Jun 9, 2017

Let me take a step back and question my own agenda here. I’ve looked at the sources of two other open source implementations (SBCL and ECL) and neither implement get-internal-real-time as monotonic. So it seems to me at least the current consensus is that monotonicity is not implied by the standard.

While I think that my interpretation of get-internal-real-time would be more useful, there are other ways to get a monotonic clock. See for instance @death’s monotonic-clock library. While this library only supports linux for the moment, it could be argued that it represents a more efficient alternative to bring the feature to all implementations in a de facto standard way via (ql:quickload "monotonic-clock").

Most importantly it would relieve implementations from bothering about it. I.e. if I had my way and CCL’s get-internal-real-time were to become monotonic, the added subroutines would have to be maintained, and it wouldn’t be portable to depend on it, unless all implementations agreed on this interpretation of the standard.

Whether or not that overhead is worth it depends on the CCL users. Does anyone except me want this behavior? It might be a net-positive for CCL to have this feature, regardless of what other implementations do. It might also be that everyone is better off if I try to contribute to @death’s monotonic-clock library instead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment