time.thread_time isn't outputting in nanoseconds in AIX

[isidentical]

BPO	40192
Nosy	@abalkin, @pitrou, @vstinner, @pganssle, @isidentical
PRs	bpo-40192: Use thread_cputime for time.thread_time to improve resolution #19381

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-05-16.10:12:51.343>
created_at = <Date 2020-04-05.01:15:32.204>
labels = ['library', '3.9']
title = "time.thread_time isn't outputting in nanoseconds in AIX"
updated_at = <Date 2020-05-16.10:14:23.164>
user = 'https://github.com/isidentical'

bugs.python.org fields:

activity = <Date 2020-05-16.10:14:23.164>
actor = 'BTaskaya'
assignee = 'none'
closed = True
closed_date = <Date 2020-05-16.10:12:51.343>
closer = 'vstinner'
components = ['Library (Lib)']
creation = <Date 2020-04-05.01:15:32.204>
creator = 'BTaskaya'
dependencies = []
files = []
hgrepos = []
issue_num = 40192
keywords = ['patch']
message_count = 9.0
messages = ['365807', '368921', '368945', '368946', '368947', '369027', '369033', '369035', '369036']
nosy_count = 5.0
nosy_names = ['belopolsky', 'pitrou', 'vstinner', 'p-ganssle', 'BTaskaya']
pr_nums = ['19381']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue40192'
versions = ['Python 3.9']

[isidentical]

The resolution for thread_time is really low on AIX, but fortunately there is a way to get thread time in nanoseconds with thread_cputime.

-bash-4.4$ ./python
Python 3.9.0a5+ (heads/master:909f4a3, Apr  4 2020, 20:15:24) [C] on aix
Type "help", "copyright", "credits" or "license" for more information.
>>> import time
>>> time.thread_time()
0.02

[vstinner]

Which implementation is currently used on AIX? What's the output of the following command?

$ ./python
Python 3.9.0a6+ (heads/master:4a12d12186, May 15 2020, 04:55:17) 
>>> import time; time.get_clock_info('thread_time')
namespace(adjustable=False, implementation='clock_gettime(CLOCK_THREAD_CPUTIME_ID)', monotonic=True, resolution=1e-09)

[isidentical]

current:
>>> import time
>>> import time
>>> time.get_clock_info('thread_time')
namespace(adjustable=False, implementation='clock_gettime(CLOCK_THREAD_CPUTIME_ID)', monotonic=True, resolution=0.01)
>>> time.thread_time()
0.07

PR 19381:
>>> import time
>>> time.get_clock_info('thread_time')
namespace(adjustable=False, implementation='thread_cputime()', monotonic=True, resolution=1e-09)
>>> time.thread_time()
0.002379953

[vstinner]

I found this documentation on AIX thread_cputime():
https://www.ibm.com/support/knowledgecenter/ssw_aix_71/t_bostechref/thread_cputime.html

"""
Syntax

#include <sys/thread.h>
int thread_cputime (tid, ctime)
tid_t tid;
thread_cputime_t * ctime ;
typedef struct {
    uint64_t utime; /* User time in nanosenconds */
    uint64_t stime; /* System time in nanoseconds */
} thread_cputime_t;

Description

The thread_cputime subroutine allows a thread to query the CPU usage of the specified thread (tid) in the same process or in another process. If a value of -1 is passed in the tid parameter field, then the CPU usage of the calling thread is retrieved.

CPU usage is not the same as the total life of the thread in real time, rather it is the actual amount of CPU time consumed by the thread since it was created. The CPU usage retrieved by this subroutine contains the CPU time consumed by the requested thread tid in user space (utime) and system space (stime).

The thread to be queried is identified using the kernel thread ID which has global scope. This can be obtained by the application using the thread_self system call. Only 1:1 thread mode is supported. The result for M:N thread mode is undefined.

The CPU usage of a thread that is not the calling thread will be current as of the last time the thread was dispatched. This value will be off by a small amount if the target thread is currently running.
"""

Ok good, it returns the user time *and* the system time, and it's the thread CPU time.

So it sounds reasonable to use it to implement time.thread_time().

By the way, the v8 project calls thread_cputime() on AIX when clock_gettime(CLOCK_THREAD_CPUTIME_ID) is requested, also to get better resolution:
https://github.com/v8/v8/blob/a5038c42283a09f65c44229907123e15a779feb7/src/base/platform/time.cc#L68

// On AIX clock_gettime for CLOCK_THREAD_CPUTIME_ID outputs time with
// resolution of 10ms. thread_cputime API provides the time in ns
#if defined(V8_OS_AIX)
  thread_cputime_t tc;
  if (clk_id == CLOCK_THREAD_CPUTIME_ID) {
#if defined(__PASE__)  // CLOCK_THREAD_CPUTIME_ID clock not supported on IBMi
    return 0;
#endif
    if (thread_cputime(-1, &tc) != 0) {
      UNREACHABLE();
    }
  }
#endif

Another question is why AIX doesn't use thread_cputime() internally to implement clock_gettime(CLOCK_THREAD_CPUTIME_ID) :-)

[vstinner]

time.thread_time() is documented as:
"Return the value (in fractional seconds) of the sum of the system and user CPU time of the current thread. It does not include time elapsed during sleep."
https://docs.python.org/dev/library/time.html#time.thread_time

So we must use utime+stime, just not stime (current PR 19381 implementation).

test_time.test_thread_time() validates that time.thread_time() doesn't include time spend during a sleep.

[vstinner]

New changeset 4541086 by Batuhan Taskaya in branch 'master':
bpo-40192: Use thread_cputime for time.thread_time to improve resolution (GH-19381)
4541086

[isidentical]

Should we mention about AIX support in availability section @vstinner? https://docs.python.org/3/library/time.html#time.thread_time

[vstinner]

I close the issue, since the commit was merged.

"Availability: Windows, Linux, Unix systems supporting CLOCK_THREAD_CPUTIME_ID." covers AIX. But you can add AIX if you consider that it's not explicit enough.

[isidentical]

"Availability: Windows, Linux, Unix systems supporting CLOCK_THREAD_CPUTIME_ID." covers AIX.
That was my thought too, thanks again!