Support time functions like time.time()

[ehsantn]

Feature request

Supporting a wall clock time function like time.time() enables more accurate benchmarking since the time call can be inside the njit function, which avoids measure boxing/unboxing. This is very useful for benchmarking strings since boxing/unboxing large string containers is expensive.

[seibert]

This is a good idea. I think the logical way to do this would be to @overload the time.time() function and call the required C function to get the time. Unfortunately, I suspect that there is not a portable way to do this, so we will need to (again) peek at the CPython implementation to see what needs to happen.

A C function (get_time_double()?) should be added to helperlib.c (encapsulating all the #ifdef mess required to make it cross-platform) so the Python side of the implementation is straightforward.

[seibert]

This is the function in CPython. We can do something simpler, I think:

https://github.com/python/cpython/blob/master/Python/pytime.c#L653-L745

[ehsantn]

Sounds like a good plan. Looking at the CPython code, it doesn't seem to be too bad.

[mha-py]

I think it would be very useful to have the acces to the time inside a njit function! For example, a minmax ai would terminate after a certain time which requires time access.

[esc]

@mha-py FWIW: for timing long running processes and quitting after a certain wall clock time, you can also use the following to escape to object-mode and check the time from there:

https://numba.pydata.org/numba-doc/latest/user/withobjmode.html?highlight=objmode

[stuartarchibald]

xref: https://numba.discourse.group/t/profiling-with-a-decorator-and-njit for ideas and things that provide workarounds.

[auderson]

After some investigations, I found the following code may be a feasible solution:

from numba import njit
import ctypes
import time

# Access the _PyTime_AsSecondsDouble and _PyTime_GetSystemClock functions from pythonapi
get_system_clock = ctypes.pythonapi._PyTime_GetSystemClock
as_seconds_double = ctypes.pythonapi._PyTime_AsSecondsDouble

# Set the argument types and return types of the functions
get_system_clock.argtypes = []
get_system_clock.restype = ctypes.c_int64

as_seconds_double.argtypes = [ctypes.c_int64]
as_seconds_double.restype = ctypes.c_double

@njit
def f():
    system_clock = get_system_clock()
    current_time = as_seconds_double(system_clock)
    return current_time

assert time.time() < f() < time.time()

We can also overload time.time() using the above codes.

Xref: https://github.com/python/cpython/blob/main/Include/internal/pycore_time.h

[slayoo]

FWIW, here's another simple ctypes-based solution that works within njitted code, tested on Linux, Windows and macOS (https://github.com/open-atmos/PyMPDATA/blob/main/PyMPDATA/impl/clock.py):

""" CPU-time returning clock() function which works from within njit-ted code """
import ctypes
import platform

if platform.system() == "Windows":
    from ctypes.util import find_msvcrt

    __LIB = find_msvcrt()
    if __LIB is None:
        __LIB = "msvcrt.dll"
else:
    from ctypes.util import find_library

    __LIB = find_library("c")

clock = ctypes.CDLL(__LIB).clock
clock.argtypes = []

usage:

import numba
from .clock import clock

@numba.njit()
def step():
  time = clock()