the results of benchmarking small functions are malformed

[thautwarm]

Correctly maintaining the gabarge collection mechanisms of the two languages causes benchmarking issues, which can be observed when using Python's timeit.

# julia
using TyPython
using TyPython.CPython

@export_py function add(a::Int, b::Int)::Int
    return a + b
end

@export_pymodule myops begin
      add = Pyfunc(add)
end
# python

import operators
%timeit myops.add(1, 2)  # 500ns
%timeit operator.add(1, 2)  # 49ns

Above benchmark for Python operator.add is fine, but quite unfair for the Julia one, as in the realworld cases, myops.add works much faster:

import time
import sys

def realworld_timeit(code: str, N = 100000, globals: 'dict | None' = None):
    code = compile(f"for i in range({N}): {code}", '<string>', 'exec')
    t = time.time_ns()
    exec(code, globals or sys._getframe().f_globals)
    print((time.time_ns() - t) / N)


realworld_timeit("myops.add(1, 2)")  # 120ns
realworld_timeit("operator.add(1, 2)")  # 80ns

This is because Julia uses a real GC which will scan the heap instead of Python's reference counting. Calling into a small function from a foreign language (like CPython) can trigger a long GC operation.

Besides, WITH_GIL is so far a bit slow as it calls is_calling_julia_from_python. Such process is required when finalizing a Julia wrapped Python object.

jnumpy/TyPython/src/CPython.APIs.jl

Lines 13 to 17 in d132c71

	function Py(::UnsafeNew, ptr::C.Ptr{PyObject}=Py_NULLPTR)
	self = new(ptr)
	finalizer(self) do x
	if G_IsInitialized[]
	WITH_GIL(GILNoRaise()) do

jnumpy/TyPython/src/CPython.APIs.jl

Lines 193 to 196 in d132c71

	if is_calling_julia_from_python() && G_IsInitialized[]
	g = PyAPI.PyGILState_Ensure()
	try
	return f()

[thautwarm]

After #18 and #19 get merged, this issue will get eased a lot:

In [9]: %timeit mylib.add(1, 2)
196 ns ± 159 ns per loop (mean ± std. dev. of 7 runs, 10,000,000 loops each)

In [10]: %timeit mylib.add(1, 2)
209 ns ± 155 ns per loop (mean ± std. dev. of 7 runs, 10,000,000 loops each)

In [11]: %timeit operator.add(1, 2)
48.5 ns ± 0.313 ns per loop (mean ± std. dev. of 7 runs, 10,000,000 loops each)