Skip to content

Imports are slower than with CPython #2854

Closed
@pitrou

Description

@pitrou

If I compile the following functions using Cython:

def _noop_bench():
    pass

def _import_bench():
    import pyarrow.pandas_compat as pdcompat

And then measure their performance:

>>> %timeit lib._noop_bench()                                                                                                                                     
66.3 ns ± 0.379 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
>>> %timeit lib._import_bench()                                                                                                                                   
987 ns ± 21.1 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
>>> %timeit lib._import_bench()                                                                                                                                   
966 ns ± 17.6 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

The import overhead (~ 900 ns. per import statement) appears much slower than what CPython achieves:

>>> %timeit import pyarrow.pandas_compat as pdcompat                                                                                                              
253 ns ± 4.52 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
>>> %timeit import pyarrow.pandas_compat as pdcompat                                                                                                              
254 ns ± 3.66 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

This is with Python 3, Cython 0.29.4 and language_level = 3.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions