ARROW-17106: [Python] Move init code to core and expose only API #13802
Conversation
|
Opened ARROW-17320 to refine which symbols should be designated as private/public from |
|
@github-actions crossbow submit -g python |
|
Revision: 5c81e45 Submitted crossbow builds: ursacomputing/crossbow @ actions-474d060045
|
Default is to not re-export symbols with leading _
|
@github-actions crossbow submit -g python |
|
Revision: 8578c01 Submitted crossbow builds: ursacomputing/crossbow @ actions-db593693db
|
python/pyarrow/parquet/core.py
Outdated
| __all__ = [ | ||
| m for m, _ in inspect.getmembers(sys.modules[__name__]) | ||
| ] |
There was a problem hiding this comment.
It seems this is the same as the following, but much slower?
| __all__ = [ | |
| m for m, _ in inspect.getmembers(sys.modules[__name__]) | |
| ] | |
| __all__ = list(__dict__) |
There was a problem hiding this comment.
Maybe I'm missing somehting, but I get
-> 3463 __all__ = list(__dict__)
NameError: name '__dict__' is not defined
Changing it to list(sys.modules[__name__].__dict__) and a smidgen faster. Thanks!
In [10]: N = 1_000
In [11]: # __dict__
In [12]: timeit.timeit('_ = importlib.reload(core)', number=N, globals=globals()) / N
Out[12]: 0.0005946487420005723
In [13]: # inspect.getmembers
In [14]: timeit.timeit('_ = importlib.reload(core)', number=N, globals=globals()) / N
Out[14]: 0.000692891882998083
In [15]: There was a problem hiding this comment.
By the way, since you're using IPython, you probably know it but the %timeit macro is useful :-)
There was a problem hiding this comment.
Recently switched to it actually, thanks for the tip! 👍
|
Hmm, the crash on the Sphinx & Numpydoc CI job is unfortunate but certainly not related I think. |
|
Benchmark runs are scheduled for baseline = 210cf06 and contender = a2f3666. a2f3666 is a master commit associated with this PR. Results will be available as each benchmark for each run completes. |
Will close ARROW-17106
Moves all current code in
pyarrow/parquet/__init__.pytopyarrow/parquet/core.pyand re-exports the public API from core.py inside of the init file.