Skip to content

WIP,ENH: Remove np.array fastpaths and move asarray, etc. to C #15270

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Mar 18, 2021

Conversation

seberg
Copy link
Member

@seberg seberg commented Jan 6, 2020

This simplifies the code (although it adds the complexity of moving
some of the code to C). It also enables METH_FASTCALL when
available on python (which it is in most relevant versions by now)

Note that as of now, the old fastpaths were faster sometimes faster
if METH_FASTCALL is not given. If METH_FASTCALL is used
(available on 3.6+, although not official on 3.6) the new version
should always outperform the old one, by a large margin.


The first two commits are the basic infrastructure and simple examples and part of gh-15269. The last commit combines all changes related to moving np.asarray and friends to see, removing all special handling.

This requires some changes to the __array_function__ code (with the like= argument) unfortunately. It does also replace some np.array() usages, which I assumed were often micro-optimization since it was faster to use np.array() than np.asanyarray().

Forgot to mark as Draft: This includes changes from gh-15269

@seberg seberg changed the title ENH: Remove np.array fastpaths and move asarray, etc. to C WIP,ENH: Remove np.array fastpaths and move asarray, etc. to C Jan 6, 2020
@seberg seberg force-pushed the splitup-faster-argparsing-optimize-asarray branch 2 times, most recently from 1f7ce3f to 9b770e8 Compare January 7, 2020 01:53
@seberg seberg marked this pull request as draft December 6, 2020 05:18
@seberg seberg force-pushed the splitup-faster-argparsing-optimize-asarray branch from 9b770e8 to 0d41da7 Compare December 6, 2020 05:22
Base automatically changed from master to main March 4, 2021 02:04
@seberg seberg force-pushed the splitup-faster-argparsing-optimize-asarray branch from 0d41da7 to bd8e89f Compare March 11, 2021 22:14
@mattip
Copy link
Member

mattip commented Mar 11, 2021

Could you modify the top comment for this new version of the PR? It says "The first commit is the new benchmarks, the second is the actual change. The third commit changes ..." Did I miss the benchmarks and the results of running them somewhere?

@seberg
Copy link
Member Author

seberg commented Mar 11, 2021

I posted benchmarks in the initial PR that includes this one. The "benchmarks" this was referring to are the small-array-coercion benchmarks merged more than a year ago :) (this PR is what boosts speeds some up be a factor of 2 – in part underestimating the real factor of probably ~4)

Seems PyPy likes it now 🚀

This is a fast argument parser (an original version also supported
dictionary unpacking (for args, kwargs call style) which supports
the new FASTCALL and VECTORCALL convention of CPython. Fastcall
is supported by all Python versions we support.

This allows todrastically reduces the overhead of methods when keyword
arguments are passed.
@seberg seberg force-pushed the splitup-faster-argparsing-optimize-asarray branch from bd8e89f to 4d8fe21 Compare March 18, 2021 01:47
seberg added 2 commits March 17, 2021 21:01
Array methods are the easiest target for the new parser. They
do not require any larger improvements and most functions in
multiarray are wrapped. Similarly the printing functions
are not dispatched through `__array_function__` and thus have the
most to gain.
@seberg seberg force-pushed the splitup-faster-argparsing-optimize-asarray branch from 4d8fe21 to 9a45332 Compare March 18, 2021 02:03
@mattip mattip merged commit 267d49f into numpy:main Mar 18, 2021
@seberg seberg deleted the splitup-faster-argparsing-optimize-asarray branch March 18, 2021 17:05
@seberg
Copy link
Member Author

seberg commented Mar 18, 2021

Arrg, did I end up picking the wrong commit range here? It seems the actual asarray stuff was missing :(. I will open a new PR, sorry about that. (which I guess made this a nice "small" PR though).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants