fused function dispatch is slow for Python calls

[robertwb]

Fused function dispatch currently generates a lot of generic code. In most cases, however, the number of distinct fused types in a function signature will be 1 or 2, so a short if-cascade would likely produce less code and be much faster.

The dispatch code is generated in make_fused_cpdef() in FusedNode.py. It generates Cython code. This could be optimised for the common case of a few (positional) arguments by dispatching directly based on the argument types, instead of comparing full signatures in a loop at runtime.

Migrated from http://trac.cython.org/ticket/829

[will-ca]

@scoder Thank you for pointing me to this issue. I did try searching for it, but I didn't see it. I'd like to try working on it as I get the time.

I'm not sure I fully understand the reasons behind the current implementation, though. As far as I can tell, it first attempts to find a matching Cython-space type name for each of the passed Python-object arguments and stores it in dest_sig, in the code generated by line 290 of FusedNode.py under ._fused_instance_checks() (as well as analogues for other special-cased objects like Numpy arrays), and then it iterates through and parses the signature of every single available permutation of specializations (presumably at O(k**n)), in the code generated around line 700 under make_fused_cpdef(), to search for exact matches to the received arguments (with a check to ignore unmatched arguments that are still None).

Assuming that the signatures for all available specializations are known ahead of time, though, and seeing how it's more or less checking for exact matches between dest_sig and signatures.keys(), why is it not possible or viable to just directly generate the signature of the required specialization from the inferred types of the input arguments in dest_sig?

[will-ca]

I suppose this issue is officially closed now, but I'd just like to note that AFAIK, even in the best-case scenarios, running a fused cpdef no-op function from CPython still takes several times longer than running a statically typed Cython function, a dynamically typed function Python function, or even an untyped Cython function using dynamic calls to isinstance() to type local variables at run time.

In particular, I noted that the first, "instance_checks" stage in .make_fused_cpdef() that generates a sequence of type name strings for incoming arguments uses quite a bit of list indexing and assignment, which could probably be sped up considerably as a single list comprehension.

That shouldn't be too hard to do I think, but there's also a bunch of conditional branching and conditional code generation for conditional branching IIRC, representing a fair bit of special-casing for specific types and external libraries. It's also already O(n) IIRC, so I stopped short of trying to change it for #3366.

In the ideal scenario, I think it may also be possible to just do a nested if-cascade, and skip the string-based signature checking, matching, indexing, and lookup entirely by directly calling the appropriate specialization with appropriately typed arguments. This is a workaround that I'm still using in project code for performance-critical fused cpdef functions that accept dynamic arguments, even with O(n) dispatch.