What should be the calling convention for ufunc inner loop signatures?

[eric-wieser]

Mentioned in a bunch of places, but would be valuable to have a single place to track this.

To be able to reuse an inner loop across multiple dtypes (ie across variable-length strings, structured arrays, etc), we need access to the dtype object within the inner loop.

Perhaps we can achieve this by adding a new npy_bool innerloop_takes_dtype flag to ufuncs, which when set indicates that the inner loop signature now also takes a PyArray_Descr **dtypes argument.

[njsmith]

If it's possible it would be nice to avoid piling new flags and if statements into the ufunc code. I'm not sure if it's possible.

There was consensus a few years ago about hiding the ufunc struct internals from the C API. At least at that point, the only user was numba.

If that's still doable, then we can hopefully reimplement the current register-a-ufunc-loop function as one that takes an old-style function pointer, wraps it in a new-style function pointer, and then register that. That way the core ufunc code will be able to assume that all loops are new-style.

[eric-wieser]

There was consensus a few years ago about hiding the ufunc struct internals from the C API

This gets a +1 from me. I hadn't considered the fact that we can wrap one into the other.

Another option... In C, is it safe to call a function pointer with more arguments that it expects? Won't it just put extra items on its stack that it can ignore?

[seberg]

Is there consensus that this is the right approach? Because my gut feeling is that it might not be.

Basically, what operation would you want the dtype for in a more generic loop where it is not better to do it either before or after the all of the inner operations? My gut feeling would rather be:

1. Run dtype specific Loop Selection/Preparation function before actual calculations:
   • This can error out (say a unit error)
   • It could in principle even return the inner loop
   • It would set/create the new dtype object.
   • It can return a void * object to be passed in as additional data to the ufunc, which could be, but does not have to be the dtype.
2. Possibly run another function after the inner loop is finshed (for example to allow error handling, since the inner loop should be able to unlock the GIL).

I find it a bit strange to specifically pass in the dtype object to the inner loop, because I think we need a "setup" step anyway, and that setup step can decide what it wants to pass in. Of course for some "generic" inner loops (wrapping python objects or so), they may end up passing in the dtype, but that would be implementation.

Not sure I make sense, but it seems to me that almost all logic derived from the dtype inside the inner loop would rather already be handled before calling the inner loop.

EDIT: Maybe to add to it. If dtypes are not created new (but reuse the old one) and we pass them in. Then it seems to me that tagging errors to them will be tricky in a thread safe manner in any case.

[mhvk]

I'm also not sure we want to pass on dtype by default, since in general the inner loops should be simple and fast, and not have to deal with details like it. Even for structured dtypes, I'm not so sure I understand the general case; even for equality, if it weren't for being unable to override the ufunc, the current outside implementation would seem the proper solution (which suggests we just need to move some of the logic from ndarray.__eq__ to one of the setup functions in the np.equal ufunc).

For strings, the case seems clearer, but it does seem that the machinery is in principle present to deal with it: we should be able to use the void * for passing on, e.g., length information.

All this said, it does seem worth checking exactly where those helper functions get called, so that we can make sure, e.g., it is possible to return a custom ufunc loop or to create an output array with a custom dtype.

[njsmith]

We probably should rework the ufunc loop APIs in general to support setup/teardown/dispatch in a more logical way. (For example, np.add on strings will need some way to compute the output dtype before the actual loop starts running.)

But we should still pass the dtype information to every loop. Forcing people to jump through hoops to access this basic information makes no sense to me. Writing np.equal on strings shouldn't require writing a special setup trampoline that just copies a pointer from one place to another.

Possible new ufunc loop signature:

• nin
• nout
• dtypes (array of dtype object pointers with nin+nout entries)
• data (array of char*'s with nin+nout entries)
• outer_count: how many times the core operation should be performed (single integer)
• outer_strides: how far to step between each core operation (array of integers with nin+nout entries)
• core_ndims: the dimensionality of each core argument (array of integers with nin+nout entries, all zero for scalar ufuncs)
• core_shapes: the shapes of each core argument (use ndims to decode; can be NULL if core_ndims is all-zero)
• core_strides: the strides within each core operation (same layout as core_shapes)
• void *per_call_user_data: arbitrary data passed from the loop setup function
• void *per_loop_user_data: arbitrary data passed when registering the loop function.

[njsmith]

Here's a use case that might clarify things. At some point we'll want to expose the loop machinery to python, so people can write loops implemented in python, for prototyping and for all the stuff people use vectorize for.

So: given a proposal for the C loop signature, how hard would it be to write a single C loop function that can be registered with any ufunc, and turns around and invokes a python function instead? Obviously this is much easier if the loop has information on dtypes, so it can convert the raw data pointers into ndarrays to pass to python.

[seberg]

Good point, all flexible dtypes will need it in some form or another, that is definitely true. It is just, for downstream it seems to me that improving setup/teardown is more important. For the support of flexible dtypes, passing the dtypes would be the most convenient thing for sure.

So I am not against it, it will hardly hurt and is indeed more convenient in many cases. Too many flags (or typically unused args) may also not be convenient though, so before that happens, I am not sure if a minimal solution may not be just as well. But, we will need an iteration on the inner loop API anyway probably, so there is probably no point to trying to make due with the current signature.

I am sure it is already on everyone's list, but one other API change is that a stop iteration/error integer return value may be a good idea.

[seberg]

@njsmith well, but you need to write a single C-loop+setup func, I think we could live with that single loop /setup func being inconvenient to write :).

[seberg]

Anyway, I am not against it. I just thought it might be focusing on the wrong thing. But that is probably just me not being up to the full picture.