Object mode or no-jit at all for unknown objects

[luk-f-a]

Hi,

I'm following up on #3907, regarding the deprecation of object mode. I almost never use object mode, but I could imagine a case like this:

import pandas as pd
import numba as nb

@nb.jit
def double(x):
    return x*2

double(0.5)

double(np.ones(3))

double(pd.DataFrame(np.ones(3)))

That example behaves as expected, all three lines produce a result. However, I think that it works by falling back on object mode. If object-mode fallback is removed, what would happen to a case like that?
I understand that there's an opportunity to simplify the pipeline by removing the fallback, but would it be possible to leave some kind of multiple-dispatch on unknown types that runs object-mode or simply interpreted code?
Unlike the current case, I don't think it needs to look line-by-line inside the function to know whether to fall back. For example, whereas today the following works via object-mode fallback

def something_non_jitable(x):
    return x

@nb.jit
def foo(x):
    return something_non_jitable(x)

foo(0.5)

in the future it would not, because it's not a problem of input parameters but of code failing. So in the future foo(0.5) would fail as it fails today with njit but double(pd.DataFrame(np.ones(3))) would not fail (as it does with njit) but would fallback to object code or interpreted code.

Something to consider. Thanks for your work on Numba.

[stuartarchibald]

Thanks for writing this up, I'm going to raise it at the core developer meeting today.

[stuartarchibald]

Make sure we also note this comment #3907 (comment)

[seibert]

Yeah, this is a good point. Things we are trying to discourage with this change:

• Automatically switching between modes of very different performance.
• Applying Numba to functions (accidentally) that it can't compile. (A common pitfall for new users.)
• Top-level usage of object mode: Object mode has several limitations, and we got ourselves into trouble by trying to reuse results from the compiler pipeline between nopython mode and object mode when we fall back.
• Magical thinking about what object mode can achieve (😄): Object mode suffers from being a compilation mode that seldom speeds up code, but exists for other reasons. New users tend to assume that if compilation succeeds, it must be faster.

That said, I agree that there is a potential use for Numba in situations where the author doesn't have total control over the types coming in, and would like some kind of best-effort compilation attempt. As you point out, this still needs assistance from the Numba dispatcher, in order to recognize argument type combinations where nopython mode will fail, and immediately dispatch to the interpreter without attempting to compile.

One possible solution (not committing to this, but just brainstorming) would be a new option interpreter_fallback=True to @jit that did the following:

• Compute type signature of input arguments and look for matches in the dispatch table. Call if match found.
• If no match in table and any of the arguments type to pyobject (Numba's "I don't know" type), then insert and call a dispatch entry to the interpreter version of the function.
• Otherwise, if compilation is still allowed on the function (i.e. the function wasn't declared with an explicit set of type signature.), compile in nopython mode.
  • If compilation succeeds, insert compiled function into dispatch table and call.
  • If compilation fails, insert dispatch entry to interpreter version of the function.
• Finally, if you get to this point, compilation is closed on the function and no matches have been found, so insert an entry into the dispatch table calling the interpreter version of the function.

This would be considered an advanced option that should only be used if the user understands the implications, and has the advantage of avoiding object mode while also allowing dispatch between compiled and interpreted implementations.

I'll bring this up at the developer meeting and see what people think. We're very hesitant to add new options to an already option-heavy @jit decorator.

[luk-f-a]

@seibert @stuartarchibald thanks for considering it. If @jithas too many options, what about a second (compounded) decorator? Something like

@jit 
@with_interpreted_fallback
def foo()

or even make room for future expansion

@jit(parallel, etc)
@with_advanced_options(interpreted_fallback=True, <other advanced otions>)
def foo()

[seibert]

The conclusion from today's discussion is that we're convinced by the arguments from you and @godaygo, and want to fuse the above proposal with this one: #3907 (comment).

@jit will get an optional fallback argument which is None by default (no fallback, raise an exception if compilation fails). If fallback is a special constant (numba.INTERPRETER), the fallback will be the decorated function, but executed in the Python interpreter. If fallback is a callable, the dispatcher will call that function (again, in the interpreter) when fallback is required, and that function will take the place of the decorated function. (i.e. the callable can return something back to the top-level caller) The rules for when to take the fallback option will be the same as I describe above.

In retrospect, this would have been a much better and more explicit way to handle our object-mode fallback logic, and it also allows library authors more freedom to use Numba for some cases, and do something entirely different when it doesn't work.

I'm not sure on the ETA for this, but we'll definitely get it into Numba at least one release before we drop object-mode fallback.

Thanks for talking through this. These discussions are very helpful for us. :)

[luk-f-a]

glad to be of help!