Feature Request: Create a Global Setting for Enabling numba engine

[jtelleriar]

Would it be possible to create a default pandas global setting to enable numba engine whenever possible?

In pandas.DataFrame.apply, .transform, etc.

Thanks!

[TomAugspurger]

cc @mroeschke. Seems reasonable.

[jorisvandenbossche]

We currently have pd.options.compute.use_numexpr/use_bottleneck options, so the interface could resemble that

[mroeschke]

I am +0 to the idea in theory, but there are some considerations that may not make this entirely convenient for the user.

• The numba behavior today doesn't have any "fall back" behavior like numexpr and bottleneck (I believe for those two). This was an intentional decision to make things simpler on our end.

• The numba and cython engines usage wise are not totally interchangeable. For example for groupby.transform, the UDF signature needs to be def f(values, index, ...) (exactly) for engine='numba' and anything for engine='cython'. So functions cannot be easily reused between engines

[mroeschke]

Just to be clear about the behavior for this feature:

1. groupby.transform(..., engine='cython') & compute.use_numba = True would use the numba engine
2. Internally we won't "fall back" to the cython behavior, mainly to reduce code complexity (and groupby.transform(..., engine='numba') already doesn't fall back as well)

[jorisvandenbossche]

groupby.transform(..., engine='cython') & compute.use_numba = True would use the numba engine

If the user explicitly specifies engine="cython", can't it then override the global config? Or why would that be complex (it's not a fallback or so)

[mroeschke]

engine='cython' is already the default keyword argument for the operations that have a numba option available.