PERF: pd.eval when using mixed types

[jreback]

In [11]: df = pd.DataFrame({"A": np.arange(1000000), "B": np.arange(1000000, 0, -1), "C": np.random.randn(1000000)})

In [13]: %timeit pd.eval('df*df')
1 loops, best of 3: 635 ms per loop

In [14]: df = df.astype(float)

In [15]: %timeit pd.eval('df*df')
100 loops, best of 3: 5.87 ms per loop

the time is all in _interleave as we need to pass values to numexpr

[cpcloud]

How could this be fixed? I don't see how we can get around a copy being made to unify the type to pass to numexpr.

[jreback]

not sure
maybe if u detect only numeric then could just either use python or astype first
in this case the expression is quite simple

[mroeschke]

Looks like the perf is pretty close now so closing

In [6]: In [11]: df = pd.DataFrame({"A": np.arange(1000000), "B": np.arange(1000000, 0, -1), "C": np.random.randn(1000000)})

In [7]: %timeit pd.eval('df*df')
39.1 ms ± 255 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

In [8]: df = df.astype(float)

In [9]: %timeit pd.eval('df*df')
34.9 ms ± 259 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)