Sparse get dummies perf

[TomAugspurger]

Previously, we did a scalar elem == -1 for every element in the ndarray.

This replaces that check with a vectorized array == -1.

Running the ASV now. In the meantime, here's a simple timeit on the same problem

# HEAD
In [3]: %timeit pd.get_dummies(s, sparse=True)
561 ms ± 4.96 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

# Master
In [3]: %timeit pd.get_dummies(s, sparse=True)
        2.18 s ± 273 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

[pep8speaks]

Hello @TomAugspurger! Thanks for updating the PR.

Cheers ! There are no PEP8 issues in this Pull Request. 🍻

Comment last updated on July 20, 2018 at 15:48 Hours UTC

[TomAugspurger]

Here's the ASV (only a 3x speedup).

[100.00%] ··· Running reshape.GetDummies.time_get_dummies_1d_sparse                                                                                    1.79s       before           after         ratio
     [272bbdc7]       [bc658b03]
-           1.79s       1.05±0.02s     0.59  reshape.GetDummies.time_get_dummies_1d_sparse

SOME BENCHMARKS HAVE CHANGED SIGNIFICANTLY.

[mroeschke]

Small nit: you can param over sparce=False/True

[TomAugspurger]

I have a slight preference for leaving them separate, since they're such distinct code paths and it's a tad easier to run just sparse with this layout. Happy to change if you feel strongly about this.

[mroeschke]

Sounds good, no strong preference to use params then.

[jreback]

lgtm.

[jreback]

thanks!