CLN: _consolidate_inplace less

[jbrockmendel]

No description provided.

[jreback]

great. any perf impact? (i know this means running a big asv.....)

[jbrockmendel]

i know this means running a big asv

im dealing with hardware issues that keep me from running this for a while

[jreback]

i know this means running a big asv

im dealing with hardware issues that keep me from running this for a while

ok happy to merge this now then

[jorisvandenbossche]

I don't think this should have been merged without any discussion or performance check?

[jreback]

sure the nightly regression should show this in any event: https://pandas.pydata.org/speed/pandas/#regressions?sort=1&dir=desc

but maybe note updating recently? @TomAugspurger

[TomAugspurger]

Seems to have segfaulted when compiling pandas a couple weeks back and never recovered. I won't have time to debug this for a little while.

…

On Wed, May 27, 2020 at 7:56 AM Jeff Reback ***@***.***> wrote: sure the nightly regression should show this in any event: https://pandas.pydata.org/speed/pandas/#regressions?sort=1&dir=desc but maybe note updating recently? @TomAugspurger <https://github.com/TomAugspurger> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#34389 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAKAOIQOPI7RNAYRXFZFH73RTUEZHANCNFSM4NKRKJXQ> .

[jorisvandenbossche]

the nightly regression should show this in any event

Not necessarily, we don't have benchmarks for all functionality and use cases.

The current BlockManager is optimized to deal with consolidated blocks. At once not consolidating anymore before certain operations, has the potential to cause a slowdown for those operations (where the slowdown could in principle be more than the gain of not consolidating). It might very well be that none of the changes has any impact. But I can't tell that without checking. And I don't think we should do that without checking.
So my proposal is to revert the PR for now, until someone has time to check it.

[jorisvandenbossche]

(and note that this is not in conflict with what I am saying on the mailing list. It's my opinion that we should optimize the case of "many 1D blocks" in the internals, but that doesn't mean that right now "many 2D blocks" is optimized).

[jreback]

its fine to revert if you want @jbrockmendel can you put up a PR. but going to continue optimizing generally anyhow.

[jorisvandenbossche]

I already made the branch before (using github's "revert" button), so -> #34407