PERF: Index.__getitem__ performance issue #6370
Labels
Indexing
Related to indexing on series/frames, not to indexes themselves
Performance
Memory or execution speed performance
Milestone
Once again, caused by #6328 investigation.
There's something very strange with how
Index
objects handle slices:Obviously, this happens because
Index
doesn't override__getslice__
provided byndarray
, henceidx[:-1]
is executed viandarray.__getslice__
->Index.__array_finalize__
andidx[slice(None, -1)]
goes viaIndex.__getitem__
->Index.__new__
.__getitem__
is made 1000x slower trying to infer slice data type and convert it to a different subclass. The problem is that interactive invocationidx[:-1]
, which is when that milliseconds-vs-microseconds issue doesn't matter, is likely to miss this feature, because it's dispatched via__getslice__
. But for programmatic invocationidx[slice(None, -1)]
which hits this soft spot, I'd argue that this type conversion magic is not at all necessary.Is there a rationale behind this?
The text was updated successfully, but these errors were encountered: