Is there a bug while slicing in the non monotonic multi indices?

[AayushSameerShah]

Please take this question lightly as asked from curiosity:

As I was trying to see how the slicing in MultiIndex works, I came across the following situation ↓

# Simple MultiIndex Creation
index = pd.MultiIndex.from_product([['a', 'c', 'b'], [1, 2]])

# Making Series with that MultiIndex
data = pd.Series(np.random.randint(10, size=6), index=index)

Returns:

a  1    5
   2    0
c  1    8
   2    6
b  1    6
   2    3
dtype: int32

NOTE that the indices are not in the sorted order ie. a, c, b is the order which will result in the expected error that we want while slicing.

# When we do slicing
data.loc["a":"c"]

Errors like:

UnsortedIndexError

----> 1 data.loc["a":"c"]
UnsortedIndexError: 'Key length (1) was greater than MultiIndex lexsort depth (0)'

That's expected. But now, after doing the following steps:

# Making a DataFrame
data = data.unstack()

# Redindexing - to unsort the indices like before
data = data.reindex(["a", "c", "b"])

# Which looks like 
   1  2
a  5  0
c  8  6
b  6  3

# Then again making series
data = data.stack()

# Reindex Again!
data = data.reindex(["a", "c", "b"], level=0)


# Which looks like before
a  1    5
   2    0
c  1    8
   2    6
b  1    6
   2    3
dtype: int32

The Problem

So, now the process is: Series → Unstack → DataFrame → Stack → Series

Now, if I do the slicing like before (still on with the indices unsorted) we don't get any error!

# The same slicing
data.loc["a":"c"]

Results without an error:

a  1    5
   2    0
c  1    8
   2    6
dtype: int32

Even if the data.index.is_monotonic → False. Then still why can we slice?

So the question is: WHY?.

I hope you got the understanding of the situation here. Because see, the same series which was before giving the error, after the unstack and stack operation is not giving any error.

So is that a bug, or a new concept that I am missing here?

Thanks!

Aayush ∞ Shah

[taozuoqiao]

I don't know whether it is relevent to #44752 I encountered, but one thing shared in common is that the _lexsort_depth (or is_lexsort) is not set right. In your example:

data.index.is_monotonic
# False

but

data.index._lexsort_depth
# 2
data.idnex.is_lexsorted
# True

[phofl]

This is not related to #44752, but thanks for checking @taozuoqiao

The codes are wrong here.

[phofl]

This is non trivial to fix, because unstack relies on the wrong codes unfortunately

[taozuoqiao]

Thanks for point this out, I have no much knowledge on the internal of pandas.