df.groupby(df.index) drops index info when df has a MultiIndex

[rben01]

Code Sample, a copy-pastable example if possible

>>> df = pd.DataFrame( 
    {'a': list(range(10))},
    index=pd.MultiIndex.from_arrays(
        [[0,1,0,1,0,1,0,1,0,1], [0,1,2,0,1,2,0,1,2,0]],
        names=['l1', 'l2'])
)

>>> df
       a
l1 l2   
0  0   0
1  1   1
0  2   2
1  0   3
0  1   4
1  2   5
0  0   6
1  1   7
0  2   8
1  0   9

# explicitly groupby on level names or indices
>>> df.groupby(['l1', 'l2']).sum()  # or df.groupby(level=list(range(df.index.nlevels))).sum()
        a
l1 l2    
0  0    6
   1    4
   2   10
1  0   12
   1    8
   2    5

# groupby on the multi index itself
# instead of a MultiIndex DataFrame,
# returns a single-level-indexed DataFrame with tuples in the index
>>> df.groupby(df.index).sum()
         a
(0, 0)   6
(0, 1)   4
(0, 2)  10
(1, 0)  12
(1, 1)   8
(1, 2)   5

Problem description

When you group a DataFrame, whose index is a MultiIndex, on its index, resulting aggregations will be a DataFrame with a single-level index containing the tuples from the original MultiIndex. This is inferior to the behavior you obtain when passing the level names to df.groupby, which returns a DataFrame with the same MultiIndex levels and names.

Expected Output

When df has a MultiIndex, df.groupby(df.index) should be be identical to df.groupby(level=list(range(df.index.nlevels))) (or df.groupby(df.index.names) in the event that all of df's index levels are named).

Output of pd.show_versions()

INSTALLED VERSIONS ------------------ commit: None python: 3.6.8.final.0 python-bits: 64 OS: Darwin OS-release: 18.2.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: en_US.UTF-8

pandas: 0.23.4
pytest: 4.0.2
pip: 18.1
setuptools: 40.6.3
Cython: None
numpy: 1.15.4
scipy: 1.1.0
pyarrow: None
xarray: None
IPython: 7.2.0
sphinx: None
patsy: 0.5.1
dateutil: 2.7.5
pytz: 2018.7
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: 3.0.2
openpyxl: None
xlrd: 1.2.0
xlwt: None
xlsxwriter: None
lxml: None
bs4: 4.6.3
html5lib: 1.0.1
sqlalchemy: 1.2.15
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

[WillAyd]

Hmm OK makes sense. Investigation into the issue and PRs are always welcome!

[rhshadrach]

Depending on how you read the API docs, passing an index is either not explicitly supported or this is the "expected" behavior (the closest acceptable input to by for an Index is "list", but maybe this should say "list-like").

This occurs because

print(list(df.index))
[(0, 0), (1, 1), (0, 2), (1, 0), (0, 1), (1, 2), (0, 0), (1, 1), (0, 2), (1, 0)]

So I think this is either a docs issue or an enhancement request. I'm open to carving out special handling of passing a MultiIndex into groupby.

cc @jbrockmendel for any thoughts.