BUG: groupby(..., dropna=False) excludes NA values when grouping on MultiIndex levels

[hickmanw]

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• (optional) I have confirmed this bug exists on the master branch of pandas.

---

Code Sample

In  [3]: mi = pd.MultiIndex.from_tuples([('a',np.nan), ('b', 2)], names=['ltr','num'])

In  [4]: df = pd.DataFrame({'foo': range(mi.size)}, index=mi)
In  [5]: df
Out [5]:
		foo
ltr	num	
a	NaN	0
b	2	1

# grouping on multindex levels
In  [6]: df.groupby(df.index.names, dropna=False).sum()
Out [6]:
		foo
ltr	num	
b	2	1

# grouping on same information as columns
In  [7]: df.reset_index().groupby(df.index.names, dropna=False).sum()
Out [7]:
		foo
ltr	num	
a	NaN	0
b	2.0	1

# grouping on multiindex levels for pd.Series
In  [8]: df['foo'].groupby(s.index.names, dropna=False).sum()
Out [8]:
ltr  num
b    2      1
Name: foo, dtype: int64

Problem description

The desired behavior of dropna=False, namely including NA values in the groups, does not work when grouping on MultiIndex levels, but does work when grouping on DataFrame columns.

Expected Output

		foo
ltr	num	
a	NaN	0
b	2.0	1

Output of pd.show_versions()

INSTALLED VERSIONS

commit : 2a7d332
python : 3.6.7.final.0
python-bits : 64
OS : Linux
OS-release : 4.14.193-149.317.amzn2.x86_64
Version : #1 SMP Thu Sep 3 19:04:44 UTC 2020
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : en_US.UTF-8
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 1.1.2
numpy : 1.19.0
pytz : 2020.1
dateutil : 2.7.4
pip : 20.1.1
setuptools : 40.4.3
Cython : None
pytest : 5.4.3
hypothesis : None
sphinx : 3.1.2
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 2.10
IPython : 7.0.1
pandas_datareader: None
bs4 : None
bottleneck : None
fsspec : None
fastparquet : None
gcsfs : None
matplotlib : 3.3.0
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pytables : None
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : 1.2.12
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
numba : None

[dsaxton]

@hickmanw I think similar issues have already been reported in various forms: #30750 #36060

[mroeschke]

Thanks for the report. Closing as a duplicate of #30750 #36060