Skip to content

BUG: += on df.loc[:, col_level_0] produces NaN when columns are MutiIndex #52134

@sdementen

Description

@sdementen

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas

df = pandas.DataFrame(index=[0, 1], columns=pandas.MultiIndex.from_tuples([("a", "a1"), ("b", "b2"), ("c", "c2")]), data=0.0)

print("initial df")
print(df)
# the first two slices leads to buggy behavior
for sl in ["a", ("b",), ("c", slice(None))]:
    print()
    print(f"assigning with df.loc[:, {repr(sl)}] += 1.")
    print("before ", df.loc[:, sl])
    df.loc[:, sl] += 1.
    print("after", df.loc[:, sl])
print()
print("final df")
print(df)

Issue Description

using df.loc[:, first_level_of_mutliindex_column] += 1 assigns NaN instead of adding 1 to the existing value.
One must index the column with df.loc[:, (first_level_of_mutliindex_column, slice(None))] to make it work.
Yet getting df.loc[:, first_level_of_mutliindex_column] works as expected.

Expected Behavior

End up with

     a    b    c
    a1   b2   c2
0  1.0  1.0  1.0
1  1.0  1.0  1.0

instead of

    a   b    c
   a1  b2   c2
0 NaN NaN  1.0
1 NaN NaN  1.0

Installed Versions

INSTALLED VERSIONS

commit : 2e218d1
python : 3.8.10.final.0
python-bits : 64
OS : Windows
OS-release : 10
Version : 10.0.19045
machine : AMD64
processor : AMD64 Family 23 Model 96 Stepping 1, AuthenticAMD
byteorder : little
LC_ALL : None
LANG : None
LOCALE : English_Belgium.1252

pandas : 1.5.3
numpy : 1.23.3
pytz : 2022.4
dateutil : 2.8.2
setuptools : 60.2.0
pip : 21.3.1
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : 2.9.5
jinja2 : 3.1.2
IPython : None
pandas_datareader: None
bs4 : 4.11.2
bottleneck : None
brotli : None
fastparquet : None
fsspec : None
gcsfs : None
matplotlib : None
numba : None
numexpr : None
odfpy : None
openpyxl : 3.0.10
pandas_gbq : None
pyarrow : None
pyreadstat : None
pyxlsb : None
s3fs : None
scipy : None
snappy : None
sqlalchemy : 1.4.41
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
zstandard : None
tzdata : None

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions