Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: Pandas 2.0.0 may raise InvalidIndexError when assigning a column with an enum as label #52652

Closed
3 tasks done
GianlucaFicarelli opened this issue Apr 13, 2023 · 2 comments · Fixed by #52877
Closed
3 tasks done
Labels
Bug Indexing Related to indexing on series/frames, not to indexes themselves
Milestone

Comments

@GianlucaFicarelli
Copy link
Contributor

GianlucaFicarelli commented Apr 13, 2023

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

from enum import Enum
import pandas as pd

class E(Enum):
    X1 = "x1"
    X2 = "x2"

# it works with both Pandas 1.5.3 and 2.0.0
df = pd.DataFrame({"a": [10, 11]})
df[E.X1] = [20, 21]

# it works with Pandas 1.5.3, it fails with Pandas 2.0.0
df = pd.DataFrame(index=pd.RangeIndex(2))
df[E.X1] = [20, 21]

# it works with Pandas 1.5.3, it fails with Pandas 2.0.0
df = pd.DataFrame(index=pd.Index([0, 1]))
df[E.X1] = [20, 21]

# it works with Pandas 1.5.3, it fails with Pandas 2.0.0
df = pd.DataFrame()
df[E.X1] = [20, 21]

Error:

Traceback (most recent call last):
  File "./dev/try_pandas.py", line 14, in <module>
    df[E.X1] = [20, 21]
  File "./venv/src/pandas/pandas/core/frame.py", line 3969, in __setitem__
    self._set_item(key, value)
  File "./venv/src/pandas/pandas/core/frame.py", line 4177, in _set_item
    self._set_item_mgr(key, value)
  File "./venv/src/pandas/pandas/core/frame.py", line 4129, in _set_item_mgr
    loc = self._info_axis.get_loc(key)
  File "./venv/src/pandas/pandas/core/indexes/range.py", line 349, in get_loc
    self._check_indexing_error(key)
  File "./venv/src/pandas/pandas/core/indexes/base.py", line 5760, in _check_indexing_error
    raise InvalidIndexError(key)
pandas.errors.InvalidIndexError: E.X1

Issue Description

Pandas 2.0.0 may raise InvalidIndexError when assigning a column with an enum as label, if the DataFrame is empty.

Expected Behavior

The code shouldn't break, as it was with Pandas 1.5.3.

After using git bisect, it appears that the issue started with #49637

Installed Versions

INSTALLED VERSIONS

commit : be170fc
python : 3.10.8.final.0
python-bits : 64
OS : Linux
OS-release : 3.10.0-1160.42.2.el7.x86_64
Version : #1 SMP Tue Aug 31 20:15:00 UTC 2021
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : en_US.UTF-8
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 2.1.0.dev0+515.gbe170fc
numpy : 1.24.2
pytz : 2023.3
dateutil : 2.8.2
setuptools : 67.6.1
pip : 23.0.1
Cython : 0.29.34
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : None
IPython : None
pandas_datareader: None
bs4 : None
bottleneck : None
brotli : None
fastparquet : None
fsspec : None
gcsfs : None
matplotlib : None
numba : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pyreadstat : None
pyxlsb : None
s3fs : None
scipy : None
snappy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
zstandard : None
tzdata : 2023.3
qtpy : None
pyqt5 : None

@GianlucaFicarelli GianlucaFicarelli added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Apr 13, 2023
@phofl phofl added Indexing Related to indexing on series/frames, not to indexes themselves and removed Needs Triage Issue that has not been reviewed by a pandas team member labels Apr 22, 2023
@phofl phofl added this to the 2.0.1 milestone Apr 22, 2023
@phofl
Copy link
Member

phofl commented Apr 22, 2023

cc @topper-123

@topper-123
Copy link
Contributor

topper-123 commented Apr 23, 2023

I've looked into this and it's not as such a problem in #49637, but that this issue exposes that Rangeindex.get_loc gave a different error type than in Index.get_loc, causing this problem.

I've made a PR #52877 that will fix this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Indexing Related to indexing on series/frames, not to indexes themselves
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants