Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: Can't I replace Ellipsis in DataFrame.replace like scalar #50373

Open
3 tasks done
jesrael opened this issue Dec 21, 2022 · 1 comment
Open
3 tasks done

BUG: Can't I replace Ellipsis in DataFrame.replace like scalar #50373

jesrael opened this issue Dec 21, 2022 · 1 comment
Labels
Bug Needs Triage Issue that has not been reviewed by a pandas team member replace replace method

Comments

@jesrael
Copy link

jesrael commented Dec 21, 2022

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

df = pd.DataFrame({'a': [1, 2, 3, 4], 'b': ['a', 'b', 'c', 'd'], 'c': [1.2, 3.4, 5.6, 7.8], 'd': [..., ..., ..., ...]})

Issue Description

Based by SO question, I cannot replace by Ellipsis:

#working 
df.mask(df == Ellipsis, 1)
df.replace(1, ...)
df.replace({Ellipsis: 1})
df .replace([...],(1))

#error
df.replace(..., 1)
df.replace(Ellipsis, 1)

TypeError: Expecting 'to_replace' to be either a scalar, array-like, dict or None, got invalid type 'ellipsis'

Expected Behavior

df =  df.replace({Ellipsis: 1})
print (df)
   a  b    c  d
0  1  a  1.2  1
1  2  b  3.4  1
2  3  c  5.6  1
3  4  d  7.8  1

Installed Versions

print (pd.show_versions())

INSTALLED VERSIONS
------------------
commit           : 2cb96529396d93b46abab7bbc73a208e708c642e
python           : 3.8.8.final.0
python-bits      : 64
OS               : Windows
OS-release       : 7
Version          : 6.1.7601
machine          : AMD64
processor        : Intel64 Family 6 Model 60 Stepping 3, GenuineIntel
byteorder        : little
LC_ALL           : None
LANG             : en
LOCALE           : sk_SK.ISO8859-2

pandas           : 1.2.4
numpy            : 1.23.1
pytz             : 2021.1
dateutil         : 2.8.1
pip              : 21.0.1
setuptools       : 52.0.0.post20210125
Cython           : 0.29.23
pytest           : 6.2.3
hypothesis       : None
sphinx           : 4.0.1
blosc            : None
feather          : None
xlsxwriter       : 1.3.8
lxml.etree       : 4.6.3
html5lib         : 1.1
pymysql          : None
psycopg2         : None
jinja2           : 2.11.3
IPython          : 7.22.0
pandas_datareader: 0.10.0
bs4              : 4.9.3
bottleneck       : 1.3.2
fsspec           : 0.9.0
fastparquet      : None
gcsfs            : None
matplotlib       : 3.3.4
numexpr          : 2.7.3
odfpy            : None
openpyxl         : 3.0.7
pandas_gbq       : None
pyarrow          : None
pyxlsb           : None
s3fs             : None
scipy            : 1.6.2
sqlalchemy       : 1.4.7
tables           : 3.6.1
tabulate         : None
xarray           : None
xlrd             : 2.0.1
xlwt             : 1.3.0
numba            : 0.53.1
None
@jesrael jesrael added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Dec 21, 2022
@jessica-writes-code
Copy link
Contributor

jessica-writes-code commented Jan 4, 2023

This bug seems to be occurring at (roughly) L7016 of pandas/core/generic.py, https://github.com/pandas-dev/pandas/blob/main/pandas/core/generic.py#L7016 . It seems to occur primarily because pandas/core/dtypes/inference.py:is_scalar doesn't recognize anything of type ellipsis as a scalar (i.e., when passed something of the ellipsis type, is_scalar returns False).

The -- kind of hacky? -- fix of including to_replace is Ellipsis as one of the or-ed conditions does appear to work. A more elegant fix seems to require editing something much lower-level. If the former is acceptable, I'm happy to open a PR with that change.

@simonjayhawkins simonjayhawkins added the replace replace method label Feb 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Needs Triage Issue that has not been reviewed by a pandas team member replace replace method
Projects
None yet
Development

No branches or pull requests

3 participants