Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: mask with alignment #45443

Closed
2 of 3 tasks
Redoute opened this issue Jan 18, 2022 · 2 comments · Fixed by #47626
Closed
2 of 3 tasks

BUG: mask with alignment #45443

Redoute opened this issue Jan 18, 2022 · 2 comments · Fixed by #47626
Labels
Docs NA - MaskedArrays Related to pd.NA and nullable extension arrays replace replace method

Comments

@Redoute
Copy link

Redoute commented Jan 18, 2022

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd
s = pd.Series(range(5))
t = pd.Series([True, False])
s.mask(t, 99)

Issue Description

Series.mask replaces values if index is missing in condition. Example output:

0 99
1 1
2 99
3 99
4 99
dtype: int64

Expected Behavior

"Replace values where the condition is True." Values should not be touched if index is missing in condition.

Installed Versions

INSTALLED VERSIONS

commit : 66e3805
python : 3.8.12.final.0
python-bits : 64
OS : Windows
OS-release : 10
Version : 10.0.19042
machine : AMD64
processor : Intel64 Family 6 Model 60 Stepping 3, GenuineIntel
byteorder : little
LC_ALL : None
LANG : None
LOCALE : de_DE.cp1252

pandas : 1.3.5
numpy : 1.22.0
pytz : 2021.3
dateutil : 2.8.2
pip : 21.3.1
setuptools : 60.2.0
Cython : None
pytest : None
hypothesis : None
sphinx : 4.3.2
blosc : None
feather : None
xlsxwriter : 3.0.2
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : 2.9.3 (dt dec pq3 ext lo64)
jinja2 : 3.0.3
IPython : 7.19.0
pandas_datareader: None
bs4 : None
bottleneck : None
fsspec : None
fastparquet : None
gcsfs : None
matplotlib : 3.5.1
numexpr : None
odfpy : None
openpyxl : 3.0.9
pandas_gbq : None
pyarrow : 6.0.1
pyxlsb : None
s3fs : None
scipy : 1.7.3
sqlalchemy : 1.4.29
tables : None
tabulate : 0.8.9
xarray : None
xlrd : 2.0.1
xlwt : None
numba : None

@Redoute Redoute added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Jan 18, 2022
@phofl
Copy link
Member

phofl commented Jan 18, 2022

This behaves as documented. See:

The mask method is an application of the if-then idiom. For each element in the calling DataFrame, if cond is False the element is used; otherwise the corresponding element from the DataFrame other is used.

But could probably be clearer

@attack68 attack68 added Docs NA - MaskedArrays Related to pd.NA and nullable extension arrays replace replace method and removed Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Jan 22, 2022
@attack68 attack68 added this to the Contributions Welcome milestone Jan 22, 2022
@rhshadrach
Copy link
Member

I find this a bit surprising myself - I'd expect ser.mask(cond, other) to default to ser rather than other when values are missing. In other words not changing values should be the status-quo, modifications should be opt-in.

That said, I can't really justify this, and if others don't share in my surprise it should be left as-is.

ahmedibrhm added a commit to ahmedibrhm/pandas that referenced this issue Jul 7, 2022
ahmedibrhm added a commit to ahmedibrhm/pandas that referenced this issue Jul 8, 2022
ahmedibrhm added a commit to ahmedibrhm/pandas that referenced this issue Jul 8, 2022
mroeschke added a commit that referenced this issue Jul 8, 2022
* "#45443 edited the documentation of where/mask functions"

* Update generic.py

* Update generic.py

* Update generic.py

* Update pandas/core/generic.py

Co-authored-by: Matthew Roeschke <emailformattr@gmail.com>

* Update generic.py

* Update generic.py

Co-authored-by: Matthew Roeschke <emailformattr@gmail.com>
yehoshuadimarsky pushed a commit to yehoshuadimarsky/pandas that referenced this issue Jul 13, 2022
…s" (pandas-dev#47626)

* "pandas-dev#45443 edited the documentation of where/mask functions"

* Update generic.py

* Update generic.py

* Update generic.py

* Update pandas/core/generic.py

Co-authored-by: Matthew Roeschke <emailformattr@gmail.com>

* Update generic.py

* Update generic.py

Co-authored-by: Matthew Roeschke <emailformattr@gmail.com>
phofl pushed a commit that referenced this issue Jul 16, 2022
* DOC #45443 edited the documentation of where/mask functions

* DOC #45443 edited the documentation of where/mask functions

* Update generic.py

* ENH: add suffixes argument to DataFrame.compare #44354

* Edited the tests

* space fixing

* Update shared_docs.py

* Update series.py

* Update series.py

* invalid argument tests

* issue reference

* syntax editing

* grammar fixing

* edit doc

* editting doc

* Update 02_read_write.rst

* Update 02_read_write.rst

* Update v1.5.0.rst

* Update v1.5.0.rst

* np

* 1.5.0 rst

* created tests for invalid input

* space

* space

* space

* editing test
mroeschke pushed a commit that referenced this issue Aug 1, 2022
…nt (#47761)

* DOC #45443 edited the documentation of where/mask functions

* DOC #45443 edited the documentation of where/mask functions

* Update generic.py

* ENH: add suffixes argument to DataFrame.compare #44354

* Edited the tests

* space fixing

* Update shared_docs.py

* Update series.py

* Update series.py

* invalid argument tests

* issue reference

* syntax editing

* grammar fixing

* edit doc

* editting doc

* Update 02_read_write.rst

* Update 02_read_write.rst

* Update v1.5.0.rst

* Update v1.5.0.rst

* np

* 1.5.0 rst

* created tests for invalid input

* space

* space

* space

* editing test

* deprecated

* syntax

* editting existed examples

* syntax

* edit past tests

* editting pivot

* ex

* editing internal use

* pivot

* warning expected

* warning

* ignore doc warning

* doc

* tests

* ignore warning

* test

* plotting

* test

* doc

* doc

* white space

* doc

* doc

* doc

* doc

* stacklevel

* pivot

* pivot

* cookbook

* flake8

* flake8

* what's new

* syntax

* itr

* car names

* test edit

* fixing tests

* fixing tests

* flake8

* rst edit

* __iter__ edit

* flake8

* flake8

* space

* test

* merge

* ignore the type

* mypy

* type

* self.keys

* tests

* .

* .

* adding keys

* order

* attribute

* ignores

* Update hist.py

* ignore

* .

* .

* .

* .

* .

* Update doc/source/whatsnew/v1.5.0.rst

Co-authored-by: Richard Shadrach <45562402+rhshadrach@users.noreply.github.com>

Co-authored-by: Richard Shadrach <45562402+rhshadrach@users.noreply.github.com>
mroeschke pushed a commit that referenced this issue Oct 3, 2022
…lling it over empty df #47705 (#47672)

* DOC #45443 edited the documentation of where/mask functions

* DOC #45443 edited the documentation of where/mask functions

* Update generic.py

* Bug 43767 fix

* fixing lines doc

* visual indent

* visual indent

* indentation

* grouby.resample bug

* groubby.sample

* syntax

* syntax

* syntax

* syntax

* what's new

* flake8 error

* pytest

* blank line

* editting resample

* editting resample

* syntax

* syntax

* syntax

* space

* space

* space

* inplace

* spelling

* test

* test resampler

* tests

* tests

* Update resample.py

* Update resample.py

* Update resample.py

* Update v1.6.0.rst
noatamir pushed a commit to noatamir/pandas that referenced this issue Nov 9, 2022
…lling it over empty df pandas-dev#47705 (pandas-dev#47672)

* DOC pandas-dev#45443 edited the documentation of where/mask functions

* DOC pandas-dev#45443 edited the documentation of where/mask functions

* Update generic.py

* Bug 43767 fix

* fixing lines doc

* visual indent

* visual indent

* indentation

* grouby.resample bug

* groubby.sample

* syntax

* syntax

* syntax

* syntax

* what's new

* flake8 error

* pytest

* blank line

* editting resample

* editting resample

* syntax

* syntax

* syntax

* space

* space

* space

* inplace

* spelling

* test

* test resampler

* tests

* tests

* Update resample.py

* Update resample.py

* Update resample.py

* Update v1.6.0.rst
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Docs NA - MaskedArrays Related to pd.NA and nullable extension arrays replace replace method
Projects
None yet
4 participants