BUG: Series.where behaves inconsistent

[arnoldfarkas]

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• (optional) I have confirmed this bug exists on the master branch of pandas.

---

Note: Please read this guide detailing how to provide the necessary information for us to reproduce your bug.

Code Sample, a copy-pastable example

>>> import pandas as pd
>>> df1 = pd.DataFrame([1, 2, 3, 4], columns=['w'])
>>> df2 = pd.DataFrame([1, 2, 3, 4], columns=['w'])
>>> c1 = df1['w'].where(df1['w'] < 4, 4)
>>> c2 = df2['w'].where(df2['w'] < 5)
>>> c1 *= 2
>>> c2 *= 2
>>> assert (c1 == c2).all()
>>> assert (df1['w'] == df2['w']).all() # df2 got changed, so assertion fails
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AssertionError
>>>
>>> df1
   w
0  1
1  2
2  3
3  4
>>> df2
   w
0  2
1  4
2  6
3  8
>>>

Problem description

The behavior is inconsistent: in case .where is used to select all elements in the series, a reference to the original data is returned instead of a copy. This way it is possible to modify the original dataframe/series - but only when all elements are selected.
In the example code, c1 is identical to c2, however changing c2 (which is created by selecting all elements) changes the parent dataframe.

Prior to 1.1.4 .where always returned a copy: checked 0.25.1, 1.1.0, 1.1.1, 1.1.2, 1.1.3 and all of them created a copy. This has been changed in 1.1.4, and 1.1.5 as well as 1.2.1 still has this behaviour.

Expected Output

There is no assertion, i.e. .where returns a copy consistently.

Please note that the documentation is unclear on this, it might worth explicitly mentioning whether or not copying is expected.

Output of pd.show_versions()

INSTALLED VERSIONS

commit : 67a3d42
python : 3.7.9.final.0
python-bits : 64
OS : Linux
OS-release : 4.20.13-1.el7.elrepo.x86_64
Version : #1 SMP Wed Feb 27 10:02:05 EST 2019
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 1.1.4
numpy : 1.19.4
pytz : 2020.1
dateutil : 2.8.1
pip : 20.2.4
setuptools : 47.3.1.post20201218
Cython : 0.29.21
pytest : 5.4.3
hypothesis : 5.30.0
sphinx : 3.0.3
blosc : None
feather : None
xlsxwriter : 1.2.9
lxml.etree : 4.5.2
html5lib : 1.1
pymysql : 0.10.1
psycopg2 : 2.8.6 (dt dec pq3 ext lo64)
jinja2 : 2.11.2
IPython : 7.17.0
pandas_datareader: None
bs4 : 4.9.3
bottleneck : 1.3.2
fsspec : 0.8.3
fastparquet : None
gcsfs : None
matplotlib : 3.3.2
numexpr : 2.7.1
odfpy : None
openpyxl : 3.0.5
pandas_gbq : None
pyarrow : 2.0.0
pytables : None
pyxlsb : None
s3fs : None
scipy : 1.5.4
sqlalchemy : 1.3.20
tables : 3.6.1
tabulate : 0.8.7
xarray : 0.15.1
xlrd : 1.2.0
xlwt : 1.3.0
numba : 0.51.2

[jorisvandenbossche]

Yes, where should consistently return a copy regardless of the exact mask. Labeling this as a regression.

Thanks for the report!

[simonjayhawkins]

moving to 1.2.3

[simonjayhawkins]

This has been changed in 1.1.4, and 1.1.5 as well as 1.2.1 still has this behaviour.

first bad commit: [d8c8cbb] REGR: inplace Series op not actually operating inplace (#37508) cc @jbrockmendel