BUG: Different behavior for different index types

[fedorkobak]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd

idx1 = pd.RangeIndex(6)
idx2 = pd.Index([0,1,2,3,4,5])
idx1_np = idx1.to_numpy()
idx2_np = idx2.to_numpy()

idx1_np[0] = 500
idx2_np[0] = 500

print(idx1)
print(idx2)

Issue Description

I created one RangeIndex and one Int64Index. Both of these indexes can be converted into numpy.array by using to_numpy function. That's what I do in the example.
But if I change the object that returned the index set explicitly it is reflected in the original index, but for the default index this does not happen. I think the behaviour should be the same regardless of the type of index.

Expected Behavior

RangeIndex(start=0, stop=5, step=1)
Int64Index([0, 1, 2, 3, 4, 5], dtype='int64')

Installed Versions

INSTALLED VERSIONS

commit : 91111fd
python : 3.10.7.final.0
python-bits : 64
OS : Linux
OS-release : 5.19.16-200.fc36.x86_64
Version : #1 SMP PREEMPT_DYNAMIC Sun Oct 16 22:50:04 UTC 2022
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : ru_RU.UTF-8
LOCALE : ru_RU.UTF-8

pandas : 1.5.1
numpy : 1.23.4
pytz : 2021.3
dateutil : 2.8.1
setuptools : 59.6.0
pip : 22.3.1
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : 3.0.2
lxml.etree : 4.7.1
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 3.0.3
IPython : 8.0.1
pandas_datareader: None
bs4 : 4.11.0
bottleneck : None
brotli : None
fastparquet : None
fsspec : None
gcsfs : None
matplotlib : 3.5.2
numba : None
numexpr : None
odfpy : None
openpyxl : 3.0.9
pandas_gbq : None
pyarrow : None
pyreadstat : None
pyxlsb : None
s3fs : None
scipy : 1.7.3
snappy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : 2.0.1
xlwt : None
zstandard : None
tzdata : None

[topper-123]

Yeah, this is a bug, thanks for catching. Probably connected to differences between RangeIndex and Int64Index implementation of to_numpy.

If you want a go at fixing this, we welcome contributions.

[topper-123]

I've looked more into this, and the underlying issue is that Index.to_numpy by default does not copy, but returns the original ndarray underlying the Index. E.g.

>>> import pandas as pd
>>> idx = pd.RangeIndex(3)
>>> arr = idx.to_numpy()
>>> arr[0] = 3
>>> arr
array([3, 1, 2], dtype=int64)
>>> idx
RangeIndex(start=0, stop=3, step=1)  # seemingly unchanged, but...
>>> idx._values
array([3, 1, 2], dtype=int64)  # underlying array changed!
>>> arr is idx._values
True
>>> idx.is_unique
True  # this is not true for `idx._values`

So the issue exists in both your examples, it's just hidden in the first one.

It seems the current behavior was added intentionally, as there are tests for this that require the returned array from to_numpy to be a reference rather than a copy of the original ._values. I personally would change the default for the copy parameter to be True and give extra warnings in the doc string about the risk when using copy=False, but that'd be a backward compatibility problem.

@jbrockmendel , do you have an opinion on this?

@Dranikf, I changed your original post to emphazise that this is a Index.to_numpy issue, not a Series issue.

[jreback]

the change should be to return a read only array here

[topper-123]

the change should be to return a read only array here

yeah, that’s also reasonable.

[jbrockmendel]

the change should be to return a read only array here

+1

[phofl]

Is this a regression? Otherwise no need to label as 1.5.2