BUG: Label and integer based indexing return different values for same column

[epmojo]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import numpy as np
import pandas as pd


def main():
    # create a data frame
    # dtypes must be the same for the issue to occur
    data = {'x': np.arange(8, dtype=np.int64), 'y': np.int64(0)}
    df = pd.DataFrame(data)

    # These next three lines are necessary to produce the issue.
    df = df.copy()
    data = df['y']
    df.iat[7, df.columns.get_loc('x')] = 9999

    # set the last row of column y to a desired value
    df.iat[7, df.columns.get_loc('y')] = 1234

    # compare get values for at and iat
    atValue = df.at[7, 'y']
    iatValue = df.iat[7, df.columns.get_loc('y')]
    print(iatValue, atValue, iatValue == atValue)

    # inspect data
    print("\nThe DataFrame:")
    print(df)

    print("\nLabel Based:")
    print(df['y'])

    print("\nInteger Based:")
    print(df.iloc[:, df.columns.get_loc('y')])

if __name__ == '__main__':
    main()

Issue Description

In the above example, a value of 1234 was set to the last row in column 'y' using an integer based method of indexing. However, if using a label based method of indexing to view the values, it returns the original value of 0 instead of 1234. This issue does not occur on pandas version 1.3.5.

1234 0 False

The DataFrame:
      x     y
0     0     0
1     1     0
2     2     0
3     3     0
4     4     0
5     5     0
6     6     0
7  9999  1234

Label Based:
0    0
1    0
2    0
3    0
4    0
5    0
6    0
7    0
Name: y, dtype: int64

Integer Based:
0       0
1       0
2       0
3       0
4       0
5       0
6       0
7    1234
Name: y, dtype: int64

Expected Behavior

The label based method returns the correct value.

Installed Versions

INSTALLED VERSIONS

commit : bb1f651
python : 3.9.10.final.0
python-bits : 64
OS : Windows
OS-release : 10
Version : 10.0.17763
machine : AMD64
processor : Intel64 Family 6 Model 142 Stepping 12, GenuineIntel
byteorder : little
LC_ALL : None
LANG : None
LOCALE : English_United States.1252

pandas : 1.4.0
numpy : 1.22.1
pytz : 2021.3
dateutil : 2.8.2
pip : 21.3.1
setuptools : 60.5.0
Cython : 0.29.26
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : 4.7.1
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : None
IPython : None
pandas_datareader: None
bs4 : None
bottleneck : 1.3.2
fastparquet : None
fsspec : None
gcsfs : None
matplotlib : 3.5.1
numba : 0.53.1
numexpr : None
odfpy : None
openpyxl : 3.0.9
pandas_gbq : None
pyarrow : 6.0.1
pyreadstat : None
pyxlsb : None
s3fs : None
scipy : 1.7.3
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
zstandard : None

[phofl]

cc @jbrockmendel The cache of the DataFrame looks off, could have been caused by #43406

[jbrockmendel]

Looks like the df.iat[7, df.columns.get_loc('x')] = 9999 causes us to split blocks without clearing df._item_cache. somehow this happens in series._set_value(index, value, takeable=True) inside DataFrame._set_value. Will need to look into this more closely.

[jbrockmendel]

an edit that fixes this case (haven't run the test suite with it though) is to go into DataFrame._set_value and change the series._set_value calls to series._mgr.setitem_inplace(loc, value), which avoids a block-splitting

[simonjayhawkins]

cc @jbrockmendel The cache of the DataFrame looks off, could have been caused by #43406

can confirm first bad commit: [03dd698] BUG: DataFrame.setitem sometimes operating inplace (#43406)

[simonjayhawkins]

an edit that fixes this case (haven't run the test suite with it though) is to go into DataFrame._set_value and change the series._set_value calls to series._mgr.setitem_inplace(loc, value), which avoids a block-splitting

on master, on 1.4.x series._mgr.setitem_inplace(loc, value) is the existing original code