Skip to content

Commit

Permalink
Backport PR pandas-dev#36114: REGR: fix consolidation/cache issue wit…
Browse files Browse the repository at this point in the history
…h take operation
  • Loading branch information
jorisvandenbossche authored and meeseeksmachine committed Sep 5, 2020
1 parent 45bf911 commit 79dfd75
Show file tree
Hide file tree
Showing 3 changed files with 26 additions and 0 deletions.
1 change: 1 addition & 0 deletions doc/source/whatsnew/v1.1.2.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ Fixed regressions
- Regression in :meth:`DatetimeIndex.intersection` incorrectly raising ``AssertionError`` when intersecting against a list (:issue:`35876`)
- Fix regression in updating a column inplace (e.g. using ``df['col'].fillna(.., inplace=True)``) (:issue:`35731`)
- Performance regression for :meth:`RangeIndex.format` (:issue:`35712`)
- Fix regression in invalid cache after an indexing operation; this can manifest when setting which does not update the data (:issue:`35521`)
- Regression in :meth:`DataFrame.replace` where a ``TypeError`` would be raised when attempting to replace elements of type :class:`Interval` (:issue:`35931`)
- Fix regression in pickle roundtrip of the ``closed`` attribute of :class:`IntervalIndex` (:issue:`35658`)
- Fixed regression in :meth:`DataFrameGroupBy.agg` where a ``ValueError: buffer source array is read-only`` would be raised when the underlying array is read-only (:issue:`36014`)
Expand Down
2 changes: 2 additions & 0 deletions pandas/core/generic.py
Original file line number Diff line number Diff line change
Expand Up @@ -3342,6 +3342,8 @@ class max_speed

nv.validate_take(tuple(), kwargs)

self._consolidate_inplace()

new_data = self._mgr.take(
indices, axis=self._get_block_manager_axis(axis), verify=True
)
Expand Down
23 changes: 23 additions & 0 deletions pandas/tests/frame/test_block_internals.py
Original file line number Diff line number Diff line change
Expand Up @@ -640,3 +640,26 @@ def test_update_inplace_sets_valid_block_values():

# smoketest for OP bug from GH#35731
assert df.isnull().sum().sum() == 0


def test_nonconsolidated_item_cache_take():
# https://github.com/pandas-dev/pandas/issues/35521

# create non-consolidated dataframe with object dtype columns
df = pd.DataFrame()
df["col1"] = pd.Series(["a"], dtype=object)
df["col2"] = pd.Series([0], dtype=object)

# access column (item cache)
df["col1"] == "A"
# take operation
# (regression was that this consolidated but didn't reset item cache,
# resulting in an invalid cache and the .at operation not working properly)
df[df["col2"] == 0]

# now setting value should update actual dataframe
df.at[0, "col1"] = "A"

expected = pd.DataFrame({"col1": ["A"], "col2": [0]}, dtype=object)
tm.assert_frame_equal(df, expected)
assert df.at[0, "col1"] == "A"

0 comments on commit 79dfd75

Please sign in to comment.