Skip to content

Comments

[SPARK-38940][PYTHON] Test Series' anchor frame for in-place updates on Series#36250

Closed
xinrong-meng wants to merge 1 commit intoapache:masterfrom
xinrong-meng:inplace_test
Closed

[SPARK-38940][PYTHON] Test Series' anchor frame for in-place updates on Series#36250
xinrong-meng wants to merge 1 commit intoapache:masterfrom
xinrong-meng:inplace_test

Conversation

@xinrong-meng
Copy link
Member

@xinrong-meng xinrong-meng commented Apr 18, 2022

What changes were proposed in this pull request?

Test Series' anchor DataFrame for in-place updates on Series.

Why are the changes needed?

In pandas, some in-place updates on Series change its anchor frame (e.g. Series.clip), while others don't.
We should test that to ensure parity with pandas.

Example on changed anchor DataFrame

>>> df = pd.DataFrame({'x': [1, 2, 3]})
>>> ser = df.x

>>> ser.clip(2, 2, inplace=True)
>>> ser
0    2
1    2
2    2
Name: x, dtype: int64
>>> df
   x
0  2
1  2
2  2

Example on unchanged anchor DataFrame

>>> df = pd.DataFrame({'x': [1, 2, 3]})
>>> ser = df.x

>>> ser.drop(1, inplace=True)
>>> ser
0    1
2    3
Name: x, dtype: int64
>>> df
   x
0  1
1  2
2  3

Does this PR introduce any user-facing change?

No. Test only.

How was this patch tested?

Unit tests.

@xinrong-meng
Copy link
Member Author

Note that Series.rename_axis has a bug so we may not test Series' anchor frame in this PR. The fix and the test will be added together in a followup PR.

@xinrong-meng xinrong-meng changed the title Test Series' anchor frame for in-place updates on Series [SPARK-38940][PYTHON] Test Series' anchor frame for in-place updates on Series Apr 18, 2022
@xinrong-meng xinrong-meng marked this pull request as ready for review April 18, 2022 23:59
@xinrong-meng
Copy link
Member Author

xinrong-meng commented Apr 18, 2022

@ueshin @HyukjinKwon @itholic Thank you!

Please refer to https://github.com/xinrong-databricks/spark/actions/runs/2186563996 for test results.

@HyukjinKwon
Copy link
Member

Merged to master.

HyukjinKwon pushed a commit that referenced this pull request Apr 21, 2022
…ename_axis`

### What changes were proposed in this pull request?
Test anchor frame for in-place `Series.rename_axis`.

### Why are the changes needed?
As a follow-up for #36250, we should ensure the in-place `rename_axis` on Series doesn't change its anchor frame.

### Does this PR introduce _any_ user-facing change?
No. Test only.

### How was this patch tested?
Unit test.

Closes #36269 from xinrong-databricks/series.rename_axis.

Authored-by: Xinrong Meng <xinrong.meng@databricks.com>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants