Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't expose EA.pad_or_backfill to users + switch to DeprecationWarning #54838

Merged
merged 6 commits into from Aug 29, 2023

Conversation

jorisvandenbossche
Copy link
Member

@jorisvandenbossche jorisvandenbossche commented Aug 29, 2023

Closes #54831

This does:

  • Rename ExtensionArray.pad_or_backfill to ExtensionArray._pad_or_backfill (added underscore -> this is only a method that needs to be implemented by developers (EA authors), and not be called by or visible to users)
  • Changed the warning in _pad_or_backfill for when the EA still only has fillna with method to a DeprecationWarning, since this is targetting EA authors, we can avoid that initially users see the warning
  • Removed limit_area keyword from ExtensionArray._pad_or_backfill signature, since it is unused (for all EAs except for numy dtypes). Removing it now because otherwise removing it later can give compatibility problems with third party EAs (and then it would need to be deprecated)
  • Fixed a bug in the Arrow array implementation

@jorisvandenbossche jorisvandenbossche added the ExtensionArray Extending pandas with custom dtypes or arrays. label Aug 29, 2023
@jorisvandenbossche jorisvandenbossche added this to the 2.1 milestone Aug 29, 2023
Comment on lines -148 to +165
msg = "|".join(
[
"ExtensionArray.fillna added a 'copy' keyword",
"Series.fillna with 'method' is deprecated",
]
)
msg = "Series.fillna with 'method' is deprecated"
with tm.assert_produces_warning(
FutureWarning, match=msg, check_stacklevel=False
FutureWarning,
match=msg,
check_stacklevel=False,
raise_on_extra_warnings=False,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

assert_produces_warning actually doesn't really work well when there are multiple warnings (it doesn't exactly check the messages, as long as one of the messages matches it seems). Therefore for this test, I tested each warning that should be raised separately (with raise_on_extra_warnings=False), to ensure those are certainly raised.
And for other tests below I went with the easier (but less strict) tm.assert_produces_warning((FutureWarning, DeprecationWarning), ...)

Comment on lines -939 to +934
if limit is not None and limit_area is not None:
if limit is None:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless I am missing something, I think this was just wrong before. This code path doesn't use limit, so it should only be used for the cases it was not specified (and this is None).

It worked before as well because the fall back option (the base class implementation) also worked.
And it only started failing now because I removed the limit_area is not None, so the remaining check started to pass when specifying just limit

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! This change looks correct to me

@jorisvandenbossche jorisvandenbossche mentioned this pull request Aug 29, 2023
@@ -140,26 +140,59 @@ def test_fillna_frame(self, data_missing):
def test_fillna_limit_pad(self, data_missing):
msg = "ExtensionArray.fillna 'method' keyword is deprecated"
with tm.assert_produces_warning(
FutureWarning, match=msg, check_stacklevel=False
DeprecationWarning,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not blocking: Would nested tm.assert_produces_warning work here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't try that today, but I seem to remember from trying that a while ago that this nesting those asserts don't work at all (although things might have changed)

Copy link
Member

@phofl phofl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@phofl phofl merged commit 6e29e3a into pandas-dev:main Aug 29, 2023
37 of 38 checks passed
meeseeksmachine pushed a commit to meeseeksmachine/pandas that referenced this pull request Aug 29, 2023
phofl pushed a commit that referenced this pull request Aug 29, 2023
…o users + switch to DeprecationWarning) (#54851)

Backport PR #54838: Don't expose EA.pad_or_backfill to users + switch to DeprecationWarning

Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
@jorisvandenbossche jorisvandenbossche deleted the EA-ffill-bfill branch August 30, 2023 13:26
mroeschke pushed a commit to mroeschke/pandas that referenced this pull request Sep 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ExtensionArray Extending pandas with custom dtypes or arrays.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

EA.fillna(method) deprecation and EA.pad_or_backfill issues
3 participants