Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deprecate DataFrame.fillna/Series.fillna with method #10349

Merged

Conversation

j-bennet
Copy link
Contributor

@j-bennet j-bennet commented Jun 12, 2023

See #10347.

Handles fillna deprecations like this one:

FAILED dask/dataframe/tests/test_dataframe.py::test_fillna - FutureWarning: DataFrame.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.
  • Tests added / passed
  • Passes pre-commit run --all-files

The remaining failures are not the same issue.

@j-bennet j-bennet marked this pull request as draft June 12, 2023 23:43
@j-bennet j-bennet requested a review from jrbourbeau June 13, 2023 19:35
@j-bennet j-bennet marked this pull request as ready for review June 13, 2023 19:35
@j-bennet j-bennet requested a review from phofl June 13, 2023 19:35
@j-bennet j-bennet changed the title fillna with 'method' is deprecated Deprecate DataFrame.fillna/Series.fillna with method Jun 13, 2023
Copy link
Collaborator

@phofl phofl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

small comment, otherwise looks generally good to me.

@j-bennet j-bennet closed this Jun 14, 2023
@j-bennet j-bennet reopened this Jun 14, 2023
@j-bennet j-bennet closed this Jun 14, 2023
@j-bennet j-bennet reopened this Jun 14, 2023
Copy link
Member

@hendrikmakait hendrikmakait left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, the code looks good. I have one open question/suggestion around test readability and one potential new issue we should file.

return parts.map_overlap(
M.fillna, before, after, method=method, limit=limit, meta=meta
)
return self.bfill(limit=limit, axis=axis)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Potential follow-up: It looks like we're not validating method anywhere and simply default to bfill even in the case of typos. If I'm correct, we should probably handle bfill in an elif and raise on a "default" else.

Copy link
Contributor Author

@j-bennet j-bennet Jun 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This preserves the previous behavior:

        if method in ("pad", "ffill"):
            ...
        else:
            method = "bfill"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment is out of scope for this PR, it was more of a general remark about the existing behavior that we may want to fix at some point.

dask/dataframe/tests/test_groupby.py Outdated Show resolved Hide resolved
Copy link
Member

@hendrikmakait hendrikmakait left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @j-bennet!

@hendrikmakait hendrikmakait merged commit 26160ff into dask:main Jun 23, 2023
23 of 24 checks passed
@j-bennet j-bennet deleted the j-bennet/fillna-with-method-deprecated branch June 23, 2023 15:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants