ENH: Enable fillna(value=None) #58085
Conversation
rhshadrach
added
Enhancement
Missing-data
| @@ -6827,6 +6827,10 @@ def fillna( | |||
| reindex : Conform object to new index. | |||
| asfreq : Convert TimeSeries to specified frequency. | |||
|
|
|||
| Notes | |||
| ----- | |||
| For non-object dtype, ``value=None`` will use the NA value of the dtype. | |||
There was a problem hiding this comment.
| Notes | ||
| ----- | ||
| For non-object dtype, ``value=None`` will use the NA value of the dtype. | ||
|
|
||
| Examples |
There was a problem hiding this comment.
Maybe you should add an example with value=None ?
There was a problem hiding this comment.
I think this is a good suggestion, but what do you think about doing it in https://pandas.pydata.org/docs/user_guide/missing_data.html#filling-missing-data instead of the docstring here? Can then link to this in the docstring.
There was a problem hiding this comment.
I think this is a good suggestion, but what do you think about doing it in https://pandas.pydata.org/docs/user_guide/missing_data.html#filling-missing-data instead of the docstring here? Can then link to this in the docstring.
That's fine, although I think the behavior of None with respect to how it is treated as a missing value isn't necessarily well explained on that page, so maybe that should be done as well.
| @@ -35,7 +35,7 @@ Other enhancements | |||
| - Support passing a :class:`Series` input to :func:`json_normalize` that retains the :class:`Series` :class:`Index` (:issue:`51452`) | |||
| - Users can globally disable any ``PerformanceWarning`` by setting the option ``mode.performance_warnings`` to ``False`` (:issue:`56920`) | |||
| - :meth:`Styler.format_index_names` can now be used to format the index and column names (:issue:`48936` and :issue:`47489`) | |||
| - | |||
| - :meth:`DataFrame.fillna` and :meth:`Series.fillna` can now accept ``value=None``; for non-object dtype the corresponding NA value will be used (:issue:`57723`) | |||
There was a problem hiding this comment.
The other change here is that you must specify a value for value, whereas before it defaulted to None
There was a problem hiding this comment.
Previously, the default value of None would also raise. In other words, both before an after this PR you have to specify value.
There was a problem hiding this comment.
Previously, the default value of None would also raise. In other words, both before an after this PR you have to specify
value.
That was true if you didn't specify method. But with 2.2:
>>> import pandas as pd
>>> import numpy as np
>>> s=pd.Series([1,np.nan,2])
>>> s.fillna(method="ffill")
<stdin>:1: FutureWarning: Series.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.
0 1.0
1 1.0
2 2.0
dtype: float64
>>>So I think it is worth saying that value is now required.
There was a problem hiding this comment.
But method is no longer an argument. In 2.x, you either had to specify value or method, otherwise we'd raise. Any code that does not raise a warning in 2.2 is already specifying the value argument. Any code that raises a warning in 2.2 will raise a TypeError in 3.0.
There was a problem hiding this comment.
So don't we have to say somewhere that we are enforcing the deprecation that method is no longer an argument?
There was a problem hiding this comment.
pandas/doc/source/whatsnew/v3.0.0.rst
Line 283 in 6e09e97
# Conflicts: # doc/source/whatsnew/v3.0.0.rst
|
Thanks @rhshadrach |
* ENH: Enable fillna(value=None) * fixups * fixup * Improve docs
value=Nonewithout method specified, but method is deprecated #57723 (Replace xxxx with the GitHub issue number)doc/source/whatsnew/vX.X.X.rstfile if fixing a bug or adding a new feature.The one behavior introduced here I think is important to highlight is using
Nonewith non-object dtypes. In such a case we treatNoneas the NA value for the given dtype for NumPy float/datetime and pandas' EA dtypes. However for external EAs, by default we try to putNonein the values and raise if this fails.For object dtype, we replace any NA value with
None.cc @Dr-Irv