Skip to content

BUG: The count() API does not consider inf as NA (with pandas.options.mode.use_inf_as_na = True) when dtype is "double[pyarrow]" #52501

@tinadu0806

Description

@tinadu0806

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

>>>import pandas as pd
>>>pd.options.mode.use_inf_as_na = True
>>>s = pd.Series([1.1, float("inf"), float("-inf"), 2.3], dtype="double[pyarrow]")
>>>s
0    1.1
1    NaN
2    NaN
3    2.3
dtype: double[pyarrow]
>>>s.count()
4
>>>type(s.count())
<class 'numpy.int64'>

Issue Description

When create a pandas series with pyarrow type "double[pyarrow]", after setting pd.options.mode.use_inf_as_na = True, the inf values are still counted. Also it returns a numpy int64 type scalar instead of pyarrow double scalar.

Expected Behavior

After setting pd.options.mode.use_inf_as_na = True, above s.count() should return 2 as result.

Installed Versions

Replace this line with the output of pd.show_versions()

Metadata

Metadata

Assignees

No one assigned

    Labels

    Arrowpyarrow functionalityBugExtensionArrayExtending pandas with custom dtypes or arrays.Missing-datanp.nan, pd.NaT, pd.NA, dropna, isnull, interpolate

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions