Skip to content

Commit

Permalink
Backport PR #52606 on branch 2.0.x (BUG: to_numeric(errors='coerce', …
Browse files Browse the repository at this point in the history
…dtype_backend='pyarrow') with ArrowDtype) (#52611)

Backport PR #52606: BUG: to_numeric(errors='coerce', dtype_backend='pyarrow') with ArrowDtype

Co-authored-by: Matthew Roeschke <10647082+mroeschke@users.noreply.github.com>
  • Loading branch information
meeseeksmachine and mroeschke committed Apr 12, 2023
1 parent 7e6261a commit 2e6c99a
Show file tree
Hide file tree
Showing 3 changed files with 13 additions and 1 deletion.
1 change: 1 addition & 0 deletions doc/source/whatsnew/v2.0.1.rst
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ Bug fixes
- Bug in :meth:`ArrowDtype.__from_arrow__` not respecting if dtype is explicitly given (:issue:`52533`)
- Bug in :func:`read_csv` casting PyArrow datetimes to NumPy when ``dtype_backend="pyarrow"`` and ``parse_dates`` is set causing a performance bottleneck in the process (:issue:`52546`)
- Bug in :class:`arrays.DatetimeArray` constructor returning an incorrect unit when passed a non-nanosecond numpy datetime array (:issue:`52555`)
- Bug in :func:`to_numeric` with ``errors='coerce'`` and ``dtype_backend='pyarrow'`` with :class:`ArrowDtype` data (:issue:`52588`)

.. ---------------------------------------------------------------------------
.. _whatsnew_201.other:
Expand Down
3 changes: 2 additions & 1 deletion pandas/core/tools/numeric.py
Original file line number Diff line number Diff line change
Expand Up @@ -270,7 +270,8 @@ def to_numeric(
# GH33013: for IntegerArray, BooleanArray & FloatingArray need to reconstruct
# masked array
if (mask is not None or new_mask is not None) and not is_string_dtype(values.dtype):
if mask is None:
if mask is None or (new_mask is not None and new_mask.shape == mask.shape):
# GH 52588
mask = new_mask
else:
mask = mask.copy()
Expand Down
10 changes: 10 additions & 0 deletions pandas/tests/tools/test_to_numeric.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@

import pandas as pd
from pandas import (
ArrowDtype,
DataFrame,
Index,
Series,
Expand Down Expand Up @@ -942,3 +943,12 @@ def test_invalid_dtype_backend():
)
with pytest.raises(ValueError, match=msg):
to_numeric(ser, dtype_backend="numpy")


def test_coerce_pyarrow_backend():
# GH 52588
pa = pytest.importorskip("pyarrow")
ser = Series(list("12x"), dtype=ArrowDtype(pa.string()))
result = to_numeric(ser, errors="coerce", dtype_backend="pyarrow")
expected = Series([1, 2, None], dtype=ArrowDtype(pa.int64()))
tm.assert_series_equal(result, expected)

0 comments on commit 2e6c99a

Please sign in to comment.