New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: maybe_convert_numeric fails with uint64 #14982

Closed
gfyoung opened this Issue Dec 25, 2016 · 1 comment

Comments

Projects
None yet
2 participants
@gfyoung
Member

gfyoung commented Dec 25, 2016

master at aba7d2:

>>> from pandas import lib, np
>>> arr = np.array([2**63], dtype=object)
>>> lib.maybe_convert_numeric(arr, set())
...
OverflowError: Python int too large to convert to C long

At first, I thought the patch would be similar to #14916, but there are several complications here:

  • Handling an array with negative and uint64 values: in maybe_convert_objects, we would just return object to avoid data truncation. However, this function currently only returns numeric arrays. And if we do decide to return non-numeric, should we then raise if coerce_numeric=True then?
  • uint64 has no convenient na_value, so what happens if a uint64 is in na_values?
@jreback

This comment has been minimized.

Show comment
Hide comment
@jreback

jreback Dec 26, 2016

Contributor

your two suggestions are fine

  • a uint64 array with negative values (or in na_values) must be converted to object.

so unfortunately maybe_convert_numeric gets more complicated :>

Contributor

jreback commented Dec 26, 2016

your two suggestions are fine

  • a uint64 array with negative values (or in na_values) must be converted to object.

so unfortunately maybe_convert_numeric gets more complicated :>

@jreback jreback added this to the Next Major Release milestone Dec 26, 2016

gfyoung added a commit to gfyoung/pandas that referenced this issue Dec 28, 2016

BUG: Convert uint64 in maybe_convert_numeric
Add handling for uint64 elements in an array
with the follow behavior specifications:

1) If uint64 and NaN are both detected, the
original input will be returned if coerce_numeric
is False. Otherwise, an Exception is raised.

2) If uint64 and negative numbers are both
detected, the origin input be returned if
coerce_numeric is False. Otherwise, an
Exception is raisd.

Closes gh-14982.

gfyoung added a commit to gfyoung/pandas that referenced this issue Dec 28, 2016

BUG: Convert uint64 in maybe_convert_numeric
Add handling for uint64 elements in an array
with the follow behavior specifications:

1) If uint64 and NaN are both detected, the
original input will be returned if coerce_numeric
is False. Otherwise, an Exception is raised.

2) If uint64 and negative numbers are both
detected, the original input be returned if
coerce_numeric is False. Otherwise, an
Exception is raised.

Closes gh-14982.
Partial fix for gh-14983.

gfyoung added a commit to gfyoung/pandas that referenced this issue Dec 28, 2016

BUG: Convert uint64 in maybe_convert_numeric
Add handling for uint64 elements in an array
with the follow behavior specifications:

1) If uint64 and NaN are both detected, the
original input will be returned if coerce_numeric
is False. Otherwise, an Exception is raised.

2) If uint64 and negative numbers are both
detected, the original input be returned if
coerce_numeric is False. Otherwise, an
Exception is raised.

Closes gh-14982.
Partial fix for gh-14983.

gfyoung added a commit to gfyoung/pandas that referenced this issue Dec 29, 2016

BUG: Convert uint64 in maybe_convert_numeric
Add handling for uint64 elements in an array
with the follow behavior specifications:

1) If uint64 and NaN are both detected, the
original input will be returned if coerce_numeric
is False. Otherwise, an Exception is raised.

2) If uint64 and negative numbers are both
detected, the original input be returned if
coerce_numeric is False. Otherwise, an
Exception is raised.

Closes gh-14982.
Partial fix for gh-14983.

gfyoung added a commit to gfyoung/pandas that referenced this issue Dec 29, 2016

BUG: Convert uint64 in maybe_convert_numeric
Add handling for uint64 elements in an array
with the follow behavior specifications:

1) If uint64 and NaN are both detected, the
original input will be returned if coerce_numeric
is False. Otherwise, an Exception is raised.

2) If uint64 and negative numbers are both
detected, the original input be returned if
coerce_numeric is False. Otherwise, an
Exception is raised.

Closes gh-14982.
Partial fix for gh-14983.

@jreback jreback closed this in 17d7ddb Dec 30, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment