Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

min and max of timedelta64 #5690

Closed
cancan101 opened this issue Dec 12, 2013 · 4 comments
Closed

min and max of timedelta64 #5690

cancan101 opened this issue Dec 12, 2013 · 4 comments
Labels
Bug Timedelta Timedelta data type
Milestone

Comments

@cancan101
Copy link
Contributor

related #5689

On pandas v12:

I thought this was fixed in #2990, but:

both of these:

pd.DataFrame(pd.Series((np.array([np.timedelta64(3000000000),np.timedelta64(3000000000)])))).min()

and

pd.Series((np.array([np.timedelta64(3000000000),np.timedelta64(3000000000)]))).min()

produce:

/usr/lib64/python2.7/site-packages/pandas/core/series.py in min(self, axis, out, skipna, level)
   1509         if level is not None:
   1510             return self._agg_by_level('min', level=level, skipna=skipna)
-> 1511         return nanops.nanmin(self.values, skipna=skipna)
   1512 
   1513     @Substitution(name='maximum', shortname='max',

/usr/lib64/python2.7/site-packages/pandas/core/nanops.py in f(values, axis, skipna, **kwds)
     78                     result = alt(values, axis=axis, skipna=skipna, **kwds)
     79             except Exception:
---> 80                 result = alt(values, axis=axis, skipna=skipna, **kwds)
     81 
     82             return result

/usr/lib64/python2.7/site-packages/pandas/core/nanops.py in nanmin(values, axis, skipna)
    279 @bottleneck_switch()
    280 def nanmin(values, axis=None, skipna=True):
--> 281     values, mask, dtype = _get_values(values, skipna, fill_value_typ = '+inf')
    282 
    283     # numpy 1.6.1 workaround in Python 3.x

/usr/lib64/python2.7/site-packages/pandas/core/nanops.py in _get_values(values, skipna, fill_value, fill_value_typ, isfinite, copy)
    130         mask = _isfinite(values)
    131     else:
--> 132         mask = isnull(values)
    133 
    134     dtype    = values.dtype

/usr/lib64/python2.7/site-packages/pandas/core/common.py in isnull(obj)
     59         given which of the element is null.
     60     """
---> 61     return _isnull(obj)
     62 
     63 

/usr/lib64/python2.7/site-packages/pandas/core/common.py in _isnull_new(obj)
     68     from pandas.core.generic import PandasContainer
     69     if isinstance(obj, np.ndarray):
---> 70         return _isnull_ndarraylike(obj)
     71     elif isinstance(obj, PandasContainer):
     72         # TODO: optimize for DataFrame, etc.

/usr/lib64/python2.7/site-packages/pandas/core/common.py in _isnull_ndarraylike(obj)
    157     else:
    158         # -np.isfinite(obj)
--> 159         result = np.isnan(obj)
    160     return result
    161 

TypeError: ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

I get the same Error with max as well.

median appears to work, although the type is incorrect:

In [144]:
pd.DataFrame(pd.Series((np.array([np.timedelta64(3000000000),np.timedelta64(3000000000)])))).median()

Out[144]:
0    3000000000
dtype: float64
@jreback
Copy link
Contributor

jreback commented Dec 13, 2013

from master

problem stems from the dtype inference when its an array (as opposed to a list) is wrong; it should be 'm8[ns]'

In [14]: pd.Series((np.array([np.timedelta64(3000000000),np.timedelta64(3000000000)]))).dtype
Out[14]: dtype('<m8')

related but this is not inferring correctly either (it needs an explicit dtype)

In [18]: pd.Series([np.timedelta64(3000000000),np.timedelta64(3000000000)],dtype='m8[ns]').min()
Out[18]: 
0   00:00:03
dtype: timedelta64[ns]

This works fine (and is prob the recommneded way in any event)
That said Series should try to infer better

In [22]: pd.to_timedelta(['00:00:03',np.timedelta64(3000000000)])
Out[22]: 
0   00:00:03
1   00:00:03
dtype: timedelta64[ns]

@cancan101
Copy link
Contributor Author

@jreback median is still not quite right in this case:

In [170]:
pd.Series([np.timedelta64(3000000000),np.timedelta64(3000000000)],dtype='m8[ns]').median()

Out[170]:
3000000000.0

@jreback
Copy link
Contributor

jreback commented Dec 13, 2013

its fine on master

In [1]: pd.Series([np.timedelta64(3000000000),np.timedelta64(3000000000)],dtype='m8[ns]').median()
Out[1]: 
0   00:00:03
dtype: timedelta64[ns]

@jreback
Copy link
Contributor

jreback commented Dec 13, 2013

going to close this in favor of #5689 these are all the same issue

@jreback jreback closed this as completed Dec 13, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Timedelta Timedelta data type
Projects
None yet
Development

No branches or pull requests

2 participants