interpolate w/ method=time does not work with timedeltas #6424

Closed
cancan101 opened this Issue Feb 20, 2014 · 14 comments

Comments

Projects
None yet
8 participants
Contributor

cancan101 commented Feb 20, 2014

In [98]: pd.DataFrame({'v':[1,2,3]},index=pd.to_timedelta([1,2,3], unit="d")).interpolate(method="time")

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-98-9c399944f329> in <module>()
----> 1 pd.DataFrame({'v':[1,2,3]},index=pd.to_timedelta([1,2,3], unit="d")).interpolate(method="time")

/usr/lib64/python2.7/site-packages/pandas/core/generic.py in interpolate(self, method, axis, limit, inplace, downcast, **kwargs)
   2530                                           inplace=inplace,
   2531                                           downcast=downcast,
-> 2532                                           **kwargs)
   2533 
   2534         if inplace:

/usr/lib64/python2.7/site-packages/pandas/core/internals.py in interpolate(self, *args, **kwargs)
   2402 
   2403     def interpolate(self, *args, **kwargs):
-> 2404         return self.apply('interpolate', *args, **kwargs)
   2405 
   2406     def shift(self, *args, **kwargs):

/usr/lib64/python2.7/site-packages/pandas/core/internals.py in apply(self, f, *args, **kwargs)
   2373 
   2374             else:
-> 2375                 applied = getattr(blk, f)(*args, **kwargs)
   2376 
   2377             if isinstance(applied, list):

/usr/lib64/python2.7/site-packages/pandas/core/internals.py in interpolate(self, method, axis, index, values, inplace, limit, fill_value, coerce, downcast, **kwargs)
    819                                      inplace=inplace,
    820                                      downcast=downcast,
--> 821                                      **kwargs)
    822 
    823         raise ValueError("invalid method '{0}' to interpolate.".format(method))

/usr/lib64/python2.7/site-packages/pandas/core/internals.py in _interpolate(self, method, index, values, fill_value, axis, limit, inplace, downcast, **kwargs)
    880 
    881         # interp each column independently
--> 882         interp_values = np.apply_along_axis(func, axis, data)
    883 
    884         blocks = [make_block(interp_values, self.items, self.ref_items,

/usr/lib64/python2.7/site-packages/numpy/lib/shape_base.pyc in apply_along_axis(func1d, axis, arr, *args)
     77     outshape = asarray(arr.shape).take(indlist)
     78     i.put(indlist, ind)
---> 79     res = func1d(arr[tuple(i.tolist())],*args)
     80     #  if res is a number, then we have a smaller output array
     81     if isscalar(res):

/usr/lib64/python2.7/site-packages/pandas/core/internals.py in func(x)
    877             return com.interpolate_1d(index, x, method=method, limit=limit,
    878                                       fill_value=fill_value,
--> 879                                       bounds_error=False, **kwargs)
    880 
    881         # interp each column independently

/usr/lib64/python2.7/site-packages/pandas/core/common.py in interpolate_1d(xvalues, yvalues, method, limit, fill_value, bounds_error, **kwargs)
   1362         if not getattr(xvalues, 'is_all_dates', None):
   1363         # if not issubclass(xvalues.dtype.type, np.datetime64):
-> 1364             raise ValueError('time-weighted interpolation only works '
   1365                              'on Series or DataFrames with a '
   1366                              'DatetimeIndex')

ValueError: time-weighted interpolation only works on Series or DataFrames with a DatetimeIndex
Contributor

jreback commented Feb 20, 2014

unless their is real interest, won't do this

jreback closed this Feb 20, 2014

Contributor

cancan101 commented Feb 20, 2014

Can we leave this open then and mark as milestone "Someday"?

jreback added this to the Someday milestone Feb 20, 2014

jreback reopened this Feb 20, 2014

Contributor

hayd commented Dec 16, 2015

Run into this today as well.

vtselfa commented May 2, 2016

I have also run into this today.

Contributor

TomAugspurger commented Nov 29, 2016

If someone is interested in working on this, it'd follow a similar pattern to #14737 (convert to i8, interpolate, convert back to timedelta).

Contributor

aileronajay commented Nov 30, 2016

@TomAugspurger i can take this up post #14737 :)

Contributor

aileronajay commented Dec 4, 2016 edited

@TomAugspurger is this still an issue? when i run the code fragment given in the initial post of this issue, i get this output

pd.DataFrame({'v':[1,2,3]},index=pd.to_timedelta([1,2,3], unit="d")).interpolate(method="time")
v
1 days 1
2 days 2
3 days 3

Contributor

cancan101 commented Dec 4, 2016 edited

I filed this a long time ago, so bear with me. This does not seem to work:

pd.DataFrame({'v':[1,np.nan,5]},index=pd.to_timedelta([1,2,3], unit="d")).interpolate(method="time")

whereas this does:

pd.DataFrame({'v':[1,np.nan,5]},index=pd.to_datetime([1,2,3], unit="d")).interpolate(method="time")

I am seeing the same issue as in the above linked SO post.

Contributor

aileronajay commented Dec 4, 2016

@cancan101 so it fails when we try to pass timedelta, is that correct?

Contributor

cancan101 commented Dec 4, 2016

What do you mean?
This line works fine:

pd.DataFrame({'v':[1,np.nan,5]},index=pd.to_timedelta([1,2,3], unit="d"))
Contributor

aileronajay commented Dec 4, 2016

@cancan101 yes that is fine, what i am trying to say is that it fails when you try to interpolate the timedelta data,

pd.DataFrame({'v':[1,np.nan,5]},index=pd.to_timedelta([1,2,3], unit="d")).interpolate(method="time")

whereas when you call interpolate for datetime data, it works. That is the difference between the two lines that you've posted earlier (as examples of what works and what doesnt), right?

Contributor

cancan101 commented Dec 4, 2016

yes

Contributor

aileronajay commented Dec 5, 2016

added this PR, #14799

@jorisvandenbossche jorisvandenbossche modified the milestone: 0.20.0, Someday Dec 10, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment