Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: add quantile method to resample #15023

Closed
jorisvandenbossche opened this issue Dec 31, 2016 · 5 comments

Comments

Projects
None yet
3 participants
@jorisvandenbossche
Copy link
Member

commented Dec 31, 2016

Problem description

The quantile method is currently not supported on resample. While resample(...).agg(lambda x: x.quantile(..)) works fine, it would be nice to add the resample(..).quantile(..) shortcut.

Code Sample

In [19]: ts = pd.Series(range(20), index=pd.date_range('2016-01-01', periods=20))

In [21]: ts.resample('W').agg(lambda x: x.quantile(0.75))
Out[21]: 
2016-01-03     1.5
2016-01-10     7.5
2016-01-17    14.5
2016-01-24    18.5
Freq: W-SUN, dtype: float64

So it would be nice the following would give the same result:

In [22]: ts.resample('W').quantile(0.75)
/home/joris/miniconda3/envs/dev/bin/ipython:1: FutureWarning: 
.resample() is now a deferred operation
You called quantile(...) on this deferred object which materialized it into a series
by implicitly taking the mean.  Use .resample(...).mean() instead
  #!/home/joris/miniconda3/envs/dev/bin/python
Out[22]: 14.25

The fact that this currently implicitly takes the mean before calculating the quantile (ts.resample('W').mean().quantile(0.75)) would make this change slightly API breaking.

Using pandas master, 0.19.0+289.g1bf94c8

@jorisvandenbossche jorisvandenbossche added this to the Next Major Release milestone Dec 31, 2016

@jorisvandenbossche

This comment has been minimized.

Copy link
Member Author

commented Dec 31, 2016

BTW, using grouby this also works (and on rolling works as well):

In [23]: ts.groupby(pd.Grouper(freq='W')).quantile(0.75)
Out[23]: 
2016-01-03     1.5
2016-01-10     7.5
2016-01-17    14.5
2016-01-24    18.5
Freq: W-SUN, dtype: float64

In general should maybe check whether there are also other methods not working on resample.

@jreback

This comment has been minimized.

Copy link
Contributor

commented Dec 31, 2016

yeah, we have specificially defined Resampler methods (as we do for groupby).

@srinivasreddy

This comment has been minimized.

Copy link
Contributor

commented Jan 1, 2017

I am taking it

@srinivasreddy srinivasreddy referenced this issue Jan 1, 2017

Closed

[WIP]: Add quantile shortcut #15032

1 of 4 tasks complete
@jorisvandenbossche

This comment has been minimized.

Copy link
Member Author

commented Jan 2, 2017

@jreback can you remember if there was a specific reason that quantile was left out? or just an oversight?

@jreback

This comment has been minimized.

Copy link
Contributor

commented Jan 2, 2017

IIRC, some methods (e.g. cython defined ones, and some others), are specifically defined in .groupby, so these are 'named' methods and were transfered to .resample. However other methods are dispatched (IOW, on the whitelist but not defined as above). I don't think I audited all of these for suitability.

So in theory, we should evaluate these and define if appropriate (or possibly make a general method to make these definitions).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.