Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: expanding/rolling_quantile() inconsistent with Series.quantile() #8084

Closed
seth-p opened this issue Aug 20, 2014 · 4 comments
Closed

BUG: expanding/rolling_quantile() inconsistent with Series.quantile() #8084

seth-p opened this issue Aug 20, 2014 · 4 comments
Labels
API Design Duplicate Report Duplicate issue or pull request Numeric Operations Arithmetic, Comparison, and Logical operations

Comments

@seth-p
Copy link
Contributor

seth-p commented Aug 20, 2014

It appears that Series.quantile() interpolates between the two bracketing values, whereas expanding/rolling_quantile() don't.

In [496]: x = Series([0,1])

In [497]: x.quantile(0.3)
Out[497]: 0.29999999999999999

In [498]: expanding_quantile(x, 0.3)
Out[498]:
0    0
1    0
dtype: float64

In [499]: rolling_quantile(x, 2, 0.3, min_periods=1)
Out[499]:
0    0
1    0
dtype: float64
@seth-p
Copy link
Contributor Author

seth-p commented Aug 21, 2014

My 2c: there really shouldn't be two different implementations of a quantiles functions.

@TomAugspurger
Copy link
Contributor

Looks like either way is mathematically valid, but I agree we should be consistent.
It's probably best to add a keyword to one of the implementations.

I'm looking at the numpy percentile function (which we use in DataFrame.quantile), and it mentions

The values and distances of the two
nearest neighbors as well as the interpolation parameter will
determine the percentile if the normalized ranking does not match q
exactly.

But it looks like this is parameter was only added in numpy 1.9 (which I think is the dev version).

numpy 1.8 docs: http://docs.scipy.org/doc/numpy-1.8.1/reference/generated/numpy.percentile.html#numpy.percentile
numpy 1.9 docs: http://docs.scipy.org/doc/numpy-dev/reference/generated/numpy.percentile.html

Anyway right now you can get the same behavior with df.interpolate(method='nearest') as long as you have scipy installed.

@TomAugspurger
Copy link
Contributor

I share your desire that rolling/expanding/regular should share an implementation, but NDFrame.interpolate allows for using a bunch of scipy methods. I'm not sure if we could do that in the Cython code.

@jreback jreback added this to the 0.15.1 milestone Oct 2, 2014
@jreback jreback modified the milestones: 0.16.0, Next Major Release Mar 6, 2015
@jreback
Copy link
Contributor

jreback commented May 3, 2017

dupe of #9413

@jreback jreback closed this as completed May 3, 2017
@jreback jreback added the Duplicate Report Duplicate issue or pull request label May 3, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API Design Duplicate Report Duplicate issue or pull request Numeric Operations Arithmetic, Comparison, and Logical operations
Projects
None yet
Development

No branches or pull requests

3 participants