New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add quantile regression #3148

Open
yangky11 opened this Issue May 13, 2014 · 16 comments

Comments

Projects
None yet
9 participants
@yangky11

yangky11 commented May 13, 2014

It seems that there are not very much packages in python with quantile regression...

@GaelVaroquaux

This comment has been minimized.

Member

GaelVaroquaux commented May 13, 2014

It seems that there are not very much packages in python with quantile
regression...

Why should it go in scikit-learn?

@yangky11

This comment has been minimized.

yangky11 commented May 13, 2014

If possible, why not?

@larsmans

This comment has been minimized.

Member

larsmans commented May 17, 2014

Because we have a lot of code to maintain already and we only include popular ML algorithms. Is this popular? Does it hold clear benefits for machine learning tasks over other approaches?

(Btw. VW has quantile regression, with loss ℓ(p,y) = τ(p - y)[[y ≤ p]] + (1 - τ)(y - p)[[y ≥ p]] where [[⋅]] are Iverson brackets.)

@glouppe

This comment has been minimized.

Member

glouppe commented May 17, 2014

GradientBoostingRegressor supports quantile regression (using loss=quantile and the alpha parameter). See http://scikit-learn.org/dev/auto_examples/ensemble/plot_gradient_boosting_quantile.html#example-ensemble-plot-gradient-boosting-quantile-py for an example.

@larsmans

This comment has been minimized.

Member

larsmans commented May 18, 2014

I should have checked that. Closing this issue.

@larsmans larsmans closed this May 18, 2014

@prcastro

This comment has been minimized.

prcastro commented Aug 27, 2017

While I don't agree that that aren't many packages for Quantile Regression on Python, I believe this is important to have pure Quantile Regression (not inside a Ensemble method) on sci-kit learn.

Quantile Regression has the advantage of targeting on a specific quantile of y. With this, it's possible to reduce the difference of the median of y_pred and y. It's similar to minimizing the absolute error in this case, but it's much more general and work for other quantiles.

Banks use this a lot when dealing with credit scoring and other models, so it's a battle tested model with real applications. R and SAS has this model implemented.

@amueller

This comment has been minimized.

Member

amueller commented Aug 29, 2017

@prcastro do you mean for a linear model?

@prcastro

This comment has been minimized.

prcastro commented Aug 30, 2017

Exactly. Today, sklearn implements quantile regression on ensemble methods. However, it's usually used as a regular linear model.

@amueller

This comment has been minimized.

Member

amueller commented Aug 30, 2017

I'd be open to adding it. @jnothman @GaelVaroquaux?

@agramfort

This comment has been minimized.

Member

agramfort commented Aug 30, 2017

@GaelVaroquaux

This comment has been minimized.

Member

GaelVaroquaux commented Aug 30, 2017

@amueller amueller reopened this Aug 30, 2017

@atorch

This comment has been minimized.

atorch commented Sep 26, 2017

Regarding the specific case of quantile regression for the median (absolute loss), as opposed to more general quantiles, it seems that http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.HuberRegressor.html would allow it if only we could pass epsilon=0.0. Why does HuberRegressor require epsilon : float, greater than 1.0? (The Huber Regressor optimizes the squared loss for the samples where |(y - X'w) / sigma| < epsilon and the absolute loss for the samples where |(y - X'w) / sigma| > epsilon.)

@GaelVaroquaux

This comment has been minimized.

Member

GaelVaroquaux commented Sep 26, 2017

@avidale

This comment has been minimized.

avidale commented Oct 22, 2017

I added classical quantile linear regression in the pull request above. Please review it!

@avidale

This comment has been minimized.

avidale commented Apr 19, 2018

Hello everyone! Please review my PR when you have time for it.

@avidale

This comment has been minimized.

avidale commented Aug 28, 2018

ping

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment