# scikit-learn/scikit-learn

Open
opened this Issue May 13, 2014 · 16 comments

Projects
None yet
9 participants

### yangky11 commented May 13, 2014

 It seems that there are not very much packages in python with quantile regression...
Member

### GaelVaroquaux commented May 13, 2014

 It seems that there are not very much packages in python with quantile regression... Why should it go in scikit-learn?

### yangky11 commented May 13, 2014

 If possible, why not?
Member

### larsmans commented May 17, 2014

 Because we have a lot of code to maintain already and we only include popular ML algorithms. Is this popular? Does it hold clear benefits for machine learning tasks over other approaches? (Btw. VW has quantile regression, with loss ℓ(p,y) = τ(p - y)[[y ≤ p]] + (1 - τ)(y - p)[[y ≥ p]] where [[⋅]] are Iverson brackets.)
Member

### glouppe commented May 17, 2014

 `GradientBoostingRegressor` supports quantile regression (using `loss=quantile` and the `alpha` parameter). See http://scikit-learn.org/dev/auto_examples/ensemble/plot_gradient_boosting_quantile.html#example-ensemble-plot-gradient-boosting-quantile-py for an example.
Member

### larsmans commented May 18, 2014

 I should have checked that. Closing this issue.

### prcastro commented Aug 27, 2017 • edited

 While I don't agree that that aren't many packages for Quantile Regression on Python, I believe this is important to have pure Quantile Regression (not inside a Ensemble method) on sci-kit learn. Quantile Regression has the advantage of targeting on a specific quantile of y. With this, it's possible to reduce the difference of the median of y_pred and y. It's similar to minimizing the absolute error in this case, but it's much more general and work for other quantiles. Banks use this a lot when dealing with credit scoring and other models, so it's a battle tested model with real applications. R and SAS has this model implemented.
Member

### amueller commented Aug 29, 2017

 @prcastro do you mean for a linear model?

### prcastro commented Aug 30, 2017

 Exactly. Today, sklearn implements quantile regression on ensemble methods. However, it's usually used as a regular linear model.
Member

### amueller commented Aug 30, 2017

 I'd be open to adding it. @jnothman @GaelVaroquaux?
Member

### agramfort commented Aug 30, 2017

 It seems fairly established indeed
Member

### GaelVaroquaux commented Aug 30, 2017

 I'd be open to adding it. @jnothman @GaelVaroquaux? No opposition. We just need a good PR, and time to review it.

### atorch commented Sep 26, 2017

 Regarding the specific case of quantile regression for the median (absolute loss), as opposed to more general quantiles, it seems that http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.HuberRegressor.html would allow it if only we could pass epsilon=0.0. Why does HuberRegressor require `epsilon : float, greater than 1.0`? (`The Huber Regressor optimizes the squared loss for the samples where |(y - X'w) / sigma| < epsilon and the absolute loss for the samples where |(y - X'w) / sigma| > epsilon.`)
Member

### GaelVaroquaux commented Sep 26, 2017

 Huber loss with epsilon=0 is a non-smooth optimization problem: one cannot use the same class of solvers.

Open

### avidale commented Oct 22, 2017

 I added classical quantile linear regression in the pull request above. Please review it!

### avidale commented Apr 19, 2018

 Hello everyone! Please review my PR when you have time for it.

 ping