Feature request: include mean absolute error (MAE) as scorer in sklearn.linear_model.ElasticNetCV

[atorch]

Description

In scikit-learn v0.19.0, ElasticNet minimizes the objective function:

1 / (2 * n_samples) * ||y - Xw||^2_2
+ alpha * l1_ratio * ||w||_1
+ 0.5 * alpha * (1 - l1_ratio) * ||w||^2_2.

In words, the objective function is mean squared error (MSE) loss on the predicted variable, plus regularization terms. My feature request is to include an argument (with 'mse' as its default value) that would allow the user to switch to mean absolute error (MAE) loss on the predicted variable. For the equivalent in R, see https://cran.r-project.org/web/packages/glmnet/glmnet.pdf, particularly the documentation for cv.glmnet:

type.measure="mae" (mean absolute error) can be used by all models except ... "cox"

It would be great to also allow this functionality in sklearn.

Edit: as @agramfort pointed out below, glmnet only supports MAE for cross-validation, but not for model fitting. My request is to add an option for MAE in sklearn ElasticNet model fitting. This already exists in sklearn.ensemble.GradientBoostingRegressor (loss='quantile' and alpha=0.50), if I am reading the documentation correctly. There's also a linear-regression-with-MAE-loss option in sklearn.linear_model.HuberRegressor.html, if epsilon=0.0 ("The Huber Regressor optimizes the squared loss for the samples where |(y - X'w) / sigma| < epsilon and the absolute loss for the samples where |(y - X'w) / sigma| > epsilon").

[agramfort]

I think glmnet only supports MAE for scoring but not for fitting. Do I miss something?

[atorch]

@agramfort You're right, I didn't read the glmnet documentation closely enough. There's a type.measure="mae" argument to cv.glmnet, but not to glmnet (the actual model fitting function).

[jnothman]

Does that mean we should close this?

[agramfort]

yes or allow a scoring parameter in Linear model CV objects...

[atorch]

The feature I'm asking for would be similar to creating an elastic net CV for sklearn.linear_model.HuberRegressor.html (i.e. regularized Huber regression).

[agramfort]

sounds doable and useful but first we need to support elastic-net penalty in HuberRegressor. it's doable while sticking with L-BFGS-B or we need to write a fast coordinate descent code. It's a bit of work...

[atorch]

Ok. I'm glad you think it's doable and useful!

[agramfort]

now we "just" need someone to take a stab at it :)