Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add elastic net to the toolset #16

Closed
Banana1530 opened this issue Aug 13, 2018 · 2 comments
Closed

Add elastic net to the toolset #16

Banana1530 opened this issue Aug 13, 2018 · 2 comments
Labels
Penalties Issues / feature requests related to supported penalty functions

Comments

@Banana1530
Copy link
Collaborator

Banana1530 commented Aug 13, 2018

The elastic net [1] eases the problems caused by high correlation in the features space. However it is not of order 1 and thus violates the requirement.

[1] https://web.stanford.edu/~hastie/Papers/B67.2%20(2005)%20301-320%20Zou%20&%20Hastie.pdf Regularization and variable selection via the elastic net

@Banana1530 Banana1530 changed the title Add elastic net to the tool set Add elastic net to the toolset Aug 13, 2018
@michaelweylandt michaelweylandt added the Penalties Issues / feature requests related to supported penalty functions label Aug 14, 2018
@Banana1530
Copy link
Collaborator Author

It is not homogeneous of order 1.

@michaelweylandt
Copy link
Member

I think we can actually get by without homogeneity of order 1: e.g., the SCAD and MCP penalties are not PH1, but they are still useful.

If you look at the proofs closely (there's a new version of the ArXiv paper), PH1 is only needed to show that the prox + project step is actually implementing the prox of the sum of two functions and hence we can analyze the convergence as a prox gradient method. I'm pretty sure that we could instead consider a projected proximal gradient (i.e., prox gradient for problems with constraints) method instead, but a quick search doesn't pull anything up.

If we do implement EN, any thoughts on how to deal with double parameters?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Penalties Issues / feature requests related to supported penalty functions
Projects
None yet
Development

No branches or pull requests

2 participants