[FEA] Support L1 regularization and ElasticNet in MNMG Dask LogisticRegression

[lijinf2]

This PR depends on PR5565.

[csadorf]

@dantegd Ok to change the base to #5565 to simplify review?

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[lijinf2]

@csadorf Thanks for the review! Just submitted the revised PR. Please help take a look.

[cjnolet]

@lijinf2 I'm sorry for being late to this review but LBFGS expects differentiable objectives and the lasso objective, while being convex, is not differentiable everywhere.

The owl-qn method is an extension of lbfgs that supports lasso. The single-gpu lasso-logistic uses owl-qn and we should be doing the same for the distributed version as well.

[lijinf2]

@lijinf2 I'm sorry for being late to this review but LBFGS expects differentiable objectives and the lasso objective, while being convex, is not differentiable everywhere.

The owl-qn method is an extension of lbfgs that supports lasso. The single-gpu lasso-logistic uses owl-qn and we should be doing the same for the distributed version as well.

Thanks for helping look at the PR! The distributed version has been modified to reuse single-gpu qn_minimize instead of single-gpu min_lbfgs in a recent PR #5558. As a result, the qn_minimize will automatically choose min_owlqn when l1 penalty is non zero, and min_lbfgs when l1 penalty is zero.

In my local pytest verbose logs, "qn_solvers.cuh:323 Running OWL-QN" was printed out in all test cases of test_l1 and test_elasticnet, while "qn_solvers.cuh:180 Running L-BFGS" was printed out in all test cases of test_lbfgs (penalty is l2) and test_noreg (penalty is none). So it seems the distributed version has been doing the same as single-gpu.

Let me know if you meant to revise the code or test cases!

[cjnolet]

In my local pytest verbose logs, "qn_solvers.cuh:323 Running OWL-QN" was printed out in all test cases of test_l1

This is perfect. Thanks @lijinf2!

[csadorf]

One suggestion for improvement of the test code, but LGTM!

[csadorf]

/merge