-
Notifications
You must be signed in to change notification settings - Fork 0
/
info.json
27 lines (27 loc) · 1.86 KB
/
info.json
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
{
"title": "On lp-hyperparameter Learning via Bilevel Nonsmooth Optimization",
"abstract": "We propose a bilevel optimization strategy for selecting the best hyperparameter value for the nonsmooth $\\ell_p$ regularizer with $0<p\\leq 1$. The concerned bilevel optimization problem has a nonsmooth, possibly nonconvex, $\\ell_p$-regularized problem as the lower-level problem. Despite the recent popularity of nonconvex $\\ell_p$-regularizer and the usefulness of bilevel optimization for selecting hyperparameters, algorithms for such bilevel problems have not been studied because of the difficulty of $\\ell_p$-regularizer. Our contribution is the proposal of the first algorithm equipped with a theoretical guarantee for finding the best hyperparameter of $\\ell_p$-regularized supervised learning problems. Specifically, we propose a smoothing-type algorithm for the above mentioned bilevel optimization problems and provide a theoretical convergence guarantee for the algorithm. Indeed, since optimality conditions are not known for such bilevel optimization problems so far, new necessary optimality conditions, which are called the SB-KKT conditions, are derived and it is shown that a sequence generated by the proposed algorithm actually accumulates at a point satisfying the SB-KKT conditions under some mild assumptions. The proposed algorithm is simple and scalable as our numerical comparison to Bayesian optimization and grid search indicates.",
"authors":
[
"Takayuki Okuno",
"Akiko Takeda",
"Akihiro Kawana",
"Motokazu Watanabe"
],
"emails":
[
"takayuki.okuno.ks@riken.jp",
"takeda@mist.i.u-tokyo.ac.jp",
"kawana.ak.pp@gmail.com",
"mwatanabe@g.ecc.u-tokyo.ac.jp"
],
"id": "18-485",
"issue": 245,
"pages":
[
1,
47
],
"volume": 22,
"year": 2021
}