A multi-convex optimization problem is one in which the variables can be partitioned into sets, over each of which the problem is convex when the other variables are fixed. It is generally a nonconvex problem. DMCP package provides methods to verify multi-convexity and to find minimal sets of variables that have to be fixed for the problem to be convex, as well as an organized heuristic for multi-convex programming. The full details of our approach are discussed in the associated paper. DMCP is built on top of CVXPY, a domain-specific language for convex optimization embedded in Python.
You should first install CVXPY 1.0. Then clone the repository and run
python setup.py install inside.
Consider an optimization problem
minimize f_0(x) subject to f_i(x) <= 0, i = 1,...,m g_i(x) = 0, i = 1,...,p,
x admits a partition of blocks of variables
x = (x_1,...,x_N), and functions
i = 0,...,m and
i = 1,...,p are proper.
Given a set of DCP atomic functions and its extension of multi-convex atomic functions,
the problem can be specified as disciplined multi-convex programming (DMCP), if there are index sets
F_1,..., F_K, such that their intersection is empty, and for every
k the problem with variables
x_i for all
i in set
F_k fixed to any value can be specified as DCP with respect to the DCP atoms set.
The following code uses DMCP to approximately solve a simple multi-convex problem.
import cvxpy as cvx import dmcp x_1 = cvx.Variable() x_2 = cvx.Variable() x_3 = cvx.Variable() x_4 = cvx.Variable() objective = cvx.Minimize(abs(x_1*x_2+x_3*x_4)) constraint = [x_1+x_2+x_3+x_4 == 1] myprob = cvx.Problem(objective, constraint) print "minimal sets:", dmcp.find_minimal_sets(myprob) # find all minimal sets print "problem is DCP:", myprob.is_dcp() # false print "problem is DMCP:", dmcp.is_dmcp(myprob) # true result = myprob.solve(method = 'bcd')
The output of the above code is as follows.
minimal sets: [[1, 3], [1, 2], [0, 3], [0, 2]] problem is DCP: False problem is DMCP: True maximum value of slack variables: 1.15081491391e-05 objective value: 1.74866042578e-05
The solutions obtained by DMCP can depend on the initial point.
The algorithm starts from the values of any variables that are already specified;
for any that are not specified, random values are used.
It is suggested that users set reasonable initial values for all variables,
which can be done by manually setting the
value field of the problem variables.
x_1.value = 1.2 x_2.value = -3 x_3.value = 4 x_4.value = 0.15 result = myprob.solve(method = 'bcd')
More examples can be found [here] (https://github.com/cvxgrp/dmcp/tree/master/examples).
Multi-convex atomic functions
In order to allow multi-convex functions, we extend the atomic function set of
The following atoms are allowed to have non-constant expressions in both arguments, while in the dictionary of
CVXPY the first argument must be constant.
expression1 * expression2
- elementwise multiplication:
Functions and attributes
is_dmcp(problem)returns a boolean indicating if an optimization problem satisfies DMCP rules.
find_minimal_sets(problem)analyzes the problem and returns a list of minimal sets of (indexes of) variables. The indexes are with respect to the list
problem.variables(), namely the variable corresponding to the index
problem.variables(). If the problem is DCP, it returns an empty list.
is_all = Truewill generate all minimal sets.
fix(obj, fix_vars)returns a new expression or a new problem with the variables in the list
fix_varsreplaced with parameters of the same dimensions and signs. The
objcan either be an expression or a problem.
Constructing and solving problems
The components of the variable, the objective, and the constraints are constructed using standard CVXPY syntax. Once the user has constructed a problem object, they can apply the following solve method:
problem.solve(method = 'bcd')applies the solving algorithm with proximal operators, and returns the number of iterations, and the maximum value of the slack variables. The solution to every variable is in its
problem.solve(method = 'bcd', update = 'minimize')applies the solving method without proximal operators.
problem.solve(method = 'bcd', update = 'prox_linear')applies the solving method with prox-linear operators.
Additional arguments can be used to specify the parameters.
Solve method parameters:
solverparameter specifies what solver to use to solve convex subproblems.
max_iterparameter sets the maximum number of iterations in the algorithm. The default is 100.
muparameter trades off satisfying the constraints and minimizing the objective. Larger
mufavors satisfying the constraints. The default is 0.001.
rhoparameter sets the rate at which
muincreases inside the algorithm. The default is 1.2.
mu_maxparameter upper bounds how large
mucan get. The default is 1e4.
lambdparameter is the parameter in the proximal operator. The default is 10.
If the convex solver for the subproblems accepts any additional keyword arguments,
then the user can set them in the
problem.solve() function, and they will be passed to the convex solver.