TypeError when calling SCS solver

[duocheng1999]

Describe the bug
encountered typeerror when I called scs solver to solve a constrained convex optimization problem. the error happens only after a certain large amount of iterations, even if I change the dataset I use (which is related to the vector 'cum_loss_est'), I still encounter such issue.

To Reproduce

for run in range(num_rep):
    cum_loss_est = np.zeros(K)
    weights = np.zeros(K)
    for t in range(T):
        w = cp.Variable(K, pos=True)
        obj = cp.Minimize(temp_lr * (cum_loss_est.T  @ w)  - cp.sum(cp.entr(w)) - (2*temp_lr) * cp.sum(cp.log(w)) )
        cons = [cp.sum(w)==1]
        prob = cp.Problem(obj,cons)
        prob.solve(solver='SCS')

Expected behavior
A clear and concise description of what you expected to happen.

Output

TypeError Traceback (most recent call last)
/var/folders/p7/mx9tw9892wl08vsqw48pzn600000gn/T/ipykernel_1552/865073087.py in
13 cons = [cp.sum(w)==1]
14 prob = cp.Problem(obj,cons)
---> 15 prob.solve(solver='SCS')
16 dec = np.random.choice(K,size=1,p=w.value/sum(w.value))
17 dec = dec[0]

/opt/anaconda3/lib/python3.9/site-packages/cvxpy/problems/problem.py in solve(self, *args, **kwargs)
501 else:
502 solve_func = Problem._solve
--> 503 return solve_func(self, *args, **kwargs)
504
505 @classmethod

/opt/anaconda3/lib/python3.9/site-packages/cvxpy/problems/problem.py in _solve(self, solver, warm_start, verbose, gp, qcp, requires_grad, enforce_dpp, ignore_dpp, canon_backend, **kwargs)
1070 return self.value
1071
-> 1072 data, solving_chain, inverse_data = self.get_problem_data(
1073 solver, gp, enforce_dpp, ignore_dpp, verbose, canon_backend, kwargs
1074 )

/opt/anaconda3/lib/python3.9/site-packages/cvxpy/problems/problem.py in get_problem_data(self, solver, gp, enforce_dpp, ignore_dpp, verbose, canon_backend, solver_opts)
694 'Compiling problem (target solver=%s).', solver_name)
695 s.LOGGER.info('Reduction chain: %s', reduction_chain_str)
--> 696 data, inverse_data = solving_chain.apply(self, verbose)
697 safe_to_cache = (
698 isinstance(data, dict)

/opt/anaconda3/lib/python3.9/site-packages/cvxpy/reductions/chain.py in apply(self, problem, verbose)
74 if verbose:
75 s.LOGGER.info('Applying reduction %s', type(r).name)
---> 76 problem, inv = r.apply(problem)
77 inverse_data.append(inv)
78 return problem, inverse_data

/opt/anaconda3/lib/python3.9/site-packages/cvxpy/reductions/dcp2cone/cone_matrix_stuffing.py in apply(self, problem)
333 # Form the constraints
334 extractor = CoeffExtractor(inverse_data, self.canon_backend)
--> 335 params_to_P, params_to_c, flattened_variable = self.stuffed_objective(
336 problem, extractor)
337 # Lower equality and inequality to Zero and NonNeg.

/opt/anaconda3/lib/python3.9/site-packages/cvxpy/reductions/dcp2cone/cone_matrix_stuffing.py in stuffed_objective(self, problem, extractor)
325 else:
326 # Extract to c.T * x + r; c is represented by a ma
--> 327 params_to_c = extractor.affine(problem.objective.expr)
328 params_to_P = None
329 return params_to_P, params_to_c, x

/opt/anaconda3/lib/python3.9/site-packages/cvxpy/utilities/coeff_extractor.py in affine(self, expr)
97 num_rows = sum([e.size for e in expr_list])
98 op_list = [e.canonical_form[0] for e in expr_list]
---> 99 return canonInterface.get_problem_matrix(op_list,
100 self.x_length,
101 self.id_map,

/opt/anaconda3/lib/python3.9/site-packages/cvxpy/cvxcore/python/canonInterface.py in get_problem_matrix(linOps, var_length, id_to_col, param_to_size, param_to_col, constr_length, canon_backend)
323 id_to_col_C = cvxcore.IntIntMap()
324 for id, col in id_to_col.items():
--> 325 id_to_col_C[int(id)] = int(col)
326
327 param_to_size_C = cvxcore.IntIntMap()

/opt/anaconda3/lib/python3.9/site-packages/cvxpy/cvxcore/python/cvxcore.py in setitem(self, *args)
736
737 def setitem(self, *args) -> "void":
--> 738 return cvxcore.IntIntMap___setitem_(self, *args)
739
740 def asdict(self) -> "PyObject *":

TypeError: Wrong number or type of arguments for overloaded function 'IntIntMap___setitem__'.
Possible C/C++ prototypes are:
std::map< int,int >::setitem(std::map< int,int >::key_type const &)
std::map< int,int >::setitem(std::map< int,int >::key_type const &,std::map< int,int >::mapped_type const &)

Version

• OS: Mac OS
• CVXPY Version: 1.4.1

Additional context
the iterations of the for loop is quite large. could this be a potential reason?

[phschiele]

Hi @duocheng1999!
Does it work with other solvers?
What is the range of values here (iterations, K)?
Can you post a full runnable example that leads to the crash?

[duocheng1999]

Hi @duocheng1999! Does it work with other solvers? What is the range of values here (iterations, K)? Can you post a full runnable example that leads to the crash?

Hi, I haven't tried any other solvers yet. The num_rep is 50 and T is around 10^6. Dimension K is between 6-8. Will post a full runnable example soon!

[phschiele]

@duocheng1999 Do you still want to provide the runnable example here, or has this been resolved?

[duocheng1999]

yes thanks for the follow up

import numpy as np
import cvxpy as cp
temp_lr = 1.0/15/8.0
num_rep = 50
T = 10**6
cum_loss_est = np.array([0.0, 1.0,2.0,3.0,4.0,5.0,6.0,7.0])
for run in range(num_rep):
    for t in range(T):
        w = cp.Variable(8, pos=True)
        obj = cp.Minimize(temp_lr * (cum_loss_est.T  @ w)  - cp.sum(cp.entr(w)) - (2*temp_lr) * cp.sum(cp.log(w)) )
        cons = [cp.sum(w)==1]
        prob = cp.Problem(obj,cons)
        prob.solve(solver='SCS')