Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

N-dimensional variables #198

Closed
SteveDiamond opened this issue Jun 22, 2015 · 23 comments
Closed

N-dimensional variables #198

SteveDiamond opened this issue Jun 22, 2015 · 23 comments
Labels

Comments

@SteveDiamond
Copy link
Member

@SteveDiamond SteveDiamond commented Jun 22, 2015

Currently cvxpy only supports scalar, vector, and matrix variables. Additional dimensions can be mimicked using a dict:

# Create a 3D k-by-m-by-n variable.
x = {}
for i in range(k):
  x[i] = Variable(m,n)
@marziacescon
Copy link

@marziacescon marziacescon commented Sep 11, 2017

Hey Steven!

I'm trying to translate an optimization problem I've written in matlab cvx into python cvxpy.
My optimization variable x is a column vector which appears in the objective function is the format x^TRx, where R is a square matrix. Now, I encounter problems in transposing the variable x. Specifically, I can't get it transposed. The code I'm using is the following:

import numpy as np
import cvxpy as cvx

create optimization variables

	x = cvx.Variable(nf_1) # a vector
            xTRx = np.dot(np.dot(np.transpose(x),R_ins),p_ins)

I'm aware of the fact that 1D array transposed returns itself, but I haven't been able to find a way around it.
Any help is appreciated.

@SteveDiamond
Copy link
Member Author

@SteveDiamond SteveDiamond commented Sep 11, 2017

Hi Marzia,

You can't ever use numpy functions on cvxpy objects. You have to use the cvxpy functions:

	x = cvx.Variable(nf_1) # a vector
            xTRx = cvx.quad_form(x, R_ins)
@rileyjmurray
Copy link
Collaborator

@rileyjmurray rileyjmurray commented Aug 22, 2018

@SteveDiamond it seems that adding support for this is pretty straightforward in 1.0.

expression.py doesn't actually impose the at-most-2d restriction; that's done in atom.py and leaf.py (note that restrictions imposed on Leaf objects are inherited by Variables, Constants, and Parameters).

When I removed these restrictions (on my fork) N-d Expressions worked with every atom I tested (all elementwise atoms, some affine atoms, some non-affine atoms). N-d Variables also behaved correctly, in the ways that I tested them.

@SteveDiamond
Copy link
Member Author

@SteveDiamond SteveDiamond commented Aug 23, 2018

That's great! I guess we did a better job than I thought with the refactor. We will need to add checks to certain operations like matrix multiplication, but it could be enough to simply ban N>2 in all relevant locations. We may not need any special case handling for N>2.

@rileyjmurray
Copy link
Collaborator

@rileyjmurray rileyjmurray commented Sep 8, 2018

Update: we should check to make sure that @ with cvxpy Expressions of constant value behaves in the same way as @ with numpy ndarrays of higher dimensions. Reason being: @ and np.dot behave differently for higher-dimensional arrays, and the original implementation of matmul in cvxpy might boil down to numpy's dot function, rather than the actual matmul function. (I haven't looked into this, but it seems like something that could easily happen.)

@vchinde
Copy link

@vchinde vchinde commented Jan 21, 2019

Hi Steve, I am new to cvxpy and python, I have created a three dimensional matrix using your suggestion.

Create a 3D k-by-m-by-n variable.

x = {}
for i in range(k):
x[i] = Variable(m,n)

I have problem slicing the 3D matrix. Like how to extract x(1, :, 1) (matlab notation) using python commands

@SteveDiamond
Copy link
Member Author

@SteveDiamond SteveDiamond commented Jan 21, 2019

You can't slice it. That's one advantage of truly supporting ND matrices.

@SteveDiamond
Copy link
Member Author

@SteveDiamond SteveDiamond commented Jan 21, 2019

Sorry, in your specific example you would do x[0][:,0]. But you can't slice the dictionary dimension.

@vchinde
Copy link

@vchinde vchinde commented Jan 21, 2019

I am running into errors if I do what you suggested

import cvxpy

N_sch=2
N_perhour=2

x = {}

for i in range(N_perhour):
x[i] = cvxpy.Variable(N_sch,19)

print(x[0][:,0])
Traceback (most recent call last):

File "", line 1, in
print(x[0][:,0])

File "C:\Users\vchinde\AppData\Local\Continuum\anaconda2\lib\site-packages\cvxpy\expressions\expression.py", line 315, in getitem
return cvxtypes.index()(self, key)

File "C:\Users\vchinde\AppData\Local\Continuum\anaconda2\lib\site-packages\cvxpy\atoms\affine\index.py", line 48, in init
self.key = ku.validate_key(key, expr.shape)

File "C:\Users\vchinde\AppData\Local\Continuum\anaconda2\lib\site-packages\cvxpy\utilities\key_utils.py", line 49, in validate_key
raise IndexError("Too many indices for expression.")

IndexError: Too many indices for expression.

@SteveDiamond
Copy link
Member Author

@SteveDiamond SteveDiamond commented Jan 21, 2019

It should be x[i] = cvxpy.Variable((N_sch,19)). Writing cvxpy.Variable(N_sch,19) actually makes a variables with dimensions (N,) and raises an error in the current version, but you may not have that version on windows.

@vchinde
Copy link

@vchinde vchinde commented Jan 21, 2019

Thanks Steve. I got it

@vchinde
Copy link

@vchinde vchinde commented Jan 29, 2019

Hi Steve, I have the following question

m_z=cvxpy.Variable(shape=(24, 19))
for i_sch in xrange(24):
m_i=m_z[i_sch,:]
constr +=[Simconst['m_min'] <= m_i]

The dimension of Simconst['m_min'] is (19L, 1L). When I run the code I get the following error "ValueError: Cannot broadcast dimensions (19, 1) (19,)". How to overcome this error? I did tried using m_i.T but has the same error

@SteveDiamond
Copy link
Member Author

@SteveDiamond SteveDiamond commented Feb 1, 2019

Write Simconst['m_min'][:,0] <= m_i

@rotcx
Copy link

@rotcx rotcx commented Sep 11, 2019

cvxpy.Variable(dim1,dim2)

some of the (dim1_index,dim2_index) combinations are meaning less

how can we "do not define variables on those meaningless combinations" instead of "define variables on the whole combination, then use constraints to fix the meaningless variables "
?

@rotcx
Copy link

@rotcx rotcx commented Sep 11, 2019

Another question:
how can I vectorize constraints instead of using for-cycling clauses? Tedious when the problem is large scale.

@SteveDiamond
Copy link
Member Author

@SteveDiamond SteveDiamond commented Sep 11, 2019

The simplest thing is to make a variable vector x representing all the meaningful combinations, then expand that to a sparse matrix using cvxpy.reshape(A@x, (dim1, dim2)), where A is a dim1*dim2 by T matrix, where T is the number of meaningful combinations. You can figure out what the entries of A should be. That exercise will hopefully teach you how to vectorize.

@carlosgmartin
Copy link

@carlosgmartin carlosgmartin commented Feb 13, 2020

Will multidimensional variables ever be added to cvxpy?

@SteveDiamond
Copy link
Member Author

@SteveDiamond SteveDiamond commented Feb 13, 2020

Not any time soon.

@rotcx
Copy link

@rotcx rotcx commented Feb 29, 2020

@rileyjmurray
Copy link
Collaborator

@rileyjmurray rileyjmurray commented Feb 29, 2020

@chenxuanhanhao thanks for letting us know. I'm curious: what's your application where the high-level logic offered by python makes it unrealistic to work only with 1d or 2d arrays?

@teichert
Copy link

@teichert teichert commented Aug 14, 2020

Thanks for cvxpy! With the uses that cvxpylayers is likely to see, I'm sure higher order tensors are going to be increasingly common (e.g. anything you wanted to do with a matrix before, you might want to do batched). In my current use case, I want to find a tensor that is close to another tensor (in the sense of kl_div) while respecting a number of constraints that deal with marginalizing out various subsets of the dimensions. (number of dimensions can be between 2 and 20.)

Allowing cp.sum to work on higher-order ndarrays would satisfy my current use case (even if I am only allowed to sum over one dimension at a time). Alternatively, if just cp.transpose and cp.reshape allowed higher order ndarray, I'm sure that would facilitate easier work-arounds in many cases.

@SteveDiamond
Copy link
Member Author

@SteveDiamond SteveDiamond commented Aug 16, 2020

You can transpose or reshape outside of cvxpy, in pytorch/tensorflow.

@teichert
Copy link

@teichert teichert commented Aug 17, 2020

You can transpose or reshape outside of cvxpy, in pytorch/tensorflow.

Thanks; that's right, and I can do fancy indexing in order to sum out "dimensions" even if my representation in cvxpy is flat. In my case, the constraints I needed to add were easy to express as summations on subsets of dimensions, but difficult to write in terms of the flat representation. So, what I ended up doing was to make my own sum function that took in a flattened ndarray (i.e. transposing and flattening having already been applied outside of cvxpy), a shape tuple, and a set of dimensions to sum over and then (within my sum function) I did the indexing (via np.ravel_multi_index, np.ix_, ndarray.transpose, and ndarray.reshape) to achieve the equivalent of multi-dimensional summations on a flat cvxpy array.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
7 participants