Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

numpy.concat does not appear to work across broadcast axes (Trac #1518) #2115

Closed
thouis opened this issue Oct 19, 2012 · 11 comments
Closed

numpy.concat does not appear to work across broadcast axes (Trac #1518) #2115

thouis opened this issue Oct 19, 2012 · 11 comments

Comments

@thouis
Copy link
Contributor

@thouis thouis commented Oct 19, 2012

Original ticket http://projects.scipy.org/numpy/ticket/1518 on 2010-06-22 by trac user eob, assigned to unknown.

When I'm trying to concatenate two tensors together, the concatenate operation does not allow me to use broadcast dimensions (using newaxis) in one of them.

@thouis
Copy link
Contributor Author

@thouis thouis commented Oct 19, 2012

trac user eob wrote on 2010-06-22

Note: the workaround we are using is to tile the numpy.array that we wanted to use newaxis on, which works but at the expense of memory.

@thouis
Copy link
Contributor Author

@thouis thouis commented Oct 19, 2012

@pv wrote on 2010-06-24

Can you write a self-contained piece of example code showing what you are trying to do, what you would like to happen, and what actually happens.

From your description, it's not completely clear to me how you'd like things to work.

@thouis
Copy link
Contributor Author

@thouis thouis commented Oct 19, 2012

@pv wrote on 2010-06-24

If you mean this,

>>> x = np.array([1,2,3,4])
>>> y = np.array([[1,2],[3,4],[5,6],[7,8]])
>>> np.concatenate((y, np.tile(x, (4,1))), axis=1)
array([[1, 2, 1, 2, 3, 4],
       [3, 4, 1, 2, 3, 4],
       [5, 6, 1, 2, 3, 4],
       [7, 8, 1, 2, 3, 4]])

you can do it without copies:

def broadcast_view(x, ref):
    """Broadcast unit dimensions in `x` to match those in `ref` without copies"""
    strides = [0 if x.shape[j] == 1 else x.strides[j] for j in range(x.ndim)]
    shape = [ref.shape[j] if x.shape[j] == 1 else x.shape[j] for j in range(x.ndim)]
    from numpy.lib.stride_tricks import as_strided
    return as_strided(x, shape=shape, strides=strides)

>>> x = np.array([1,2,3,4])
>>> y = np.array([[1,2],[3,4],[5,6],[7,8]])
>>> np.concatenate((y, broadcast_view(x[None,:], y)), axis=1)
array([[1, 2, 1, 2, 3, 4],
       [3, 4, 1, 2, 3, 4],
       [5, 6, 1, 2, 3, 4],
       [7, 8, 1, 2, 3, 4]])
>>> broadcast_view(x[None,:], y).base.base.base is x
True

But yes, I guess np.concatenate should do this automatically.

@jluttine
Copy link

@jluttine jluttine commented Mar 14, 2015

Any news/progress on np.concatenate supporting broadcasting?

@jaimefrio
Copy link
Member

@jaimefrio jaimefrio commented Mar 14, 2015

There has been some discussions about this on the list in the past, and the collective wisdom dictated that broadcasting in a concatenate operation was more likely to be masking unintended mistakes, than a useful functionality. We probably should close this.

In any case, if you would like to stirr that conversation up again, the right place would be the mailing list, not here.

@jaimefrio
Copy link
Member

@jaimefrio jaimefrio commented Mar 27, 2016

Should this be closed as a "won't fix"?

@shoyer
Copy link
Member

@shoyer shoyer commented Mar 27, 2016

Agreed. Note that manual broadcasting is now a little bit easier because we have broadcast_to.

@jaimefrio
Copy link
Member

@jaimefrio jaimefrio commented Mar 27, 2016

I just needed a tiny bit of positive reinforcement, closed it is. If anyone has a strong opinion against this, feel free to reopen.

@jaimefrio jaimefrio closed this Mar 27, 2016
@davidmashburn
Copy link

@davidmashburn davidmashburn commented Jul 8, 2016

I found a solution that seems solid to me... just use broadcast_arrays in the form np.concatenate(np.broadcast_arrays(*arrays), axis=...). Ex: np.concatenate(np.broadcast_arrays(0, [1, 2], [[3, 4], [5, 6]]), axis=1)array([[0, 0, 1, 2, 3, 4], [0, 0, 1, 2, 5, 6]])

@oulenz
Copy link

@oulenz oulenz commented Mar 9, 2020

@davidmashburn I think there is a problem with this solution (and with manual broadcasting in general as suggested by others in this thread). I don't think it makes sense to broadcast in the dimension in which we concatenate. In other words, in your case I would only expect to end up with one column of zeros.

@boeddeker
Copy link
Contributor

@boeddeker boeddeker commented Apr 6, 2020

If someone else also finds this Issue, while searching for a solution of broadcasting in concatenate, I posted my solution at https://stackoverflow.com/questions/56357047/concatenate-with-broadcast/61061019#61061019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
7 participants