Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added NotImplemented error to Array.concatenate #508

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
+8 −0
Open

Added NotImplemented error to Array.concatenate #508

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
37ff455
Added NotImplemented error to Array.concatenate
panchoop Aug 5, 2021
940cfcf
fixed syntax error in Error message
panchoop Aug 11, 2021
33e83dd
fixed Flake8 errors
panchoop Aug 12, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
8 changes: 8 additions & 0 deletions pyopencl/array.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2604,11 +2604,19 @@ def make_func_for_chunk_size(chunk_size):

def concatenate(arrays, axis=0, queue=None, allocator=None):
"""
Return a :class:`Array` that is a concatenation of the input tuple of
:class:`Array` along :arg axis:. **Warning** The current implementation
allows only concatenation of row-major arrays along axis = 0.

.. versionadded:: 2013.1
"""
# {{{ find properties of result array

shape = None
if axis != 0:
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't correct. This should instead be based on contiguity. Column-major arrays for example allow concatenation along the last axis.

Copy link
Author

@panchoop panchoop Aug 10, 2021 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed that the code failed the tests because I missed a syntax error. Sorry for the noise.

About contiguity, I did some tests, and the function does not work in any axis for colum-major arrays. In line 2571 of the Array.concatenate function, the output array result is always a row-major array, leading to the same NotImlementedError due to miss-matching strides.

Simple example:

import pyopencl as cl
platform = cl.get_platforms()[0]
device = platform.get_devices()[0]
context = cl.create_some_context([device])
queue = cl.CommandQueue(context)

a = np.random.rand(2,2)
b = np.random.rand(2,2)

a_cl = clarray.to_device(queue, np.require(a, np.float64, 'F'))
b_cl = clarray.to_device(queue, np.require(b, np.float64, 'F'))

c_cl = clarray.concatenate( (a_cl, b_cl), axis=1)

It shouldn't be hard to allow column-major arrays, it is just a matter of initializing the result array with order="F" in these cases. The issue I see is that the Array class has no member that indicates the ordering. One could deduce the order by looking at the strides (if Array.strides[0] > Array.strides[-1]: row-major, else: column-major) but maybe it is too hacky. what do you think?

Another approach is the one taken in the Array.stack function, in which when the axis != 0, it is assumed that the input is column mayor. But it would also lead to unclear error messages.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for pointing out the issue with concatenate and column-major arrays, we should look into fixing that. I agree with your assessment about inspection of the strides, it's not quite elegant.

FWIW, CI is still not passing on this branch.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh yeah! I forgot to push. It should work now.

raise NotImplementedError("Axis != 0. "
+ "To be implemented when Array.setitems "
+ "allows values with different stride.")

for i_ary, ary in enumerate(arrays):
queue = queue or ary.queue
Expand Down