Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: multiple-output ufuncs e.g. divmod #2654

Merged
merged 8 commits into from
Aug 17, 2023
Merged

Conversation

agoose77
Copy link
Collaborator

@agoose77 agoose77 commented Aug 17, 2023

It turns out that we didn't yet test the nout != 1 ufuncs, such as divmod.

>>> import awkward as ak
>>> import numpy as np
>>> array = ak.Array([
...     [1, 2, 3],
...     [4]
... ])
>>> np.divmod(array, 2)
[[[0, 1, 1, 2], [1, 0, 1, 0]], []]

This PR adds support for this ufunc, with some refactoring work along the way.

>>> import awkward as ak
>>> import numpy as np
>>> array = ak.Array([
...     [1, 2, 3],
...     [4]
... ])
>>> np.divmod(array, 2)
(<Array [[0, 1, 1], [2]] type='2 * var * int64'>, <Array [[1, 0, 1], [0]] type='2 * var * int64'>)

@agoose77 agoose77 changed the title fix: assume known length in length_of_broadcast fix: multiple-output ufuncs e.g. divmod Aug 17, 2023
@agoose77 agoose77 temporarily deployed to docs-preview August 17, 2023 12:39 — with GitHub Actions Inactive
@agoose77
Copy link
Collaborator Author

@jpivarski in this PR so-far, I removed the unary pathway for ufuncs. I can't see why this was introduced in 0e0202a

It might just be that our broadcasting code has changed since then to make the need redundant. Are you OK with the removal?

@agoose77 agoose77 temporarily deployed to docs-preview August 17, 2023 12:52 — with GitHub Actions Inactive
src/awkward/_broadcasting.py Show resolved Hide resolved
src/awkward/_broadcasting.py Show resolved Hide resolved
@agoose77 agoose77 marked this pull request as ready for review August 17, 2023 12:55
@agoose77 agoose77 temporarily deployed to docs-preview August 17, 2023 13:02 — with GitHub Actions Inactive
@codecov
Copy link

codecov bot commented Aug 17, 2023

Codecov Report

Merging #2654 (939c1b3) into main (61c3660) will decrease coverage by 0.02%.
The diff coverage is 100.00%.

Additional details and impacted files
Files Changed Coverage Δ
src/awkward/_broadcasting.py 95.43% <100.00%> (+0.52%) ⬆️
src/awkward/_connect/numpy.py 91.81% <100.00%> (-0.62%) ⬇️
src/awkward/operations/ak_transform.py 95.38% <100.00%> (+0.14%) ⬆️

... and 2 files with indirect coverage changes

Copy link
Member

@jpivarski jpivarski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for catching this! At one point, the return value for broadcasting actions had to be promoted from a single Content to a tuple of Contents, and I think it was because of ufuncs that return multiple values. But when that went it, it should have gone in with a test, like this one.

If you're done with this PR, go ahead and merge it.

Comment on lines -423 to -448
if sum(int(isinstance(x, ak.contents.Content)) for x in inputs) == 1:
where = None
for i, x in enumerate(inputs):
if isinstance(x, ak.contents.Content):
where = i
break
assert where is not None

nextinputs = list(inputs)

def unary_action(layout, **ignore):
nextinputs[where] = layout
result = action(tuple(nextinputs), **ignore)
if result is None:
return None
else:
assert isinstance(result, tuple) and len(result) == 1
return result[0]

out = ak._do.recursively_apply(
inputs[where],
unary_action,
behavior,
function_name=ufunc.__name__,
allow_records=False,
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jpivarski in this PR so-far, I removed the unary pathway for ufuncs. I can't see why this was introduced in 0e0202a

That commit is a PR merge; for finer granularity, the commit within that PR is

5b0a273#diff-e7b6632c2ca4228bf21505ced52be60d4fda8813e00cbaf65cd1c5d2e228d721R242-R263

It was accompanied by a large batch of single-line assertions, some of which have only one Content in the ufunc (such as + with one Content and one Python number).

Broadcasting a single layout should be equivalent to recursively_apply on that one layout. The fact that numerous tests were probably going through that and they passed either way is good evidence that the "should be" holds. (I don't see anything special about it, that it has to be there, and the context of the commit isn't reminding me of any gotchas.)

@agoose77 agoose77 merged commit f6b8e76 into main Aug 17, 2023
36 checks passed
@agoose77 agoose77 deleted the agoose77/fix-ufunc-divmod branch August 17, 2023 17:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants