Skip to content

Fix map_overlap trimming behavior when drop_axis is not None#7894

Merged
jsignell merged 6 commits intodask:mainfrom
grlee77:fix-map-overlap-trim
Aug 5, 2021
Merged

Fix map_overlap trimming behavior when drop_axis is not None#7894
jsignell merged 6 commits intodask:mainfrom
grlee77:fix-map-overlap-trim

Conversation

@grlee77
Copy link
Copy Markdown
Contributor

@grlee77 grlee77 commented Jul 14, 2021

This is a bug fix PR

It regenerates the depth and boundary variables, taking drop_axis into account before calling trim_internal

@github-actions github-actions bot added the array label Jul 14, 2021
@GenevieveBuckley
Copy link
Copy Markdown
Contributor

@jakirkham could you (or someone else) please start the CI workflows? Thanks!



def test_map_overlap_trim_using_drop_axis_and_different_depths():
x = da.ones((5, 10), dtype=float)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to see some non-2D examples in the tests, as well

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay. I should also add a non-uniform boundary

# note that keys are relabeled to match values in range(x.ndim)
x_depth = {n: x_depth[ax] for n, ax in enumerate(x_axis)}
x_boundary = {n: x_boundary[ax] for n, ax in enumerate(x_axis)}
return trim_internal(x, x_depth, x_boundary)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like the x_ prefix on depth, axis and boundary is more confusing than not.

@jakirkham
Copy link
Copy Markdown
Member

@jakirkham could you (or someone else) please start the CI workflows? Thanks!

Yep done :)

grlee77 added 2 commits July 14, 2021 23:25
vary both depth and boundary across axes

test many drop_axis combinations

test values via assert_array_almost_equal
@grlee77
Copy link
Copy Markdown
Contributor Author

grlee77 commented Jul 15, 2021

I have updated the test cases here.

One other issue I noticed when developing the test cases is that the use of negative values such as drop_axis=(-1,) to indicate dropping the last axis does not work. Let me know if you want me to try and fix that in this same PR.

@GenevieveBuckley
Copy link
Copy Markdown
Contributor

@jakirkham could you (or someone else) please start the CI workflows? Thanks!

Yep done :)

Huh, that's weird. On my end I still see " 4 workflows awaiting approval. First-time contributors need a maintainer to approve running workflows."

@grlee77
Copy link
Copy Markdown
Contributor Author

grlee77 commented Jul 15, 2021

Huh, that's weird. On my end I still see " 4 workflows awaiting approval. First-time contributors need a maintainer to approve running workflows."

I think that is because I pushed new commits afterwards. Annoyingly, you have to approve every time

@GenevieveBuckley
Copy link
Copy Markdown
Contributor

One other issue I noticed when developing the test cases is that the use of negative values such as drop_axis=(-1,) to indicate dropping the last axis does not work. Let me know if you want me to try and fix that in this same PR.

This seems like an important thing to fix, but probably makes better sense to put in a new PR. (Bundling stuff together makes it kind of hard to look back through the commit log at the PR titles and figure out when something was introduced)

@grlee77
Copy link
Copy Markdown
Contributor Author

grlee77 commented Jul 15, 2021

I think the issue with negative values in drop_axis is not exclusive to the newly added code. The following line in the underlying map_blocks function seems to assume the axes are non-negative:

out_ind = tuple(x for i, x in enumerate(out_ind) if i not in drop_axis)

@jakirkham
Copy link
Copy Markdown
Member

@jakirkham could you (or someone else) please start the CI workflows? Thanks!

Yep done :)

Huh, that's weird. On my end I still see " 4 workflows awaiting approval. First-time contributors need a maintainer to approve running workflows."

Sadly each new commit needs to be approved

@grlee77
Copy link
Copy Markdown
Contributor Author

grlee77 commented Jul 22, 2021

I fixed a bug in the test case. can someone approve the workflows again here?

@jsignell
Copy link
Copy Markdown
Member

It seems like this is good to go. Is that right @GenevieveBuckley?

@jsignell
Copy link
Copy Markdown
Member

jsignell commented Aug 5, 2021

This looks great! Thanks so much for the contribution @grlee77!!

@GPUtester
Copy link
Copy Markdown
Collaborator

Can one of the admins verify this patch?

@jsignell jsignell merged commit bf6a22a into dask:main Aug 5, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

map_overlap does not always trim properly when drop_axis is not None

5 participants