Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

da.eye fix for chunks=-1 #7854

Merged
merged 6 commits into from
Jul 2, 2021
Merged

da.eye fix for chunks=-1 #7854

merged 6 commits into from
Jul 2, 2021

Conversation

ncclementi
Copy link
Member

Fixes bug in da.eye case of chunks=-1.

@github-actions github-actions bot added the array label Jul 1, 2021
@ncclementi ncclementi changed the title Eye bug da.eye bug fix - case chunks=-1 Jul 1, 2021
@mrocklin
Copy link
Member

mrocklin commented Jul 1, 2021

Nice minimal fix 👍

Copy link
Member

@jrbourbeau jrbourbeau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ncclementi! Overall this looks great. Passing user-provided chunks through normalize_chunks is definitely the way to go.

Fortunately normalize_chunks has some of the same chunksize calculation logic that we've implemented here in da.eye. So since we're now always calling normalize_chunks, we can remove some code from da.eye which is now unnecessary:

diff --git a/dask/array/creation.py b/dask/array/creation.py
index 2e8b0eab..87cc715b 100644
--- a/dask/array/creation.py
+++ b/dask/array/creation.py
@@ -544,18 +544,12 @@ def eye(N, chunks="auto", M=None, k=0, dtype=float):
     if not isinstance(chunks, (int, str)):
         raise ValueError("chunks must be an int or string")

-    chunks = normalize_chunks(chunks, shape=(N, M), dtype=dtype)
-    chunks = chunks[0][0]
+    vchunks, hchunks = normalize_chunks(chunks, shape=(N, M), dtype=dtype)
+    chunks = vchunks[0]
+
     token = tokenize(N, chunks, M, k, dtype)
     name_eye = "eye-" + token

-    vchunks = [chunks] * (N // chunks)
-    if N % chunks != 0:
-        vchunks.append(N % chunks)
-    hchunks = [chunks] * (M // chunks)
-    if M % chunks != 0:
-        hchunks.append(M % chunks)
-
     for i, vchunk in enumerate(vchunks):
         for j, hchunk in enumerate(hchunks):
             if (j - i - 1) * chunks <= k <= (j - i + 1) * chunks:

Copy link
Member

@jrbourbeau jrbourbeau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ncclementi! Will merge after CI finishes up

@jrbourbeau
Copy link
Member

Hmm there's an unrelated daily stock test failure (which I'm able to reproduce locally). Given that the error message says

...
<h1 style="margin-top:20px;">Will be right back...</h1>\n      <p id="message-1">Thank you for your patience.</p>\n      <p id="message-2">Our engineers are working quickly to resolve the issue.</p>\n      </td>\n  </tr>\n  </tbody></table>\n  </body></html>'

I'll wait a bit and see if the test passes then

@jrbourbeau
Copy link
Member

Just merged main which should resolve the unrelated CI failure

@jrbourbeau jrbourbeau changed the title da.eye bug fix - case chunks=-1 da.eye fix for chunks=-1 Jul 2, 2021
@jrbourbeau jrbourbeau merged commit 2698634 into dask:main Jul 2, 2021
@ncclementi ncclementi deleted the eye_bug branch August 31, 2022 21:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Dask array eye key error
3 participants