Skip to content

da.eye fix for chunks=-1#7854

Merged
jrbourbeau merged 6 commits intodask:mainfrom
ncclementi:eye_bug
Jul 2, 2021
Merged

da.eye fix for chunks=-1#7854
jrbourbeau merged 6 commits intodask:mainfrom
ncclementi:eye_bug

Conversation

@ncclementi
Copy link
Copy Markdown
Member

Fixes bug in da.eye case of chunks=-1.

@github-actions github-actions bot added the array label Jul 1, 2021
@ncclementi ncclementi changed the title Eye bug da.eye bug fix - case chunks=-1 Jul 1, 2021
@mrocklin
Copy link
Copy Markdown
Member

mrocklin commented Jul 1, 2021

Nice minimal fix 👍

Copy link
Copy Markdown
Member

@jrbourbeau jrbourbeau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ncclementi! Overall this looks great. Passing user-provided chunks through normalize_chunks is definitely the way to go.

Fortunately normalize_chunks has some of the same chunksize calculation logic that we've implemented here in da.eye. So since we're now always calling normalize_chunks, we can remove some code from da.eye which is now unnecessary:

diff --git a/dask/array/creation.py b/dask/array/creation.py
index 2e8b0eab..87cc715b 100644
--- a/dask/array/creation.py
+++ b/dask/array/creation.py
@@ -544,18 +544,12 @@ def eye(N, chunks="auto", M=None, k=0, dtype=float):
     if not isinstance(chunks, (int, str)):
         raise ValueError("chunks must be an int or string")

-    chunks = normalize_chunks(chunks, shape=(N, M), dtype=dtype)
-    chunks = chunks[0][0]
+    vchunks, hchunks = normalize_chunks(chunks, shape=(N, M), dtype=dtype)
+    chunks = vchunks[0]
+
     token = tokenize(N, chunks, M, k, dtype)
     name_eye = "eye-" + token

-    vchunks = [chunks] * (N // chunks)
-    if N % chunks != 0:
-        vchunks.append(N % chunks)
-    hchunks = [chunks] * (M // chunks)
-    if M % chunks != 0:
-        hchunks.append(M % chunks)
-
     for i, vchunk in enumerate(vchunks):
         for j, hchunk in enumerate(hchunks):
             if (j - i - 1) * chunks <= k <= (j - i + 1) * chunks:

Copy link
Copy Markdown
Member

@jrbourbeau jrbourbeau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ncclementi! Will merge after CI finishes up

@jrbourbeau
Copy link
Copy Markdown
Member

Hmm there's an unrelated daily stock test failure (which I'm able to reproduce locally). Given that the error message says

...
<h1 style="margin-top:20px;">Will be right back...</h1>\n      <p id="message-1">Thank you for your patience.</p>\n      <p id="message-2">Our engineers are working quickly to resolve the issue.</p>\n      </td>\n  </tr>\n  </tbody></table>\n  </body></html>'

I'll wait a bit and see if the test passes then

@jrbourbeau
Copy link
Copy Markdown
Member

Just merged main which should resolve the unrelated CI failure

@jrbourbeau jrbourbeau changed the title da.eye bug fix - case chunks=-1 da.eye fix for chunks=-1 Jul 2, 2021
@jrbourbeau jrbourbeau merged commit 2698634 into dask:main Jul 2, 2021
@ncclementi ncclementi deleted the eye_bug branch August 31, 2022 21:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Dask array eye key error

3 participants