Faster and simpler iris.util.array_equal

[bouweandela]

🚀 Pull Request

Description

The function iris.util.array_equal realizes the data multiple times, instead of just once. This means that data is pulled from disk multiple times, which is slower than necessary. This pull request simplifies the code so it lets Dask figure out what the best way to handle the computation is.

---

Consult Iris pull request check list

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (db43fde) 89.68% compared to head (6bcc078) 89.69%.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #5610   +/-   ##
=======================================
  Coverage   89.68%   89.69%           
=======================================
  Files          90       90           
  Lines       22818    22807   -11     
  Branches     5446     5441    -5     
=======================================
- Hits        20465    20456    -9     
+ Misses       1619     1618    -1     
+ Partials      734      733    -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[pp-mo]

Great, this is a really useful simplification

• I must say I found the old code hard to grasp, though I think I originally wrote it !

[pp-mo]

Thanks @bouweandela a definite improvement !

And this really shows how the array API dispatch improves things -- so the new code is no longer concerned at all whether data is lazy or not.

However, it does rely on the odd fact that bool() forces computation of lazy arrays.
Out of interest, do you know why that is ?
I have never quite understood this, since I never found any explanation of it in the Dask docs.

[bouweandela]

Thanks for reviewing!

However, it does rely on the odd fact that bool() forces computation of lazy arrays.
Out of interest, do you know why that is ?
I have never quite understood this, since I never found any explanation of it in the Dask docs.

The answer is simpler than I expected, have a look at the dask.array.Array.__bool__ method:

https://github.com/dask/dask/blob/191d39177009d2cce25b818878118e35329b6db3/dask/array/core.py#L1860-L1867

[pp-mo]

The answer is simpler than I expected, have a look at the dask.array.Array.__bool__ method:
https://github.com/dask/dask/blob/191d39177009d2cce25b818878118e35329b6db3/dask/array/core.py#L1860-L1867

Thanks, I see !

I still think it's a shame this is not highlighted in the user docs somewhere, though.
I was surprised when I found out !

[bouweandela]

Good point. I opened a pull request to document it: dask/dask#10686