Tests failing after Distributed upstream changes #1224
Assignees
Comments
|
@rjzamora if you have time can you also take a look here ? |
|
Let me have a go... |
wence-
added a commit
to wence-/dask-cuda
that referenced
this issue
Aug 25, 2023
Now that task keys are no longer strings when stored in the worker data dictionary, we should not stringify the set of keys when staging. - Closes rapidsai#1224
|
Should be fixed by #1225. |
rapids-bot bot
pushed a commit
that referenced
this issue
Aug 25, 2023
Now that keys are no longer strings there are two places we must adapt here. 1. Explicit comms must no longer manually stringify task keys before staging and intersection with the on-worker data (since that data mapping doesn't use the stringified version) 2. The `zict.File`-backed slow buffer in `DeviceHostFile` needs to translate non-string keys to string keys before writing to disk, to do this, use the same implementation that distributed uses for its own spilling buffer. - Closes #1224 Authors: - Lawrence Mitchell (https://github.com/wence-) Approvers: - Peter Andreas Entschev (https://github.com/pentschev) URL: #1225
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Changes in dask/distributed#8083 have broken all nightly tests. If I didn't overlook anything, only
test_explicit_comms.pyandtest_spill.pytests were broken.@madsbk would you mind looking at the explicit comms tests? I have to still look in more details of what exactly changed w.r.t. spilling but I see the size of data that gets stored in
zict.Bufferhas changed, which indicates a change in how sizes are computed or that the stringify changes really changed the underlying stored data.The text was updated successfully, but these errors were encountered: