-
-
Notifications
You must be signed in to change notification settings - Fork 716
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
"Set changed size during iteration" Error in Worker.put_key_in_memory() #4371
Comments
Maybe that happens because I submit tasks from tasks? Wrapping the loop with this seems to fix the error:
|
… On Thu, Dec 17, 2020 at 2:17 AM manuels ***@***.***> wrote:
Maybe that happens because I submit tasks from tasks
<https://docs.dask.org/en/latest/futures.html#submit-tasks-from-tasks>?
Wrapping the loop with this seems to fix the error:
while True:
try:
for dep in ts.dependents:
...
break
except RuntimeError as e:
if e.message == 'Set changed size during iteration':
continue
raise e
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#4371 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AACKZTB7OYVMYLOITOFGOT3SVHLDVANCNFSM4U7HT2ZQ>
.
|
Even with the |
It seems to work fine if I downgrade to v2.30. |
is this still an issue in 2021.01.1? Has anyone checked or verified? |
@fjetter In case this is useful, I can confirm I get this on 2021.7.2. Worker logs:
On the client side I get
Unfortunately I don't have a good repro here. |
it's a shot in the dark, but maybe #5285 helps? Is it easy for you to test your workload against that branch? |
Thanks - will try that as soon as I can. |
Thanks @gerrymanoim |
I am getting an error in this line:
distributed/distributed/worker.py
Line 1970 in d54388c
Maybe a temporary copy of ts.dependents helps here?
The text was updated successfully, but these errors were encountered: