-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Global parallelization doesn't play nice with @runs_once? #681
Comments
I actually cannot replicate this now, and upon inspection, we are explicitly handling this by having @bracki I think you were the one who mentioned this on IRC; can you please confirm your instance of this bug and verify the version you're seeing it with? (And ideally how to replicate it.) |
Then it is a misunderstanding stemming from the documentation on |
Aha, totally missed that. It was probably added before we implemented the solution I mentioned. I'll fix that posthaste, thanks! |
Quick question here though, in the parallel mode, how can I ensure that a task is run only once, such as a cleanup task at the end of the execution? |
@daniyalzade please use the mailing list or irc for that sort of question, thanks! |
Haven't looked but suspect this is simply because the parallel machinery is unaware of
@runs_once
and so by the time@runs_once
's memoization implementation kicks in, it's already inside its own multiprocessing subprocess, and so of course the reliance on shared state for memoization to function, means it runs once per host.Assuming that's accurate, make the parallel machinery aware of
@runs_once
's desired behavior, either by looking for an existing attribute, or adding a 'hint' in@runs_once
's implementation and look for that.The text was updated successfully, but these errors were encountered: