New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Global parallelization doesn't play nice with @runs_once? #681

Closed
bitprophet opened this Issue Jul 12, 2012 · 5 comments

Comments

Projects
None yet
3 participants
@bitprophet
Member

bitprophet commented Jul 12, 2012

Haven't looked but suspect this is simply because the parallel machinery is unaware of @runs_once and so by the time @runs_once's memoization implementation kicks in, it's already inside its own multiprocessing subprocess, and so of course the reliance on shared state for memoization to function, means it runs once per host.

Assuming that's accurate, make the parallel machinery aware of @runs_once's desired behavior, either by looking for an existing attribute, or adding a 'hint' in @runs_once's implementation and look for that.

@bitprophet

This comment has been minimized.

Member

bitprophet commented Jul 14, 2012

I actually cannot replicate this now, and upon inspection, we are explicitly handling this by having @runs_once mark the decorator as serial, and have since at least 1.3.x.

@bracki I think you were the one who mentioned this on IRC; can you please confirm your instance of this bug and verify the version you're seeing it with? (And ideally how to replicate it.)

@bracki

This comment has been minimized.

bracki commented Jul 16, 2012

Then it is a misunderstanding stemming from the documentation on runs_once. It explicitly warns that runs_once does not go along with parallel mode.
https://github.com/fabric/fabric/blob/master/fabric/decorators.py#L129

@bitprophet

This comment has been minimized.

Member

bitprophet commented Jul 16, 2012

Aha, totally missed that. It was probably added before we implemented the solution I mentioned. I'll fix that posthaste, thanks!

bitprophet added a commit that referenced this issue Mar 19, 2013

@daniyalzade

This comment has been minimized.

daniyalzade commented Dec 4, 2015

Quick question here though, in the parallel mode, how can I ensure that a task is run only once, such as a cleanup task at the end of the execution?

@bitprophet

This comment has been minimized.

Member

bitprophet commented Dec 6, 2015

@daniyalzade please use the mailing list or irc for that sort of question, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment