Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
yum_repository in item loops causes repo duplication when used with mitogen #154
Using mitogen master cloned at b1bfe58
I have a role task that calls yum_repository using with_items (also tested with_dict, same result) to populate a default list of repos and any 'extras' provided by the calling playbook. Expected behavior is one repo file per list item, with only that item in the repo file. With mitogen disabled, that's what I get. With mitogen enabled, each successive repo file in /etc/yum.repos.d/ contains all of the prior items in the loop. Simply enabling/disabling mitogen and re-running the playbook (deleting the created repos between each run) will demonstrate.
Congrats, looks like you've found the first real example of a module storing global state that persists across runs :)
In this case, the global state is the YumRepo.repofile variable.
So the solution here is to start forking the child after compilation, to prevent that global state persisting. Hopefully a fix won't take too long. Thanks for reporting!
Regarding avoiding having the threading/forking deadlock of #150 not reappear in the child, some careful dancing is required to repair any locks that may be held across the fork. Python takes care of the GIL (naturally), but we must take care of the logging package. A cute workaround can be found here: celery/celery#496
added a commit
Mar 18, 2018
I've just pushed a workaround for this specific case, forking is likely to take a little longer. Note that due to the yum_repository module, rerunning Ansible won't magically clean up those bad repo files. You need to delete them and let it recreate them. :)
Thanks for the report!