Add cli flag to start worker processes with daemon=False #289
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This allows the use of multiprocessing.Pool inside an actor as discussed in https://www.reddit.com/r/dramatiq/comments/fdus4w/support_for_cpu_intensive_workloads_and/
I initially ran the tests with daemon hardcoded to False and they passed but I made a cli option to reduce any chances altering the current functionality. I furthemore tried to analyze what changes it could create
Since you have already written sighandlers and logic about waiting the worker processes to exit using daemon=False should't leave any orphaned workers assuming cli executes normally and gets shutdowned by signals other than sigkill or sigquit(those also leaves the worker processes orphaned even with daemon=True it could probably only be solved from their side).
If an exception happens in cli before or inside the current process waiting code, it seem to be causing a deadlock in exiting in either way the daemon flag is set.
To conclude if you want to enable actors using multiprocessing I believe that this will cause no harm to existing functionality with either option set. Solving any of the afformentioned issues should probably be tackled in different issues. I would like to hear your thoughts!