Add cli flag to start worker processes with daemon=False #289
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
This allows the use of multiprocessing.Pool inside an actor as discussed in https://www.reddit.com/r/dramatiq/comments/fdus4w/support_for_cpu_intensive_workloads_and/
I initially ran the tests with daemon hardcoded to False and they passed but I made a cli option to reduce any chances altering the current functionality. I furthemore tried to analyze what changes it could create
Since you have already written sighandlers and logic about waiting the worker processes to exit using daemon=False should't leave any orphaned workers assuming cli executes normally and gets shutdowned by signals other than sigkill or sigquit(those also leaves the worker processes orphaned even with daemon=True it could probably only be solved from their side).
If an exception happens in cli before or inside the current process waiting code, it seem to be causing a deadlock in exiting in either way the daemon flag is set.
To conclude if you want to enable actors using multiprocessing I believe that this will cause no harm to existing functionality with either option set. Solving any of the afformentioned issues should probably be tackled in different issues. I would like to hear your thoughts!