Skip to content

Conversation

@takhs91
Copy link
Contributor

@takhs91 takhs91 commented Mar 23, 2020

This allows the use of multiprocessing.Pool inside an actor as discussed in https://www.reddit.com/r/dramatiq/comments/fdus4w/support_for_cpu_intensive_workloads_and/

I initially ran the tests with daemon hardcoded to False and they passed but I made a cli option to reduce any chances altering the current functionality. I furthemore tried to analyze what changes it could create

Since you have already written sighandlers and logic about waiting the worker processes to exit using daemon=False should't leave any orphaned workers assuming cli executes normally and gets shutdowned by signals other than sigkill or sigquit(those also leaves the worker processes orphaned even with daemon=True it could probably only be solved from their side).

If an exception happens in cli before or inside the current process waiting code, it seem to be causing a deadlock in exiting in either way the daemon flag is set.

To conclude if you want to enable actors using multiprocessing I believe that this will cause no harm to existing functionality with either option set. Solving any of the afformentioned issues should probably be tackled in different issues. I would like to hear your thoughts!

@takhs91 takhs91 marked this pull request as ready for review March 27, 2020 12:48
@Bogdanp
Copy link
Owner

Bogdanp commented Mar 28, 2020

Thanks! Let's go the other route: might as well not add a new flag unless we need to. I'll go through some of my own test cases once you make the change.

@Bogdanp
Copy link
Owner

Bogdanp commented Mar 28, 2020

Since I closed the other PR, please add yourself to the contributors file here. Thanks again!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants