Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add an option allowing use without a multiprocessing pool... #251

Merged
merged 1 commit into from
Jan 30, 2020
Merged

Add an option allowing use without a multiprocessing pool... #251

merged 1 commit into from
Jan 30, 2020

Conversation

valrus
Copy link
Contributor

@valrus valrus commented Jan 27, 2020

[Spleeter] - Add an option allowing use without a multiprocessing pool

Recreating this PR from #231 by request from reviewers to trigger CI.

This PR adds a keyword argument allowing a Separator to be used asynchronously within a multiprocessing-created subprocess (by bypassing its multiprocessing.Pool and doing its work in the main process). Without this option enabled, attempting to do so will fail with an error "AssertionError: daemonic processes are not allowed to have children."

How this patch was tested

I'm working on a tool that calls separate_to_file using Pool.apply_async. Prior to this patch it failed with the AssertionError above. After applying the patch and passing multiprocess=False to the Separator constructor the separation works as expected.

Documentation link and external references

Info about the exception in the Python docs: https://docs.python.org/3/library/multiprocessing.html#multiprocessing.Process.daemon

Here's a SO example of someone encountering the same exception in a similar scenario: https://stackoverflow.com/questions/51485212/multiprocessing-gives-assertionerror-daemonic-processes-are-not-allowed-to-have

It seems that this problem could also be avoided by using a ThreadPool but I understand spleeter to be fairly CPU-bound so threads aren't a great solution. However if spleeter does most of its heavy lifting in its own subprocesses, it might work to use a ThreadPool instead. I'm willing to try it if you'd prefer.

...this allows spleeter itself to be run asynchronously from another
process using multiprocessing, which otherwise fails with "daemonic
processes are not allowed to have children"
@Faylixe Faylixe merged commit d5d372d into deezer:master Jan 30, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants