New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Always use the spawn start method for multiprocessing for wpt commands #29051
Conversation
I mean the hope was that we end up with consistent usage of multiprocesssing and in particular always stick to the safest context type. I thought it might affect #29024 but I"m not really surprised it doesn't given that this should be the default on macOS anyway. |
https://docs.python.org/3/library/multiprocessing.html#multiprocessing.set_start_method says "it should be protected inside the if __name__ == '__main__' clause of the main module" which would require putting it in the root |
try: | ||
multiprocessing.set_start_method('spawn') | ||
except RuntimeError as e: | ||
# This can happen if we call back into wpt having already set the context |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should be possible to avoid if we put it in the root wpt
right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think so. But the reason I noticed this is that we have unit tests that call wpt.main
instead of going via the command line. So I think we'd run the risk of those tests running under an unsupported configuration if we so this in a pre-main location.
I tried a test and I can't see that calling this in main
vs if __name__ == "__main__"
makes any difference to the behaviour; in each case the subprocess gets the same default spawn method. It seems like the important thing is really that it's called at the main entrypoint to the code; as long as we're confident everything goes through main
it's all fine.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, probably the Python documentation says that as a slightly inaccurate way of saying "please make sure to call it always and early".
No description provided.