Set multiprocessing to spawn for Linux

[fonnesbeck]

Description

Multiprocess sampling often fails randomly on Linux due to the start method defaulting to "fork". This sets it to "spawn" when mcmc is imported.

Related Issue

• Closes ConnectionResetError from multiprocess sampling #7354
• Related to #

Checklist

• Checked that the pre-commit linting/style checks pass
• Included tests that prove the fix is effective or that the new feature works
• Added necessary documentation (docstrings and/or example notebooks)
• If you are a pro: each commit corresponds to a relevant logical change

Type of change

• New feature / enhancement
• Bug fix
• Documentation
• Maintenance
• Other (please specify):

---

📚 Documentation preview 📚: https://pymc--7804.org.readthedocs.build/en/7804/

[Copilot]

Pull Request Overview

This PR aims to fix random failures in multiprocess sampling on Linux by forcing the multiprocessing start method to "spawn".

• Added a platform check for Linux
• Configured the multiprocessing start method to "spawn" when importing the mcmc module

[ricardoV94]

We should be very careful about doing this change as it has been working for decades now just fine and it's the fastest form of starting multiprocessing.

Are you doing this because of JAX? Because JAX is still pretty useless without special handling on our side.

[ricardoV94]

Also there's at least one known issue with spawn: #7790

[fonnesbeck]

That issue appears to be restricted to MacOS.
This change in this PR only applies to Linux.

At the moment I have to manually switch to spawn on pretty much every model I run because it fails so frequently. Do we have any reports of failure using spawn on Linux?

[ricardoV94]

Can you show some code to see if anybody can reproduce, before changing for everyone?

This multiprocessing thing is something I see people come and flip flop around because it fails on their machine, then someone comes and puts it back because the new one fails on their machine.

I don't think the issue I linked is due to mac, but due to spawn which mac uses by default. If we make that default in Linux we would also see it. It's just an example of something subtle that may break for existing users who have no problem with fork.

Also I think you can change the default python method once and that's what pymc would use?

[fonnesbeck]

This issue continues to absolutely plague me on Linux. Can't run a notebook of any sort without seeing it at least once.

[codecov]

Codecov Report

❌ Patch coverage is 90.00000% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 93.58%. Comparing base (c8d83ca) to head (8b6f9e6).
⚠️ Report is 65 commits behind head on main.

Files with missing lines	Patch %	Lines
pymc/sampling/parallel.py	90.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #7804      +/-   ##
==========================================
+ Coverage   92.84%   93.58%   +0.73%     
==========================================
  Files         107      116       +9     
  Lines       18380    24016    +5636     
==========================================
+ Hits        17065    22475    +5410     
- Misses       1315     1541     +226     

Files with missing lines	Coverage Δ
pymc/sampling/parallel.py	88.52% <90.00%> (+1.44%)	⬆️

... and 55 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[ricardoV94]

Can you confirm the problem is with threadpool_limits? #7354 (comment)