Spotty coverage of code run through multiprocessing

[sprezz-arthur]

Describe the bug
The coverage of the code run through python's multiprocessing can be taken into account or not, randomly.
There seems to be some race condition happening when SIGTERM is received, sometimes coverage has time to properly report run code, sometimes it doesn't.

To Reproduce

1. What version of Python are you using? Python 3.12.4
2. What version of coverage.py shows the problem? coverage_version: 7.6.9
3. What versions of what packages do you have installed? pytest@8.3.4
4. What code shows the problem? https://github.com/sprezz-arthur/flaky-coverage-mre
5. What commands should we run to reproduce the problem?
   rm -rf htmlcov && rm -f .coverage && coverage run -m pytest && coverage combine && coverage html

Expected behavior
I expected the code run through multiprocessing to always be taken into account

Additional context
I'm using pyproject.toml for config with the following content:

[tool.coverage.run]
branch = true
parallel = true
sigterm = true
concurrency = [
    "multiprocessing",
]

[nedbat]

When I try this on my Mac, it's 100% every time. What OS are you using?

[sprezz-arthur]

I'm using Microsoft Windows 11 Home:

Version 10.0.22631 Build 22631

More importantly, I'm using WSL2:

Kernel version: 5.15.167.4-1
WSLg version: 1.0.65
MSRDC version: 1.2.5620
Direct3D version: 1.611.1-81528511
DXCore version: 10.0.26100.1-240331-1435.ge-release
Windows version: 10.0.22631.4602

With Ubuntu:

Description:    Ubuntu 22.04.3 LTS
Release:        22.04
Codename:       jammy

[nedbat]

The Pool.join() docs say:

One must call close() or terminate() before using join().

and

__exit__() calls terminate().

It doesn't make sense to me that __exit__ doesn't call close(), but that's what the docs say.

Does it work better if you change your code to:

def test_foo():
    with multiprocessing.Pool(processes=1) as pool:
        pool.apply(complex_math_verification)
        pool.close()
    pool.join()

[sprezz-arthur]

I've already attempted explicitly calling pool.close and/or pool.terminate with no change in behavior.

I've also added small script to run it N times and gather the stats, which makes it easier to detect the flaky behavior.

I'm getting around 60% success rates.

[sprezz-arthur]

I've also modified the MRE to further simplify it.
Now it does not use pytest nor the branch = true property.

The spotty coverage still persists at around 60%.

[nedbat]

Thanks, the script shows that on a Mac, your code succeeds 98% of the time.

If I change your code like this, it succeeds 100%:

def test_foo():
    with multiprocessing.Pool(processes=1) as pool:
        pool.apply(complex_math_verification)
        pool.close()
        pool.join()

I hope it does for you too.

I'm not a fan of how much explicit closing is needed on a Pool, but maybe this is the correct solution?

[sprezz-arthur]

Thanks for the workaround.
I'm also not a fan of explicit context-manager handling, (given that multiprocessing is already supposed to do it), nor is it clear to me how and why this workaround works.

I guess I'll just monkey-patch multiprocessing.Pool's context manager for the time being.

Ultimately my goal is making coverage work with billiard.Pool, so I'll end up writing some custom Pool anyway. This goes beyond the scope of this issue.

For all I'm concerned, we can close this issue as the workaround is sufficient.
If I come up with a better understanding and solution, I'll propose a PR.

[nedbat]

Just to be clear, I don't consider this a workaround. This is the correct way to use a multiprocessing Pool and ensure that all of the processes terminate cleanly. I wish it weren't so cumbersome, but the documentation makes clear what __exit__ and terminate do, so this is the way to do what you need to do.

I had a similar question elsewhere about the same thing, so maybe I'll add a tip about this to the coverage docs.