Unable to collect accurate code coverage

[audiodude]

Repository commit

34f48b6

Python version (python --version)

3.12.0

Dependencies version (pip freeze)

beautifulsoup4==4.12.2
certifi==2023.7.22
charset-normalizer==3.3.0
contourpy==1.1.1
coverage==7.3.2
cycler==0.12.1
fake-useragent==1.3.0
fonttools==4.43.1
idna==3.4
imageio==2.31.5
iniconfig==2.0.0
joblib==1.3.2
kiwisolver==1.4.5
lxml==4.9.3
markdown-it-py==3.0.0
matplotlib==3.8.0
mdurl==0.1.2
mpmath==1.3.0
numpy==1.26.1
oauthlib==3.2.2
opencv-python==4.8.1.78
packaging==23.2
pandas==2.1.1
patsy==0.5.3
Pillow==10.1.0
pip==23.3
pluggy==1.3.0
Pygments==2.16.1
pyparsing==3.1.1
pytest==7.4.2
pytest-cov==4.1.0
python-dateutil==2.8.2
pytz==2023.3.post1
requests==2.31.0
requests-oauthlib==1.3.1
rich==13.6.0
scikit-learn==1.3.1
scipy==1.11.3
setuptools==68.2.2
six==1.16.0
soupsieve==2.5
statsmodels==0.14.0
sympy==1.12
threadpoolctl==3.2.0
tweepy==4.14.0
tzdata==2023.3
urllib3==2.0.7
wheel==0.41.2
xgboost==2.0.0

Expected behavior

When I run the top level code coverage for the entire project, the percentage code coverage for an individual file is the same as wehn I run it on that file alone.

Actual behavior

These values differ. For example:

(venv) tmoneys-MacBook-Pro:algorithms-python tmoney$ grep closest_pair_of_points coverage_readable.txt
divide_and_conquer/closest_pair_of_points.py                                     39     32    18%   28, 36, 54-59, 76-81, 98-124, 132-134, 142-143
(venv) tmoneys-MacBook-Pro:algorithms-python tmoney$ grep closest_pair_of_points coverage_one_file.txt
divide_and_conquer/closest_pair_of_points.py ......                      [100%]
divide_and_conquer/closest_pair_of_points.py                              39     13    67%   102-124, 142-143
0.01s call     divide_and_conquer/closest_pair_of_points.py::divide_and_conquer.closest_pair_of_points.closest_pair_of_points

[tianyizheng02]

Without more info, I can't tell where the discrepancy comes from. However, this seems more likely to be a problem with pytest-cov rather than this repository.

[audiodude]

I understand. This was in context of a Hacktoberfest issue that encouraged contributors to write doctests and look for files with low test coverage. I saw that this file had "18%" coverage, but when I opened the file, almost all of the functions had doctests, so I was confused.

You're right, it's probably that pytest-cov is barfing on doctests unless the module is loaded a particular way or something.

[audiodude]

Perhaps this makes it more clear, without using the output files that I created for my own analysis:

$ pytest --ignore=quantum/q_fourier_transform.py --ignore=project_euler/ --ignore=scripts/validate_solutions.py --cov-report=term-missing:skip-covered --cov=. . | grep closest_pair_of_points.py
divide_and_conquer/closest_pair_of_points.py                                     39     32    18%   28, 36, 54-59, 76-81, 98-124, 132-134, 142-143

vs

$ pytest --ignore=quantum/q_fourier_transform.py --ignore=project_euler/ --ignore=scripts/validate_solutions.py --cov-report=term-missing:skip-covered --cov=. divide_and_conquer/closest_pair_of_points.py | grep closest_pair_of_points.py
divide_and_conquer/closest_pair_of_points.py ......                      [100%]
divide_and_conquer/closest_pair_of_points.py                              39     13    67%   102-124, 142-143

18% when the full suite is run (copying the command directly from the build workflow) but 67% when the file is run on it's own.