Skip to content

Commit

Permalink
Fixed failure of exit-behavior tests during coverage reporting.
Browse files Browse the repository at this point in the history
Just like in the recently revamped test runner (`test.py`), the project
root folder must be added to the Python search path. While pyTest does
add the root folder to `sys.path` in its own process, it is not available
in the subprocesses testing the exit behavior. Which is why these tests
failed when `tools/coverage.py` was run.
  • Loading branch information
john-hen committed Jan 23, 2022
1 parent 2a4b82d commit 997f54b
Show file tree
Hide file tree
Showing 2 changed files with 51 additions and 25 deletions.
14 changes: 13 additions & 1 deletion tools/codecov.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,24 @@
"""Uploads the coverage report to CodeCov."""
"""
Uploads the coverage report to CodeCov.
The script expects the CodeCov uploader to be installed locally. On
Windows, for example, `codecov.exe` would have to be on the search path.
It also expects the CodeCov upload token for this project to be set as
an environment variable.
CodeCov does not accept Coverage.py's standard report format, i.e. the
`.coverage` file. It must be converted to XML format before upload.
"""

from subprocess import run
from pathlib import Path
from os import environ


token = environ.get('MPh_CodeCov_token', None)
if not token:
raise RuntimeError('CodeCov upload token not set in environment.')

root = Path(__file__).resolve().parent.parent
run(['coverage', 'xml'], cwd=root)
run(['codecov', '--file', 'coverage.xml', '--token', token], cwd=root)
62 changes: 38 additions & 24 deletions tools/coverage.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,32 +5,46 @@
of the repo) because the individual scripts there all start a client in
their respective setup routine. That is, they all start the Java VM,
which will fail once pyTest gets to the second script in the sequence.
Instead, we run pyTest for each script separately, with the coverage
plug-in enabled, and generate the coverage report incrementally.
Instead, we run pyTest for each test group separately, with the coverage
plug-in enabled, and thus generate the coverage report incrementally.
If the coverage report file (`coverage.xml`) already exists, we instead
render it as HTML for inspection. This is helpful during development.
The coverage report may also be uploaded to an online service such as
CodeCov for each published release or even commit, see `codecov.py`.
We also render the coverage report (in `.coverage`) as HTML for easy
inspection. This is helpful during development.
The coverage report may be uploaded to the online service CodeCov. This is
usually only done for a new release, but could also happen on each commit.
There's a separate script, `codecov.py`, to automate that.
"""

from subprocess import run
from pathlib import Path
import sys

tests = ['meta', 'config', 'discovery', 'server', 'session', 'standalone',
'client', 'multi', 'node', 'model', 'exit']

here = Path(__file__).resolve().parent
root = here.parent
file = root/'coverage.xml'
python = sys.executable
if not file.exists():
command = [python, '-m', 'pytest',
'--cov', '--cov-append', '--cov-report', 'xml']
for test in tests:
run(command + [f'tests/test_{test}.py'], cwd=root)
from pathlib import Path
from sys import executable as python
from os import environ, pathsep


# Define order of test groups.
groups = ['meta', 'config', 'discovery', 'server', 'session', 'standalone',
'client', 'multi', 'node', 'model', 'exit']

# Determine path of project root folder.
here = Path(__file__).resolve().parent
root = here.parent

# Run MPh in project folder, not a possibly different installed version.
if 'PYTHONPATH' in environ:
environ['PYTHONPATH'] = str(root) + pathsep + environ['PYTHONPATH']
else:
print('Rendering existing coverage report as HTML.')
folder = (here/'coverage').relative_to(root)
run(['coverage', 'html', f'--directory={folder}'], cwd=root)
environ['PYTHONPATH'] = str(root)

# Report code coverage one by one for each test group.
report = root/'.coverage'
if report.exists():
report.unlink()
for group in groups:
run([python, '-m', 'pytest', '--cov', '--cov-append',
f'tests/test_{group}.py'], cwd=root)

# Render coverage report locally.
print('Exporting coverage report as HTML.')
folder = (here/'coverage').relative_to(root)
run(['coverage', 'html', f'--directory={folder}'], cwd=root)

0 comments on commit 997f54b

Please sign in to comment.