Skip to content

Commit

Permalink
Added reporting of test code coverage.
Browse files Browse the repository at this point in the history
Code coverage by the test suite is measured via pyTest and its
pytest-cov plug-in. As pyTest cannot run the entire test suite
in one go (as the Java VM is started multiple times, by separate
test scripts), the script `coverage.py` is provided in the `deploy`
folder to automate the runs of inidividual test scripts and combine
the reports into a unified file.

The script also creates the coverage badge, unless that file already
exists. In other words, delete the badge to update the coverage
percentage.

Test coverage is reported for Windows only. Code that targets other
platforms is therefore exluded from the reporting.
  • Loading branch information
john-hen committed Apr 30, 2021
1 parent dabaa4f commit 9baf93c
Show file tree
Hide file tree
Showing 9 changed files with 76 additions and 21 deletions.
6 changes: 4 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,12 +1,14 @@
/develop/
/deploy/docs/
/deploy/dist/
/deploy/coverage/
/deploy/coverage.sqlite
/dist/
**/__pycache__
**/.pytest_cache
flake8.txt
.spyproject
.idea
*.lnk
.DS_Store
Thumbs.db
.spyproject
.idea
1 change: 1 addition & 0 deletions ReadMe.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,4 +30,5 @@ Find the full [documentation on Read-the-Docs][docs].
[![license](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT)
[![release](https://img.shields.io/pypi/v/mph.svg)](https://pypi.python.org/pypi/mph)
[![downloads](https://pepy.tech/badge/mph)](https://pepy.tech/project/mph)
![coverage](tests/coverage.svg?raw=true)
[![documentation](https://readthedocs.org/projects/mph/badge/?version=latest)](https://mph.readthedocs.io/en/latest)
4 changes: 2 additions & 2 deletions deploy/clean.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,5 @@
for folder in root.rglob('.pytest_cache'):
rmtree(folder)

for folder in (root/'deploy'/'dist', root/'deploy'/'docs'):
rmtree(folder, ignore_errors=True)
for folder in ('docs', 'dist', 'coverage'):
rmtree(root/'deploy'/folder, ignore_errors=True)
25 changes: 25 additions & 0 deletions deploy/coverage.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
"""Measures code coverage by test suite."""
__license__ = 'MIT'

from subprocess import run
from pathlib import Path

here = Path(__file__).resolve().parent
root = here.parent

file = here/'coverage.sqlite'
if file.is_file():
print('Rendering coverage report.')
folder = (here/'coverage').relative_to(root)
run(['coverage', 'html', f'--directory={folder}'], cwd=root)
badge = root/'tests'/file.with_suffix('.svg').name
if badge.exists():
print('Coverage badge already exists.')
else:
print('Rendering coverage badge.')
run(['coverage-badge', '-f', '-o', str(badge)], cwd=root)
else:
command = ['pytest', '--cov', '--cov-append']
tests = ['discovery', 'server', 'session', 'node', 'model', 'processes']
for test in tests:
run(command + [f'tests/test_{test}.py'], cwd=root)
3 changes: 0 additions & 3 deletions docs/demonstrations.md
Original file line number Diff line number Diff line change
Expand Up @@ -381,9 +381,6 @@ block of ice
├─ tables
├─ plots
└─ exports

>>> mph.tree(model)
block of ice
```

The parameter group, model component, default view and coordinate
Expand Down
19 changes: 5 additions & 14 deletions docs/limitations.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,18 +16,11 @@ to be started as separate Python subprocesses. This is a feasible
work-around, but a limitation nonetheless. Refer to section ["Multiple
processes"](demonstrations.md#multiple-processes) for a demonstration.

For the same reason, the [test suite][tests] that comes with MPh fails
when collected and run via the testing framework [pyTest][pytest].
The tests must be run directly from the command line. Since they cannot
be used for continuous integration testing anyway, given that they
depend on Comsol being installed on the machine, this is but a minor
inconvenience.

More importantly, there are some known, but unresolved issues with
JPype's shutdown of the Java virtual machine. Notably, pressing
<kbd>Ctrl+C</kbd> to interrupt an ongoing operation will usually crash
the Python session. So do not rely on catching [`KeyboardInterrupt`][kbint]
exceptions in application code.
Additionally, there are some known, but unresolved issues with JPype's
shutdown of the Java virtual machine. Notably, pressing <kbd>Ctrl+C</kbd>
to interrupt an ongoing operation will usually crash the Python session.
So do not rely on catching [`KeyboardInterrupt`][kbint] exceptions in
application code.

(There is an alternative Java bridge, [pyJNIus][jnius], which is
not limited to one virtual machine, but then fails in another regard:
Expand Down Expand Up @@ -112,8 +105,6 @@ where this works reliably.

[jpype]: https://github.com/jpype-project/jpype
[japi]: https://comsol.com/documentation/COMSOL_ProgrammingReferenceManual.pdf
[tests]: https://github.com/MPh-py/mph/tree/main/tests
[pytest]: https://docs.pytest.org
[kbint]: https://docs.python.org/3/library/exceptions.html#KeyboardInterrupt
[jnius]: https://pyjnius.readthedocs.io
[issue8]: https://github.com/MPh-py/MPh/issues/8
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ classifiers = [

[tool.flit.metadata.requires-extra]
doc = ['Sphinx', 'Furo', 'MyST-parser', 'CommonMark']
test = ['pyTest', 'pyTest-cov', 'coverage-badge']

[tool.flit.metadata.urls]
Documentation = 'https://mph.readthedocs.io'
Expand Down
21 changes: 21 additions & 0 deletions tests/coverage.svg
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
17 changes: 17 additions & 0 deletions tox.ini
Original file line number Diff line number Diff line change
Expand Up @@ -17,3 +17,20 @@ ignore =
W503,
per-file-ignores =
mph/__init__.py:F401

[pytest]
addopts = --verbose
testpaths =
tests
filterwarnings =
ignore::UserWarning

[coverage:run]
source = mph/
omit = mph/discovery.py
data_file = deploy/coverage.sqlite

[coverage:report]
exclude_lines =
pragma: no cover
def check_environment

0 comments on commit 9baf93c

Please sign in to comment.