New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ignore cache when producing reports #6076

merged 3 commits into from Dec 21, 2018


None yet
2 participants
Copy link

mthuurne commented Dec 17, 2018

I had to add a bit of functionality to the test runner, so command line tests can do multiple runs. That is in the first commit. The second commit contains the actual fix.

mthuurne added some commits Dec 17, 2018

Support multiple steps in command line tests
PythonCmdlineSuite uses DataDrivenTestCase, but inspects the test
case properties rather than calling runtest(). Therefore it could
parse tests with multiple steps, but during execution only one
step was run.
Disable cache when producing reports
If the cache is used, coverage information will only be reported for
modules that were not in the cache.

Fixes #5103

@mthuurne mthuurne force-pushed the mthuurne:coverage_ignore_cache branch from e5bbe4d to 2844779 Dec 17, 2018

Copy link

msullivan left a comment

One requested change about how to make the changes to the test framework.

Otherwise looks great!

for step in [1] + sorted(testcase.output2):
test_python_cmdline(testcase, step)
except AssertionError as ex:

This comment has been minimized.


msullivan Dec 20, 2018


I'd prefer to tweak the message at creation time (like is done in, rather than depending on catching it and mucking around with the internals.

Update command line test exception messages to include step number
Instead of intercepting and altering the exception messages, the
step number is now inserted into the message when it is initially

@msullivan msullivan merged commit bb0017d into python:master Dec 21, 2018

2 checks passed

continuous-integration/appveyor/pr AppVeyor build succeeded
continuous-integration/travis-ci/pr The Travis CI build passed

@mthuurne mthuurne deleted the mthuurne:coverage_ignore_cache branch Dec 21, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment