-
Notifications
You must be signed in to change notification settings - Fork 42
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Code coverage appears much lower on python 3.8 #132
Comments
I just ran into this problem after eliminating all other variables. I've isolated it down to a change in Python version causing our CI to fail because of lower-than-expected test coverage on otherwise unmodified code. Our test coverage drops after Python is upgraded to 3.8.x from 3.7.x I see this issue was closed 3 years ago, I'm wondering what the resolution is? Was Python 3.7 calculating coverage incorrectly? |
There were some CPython optimization that affected coverage, see also https://github.com/nedbat/coveragepy/. I can't recall how we truly resolved the issue here. It could be we simply bit the bullet and added tests to bump the coverage on all versions over the threshold. Or it was as simple as updating coveragepy. Too long ago for me to remember all the details :) |
The drop in code coverage reported by codecov seems to be directly related to python 3.8. The travis builds with any other version of python yield a code coverage of ~70% while the python 3.8 build yields a coverage of ~50%. I can reproduce the same numbers locally.
On tonight master (053e486), both python 3.5 and 3.8 report the same number of tests, and the success status is the same. Python 3.8 reports a few more warning, though. See #130 and #131 for the warnings.
At least one issue has been raised on the coverage module regarding changes in behaviour with python 3.8: nedbat/coveragepy#707
We probably should not upload coverage report from python 3.8 until the issue is solved.
The text was updated successfully, but these errors were encountered: