update tox/travis to check code coverage on py3 #1471
Conversation
coverage is combined between py2 and py3 for an aggregate coverage metric. This means we can stop putting "no cover" gates around py3 code and ignoring whether it's ever tested.
|
The |
|
Damn it passed locally but it might be running differently on travis. Time to investigate. |
|
Yeah so, I'm not sure how to get around this issue. Each sub-job is executed in its own sandbox on travis in parallel so they cannot share the coverage file. |
|
The best way I can think of to get around this is to create one final job that does this. So each individual job does not run coverage and goes quickly. Then there is a final job that runs "TOXENV=clean,py26,py32,cover" or something like that. What do you think @tseaver? This seems reasonable to me. |
|
You have to have coverage enbaled during the |
|
@tseaver That's correct but I could send that in as a separate env option like |
|
I'm open to some alternatives but this latest push 14126ca has the following outputs:
Main change required to our jenkins box is to read Advantages:
Disadvantages:
|
|
LGTM overall. I think I would prefer to make |
|
I've thought about this a bit and the main change I want to make before this is merged is that the tests run dog-damn slow on travis when running coverage. I don't have a good idea how to fix it though. Tentative idea is:
|
|
@tseaver can you please re-evaluate this based on my latest changes? Minus some quirks on tox syntax I'm pretty happy with this configuration. The outputs are the following:
|
mmerickel
force-pushed
the
feature.py3-coverage
branch
from
b794b61
to
d476e69
Compare
|
I stole a bit of this from https://github.com/tornadoweb/tornado/blob/0b2ecd3f71619ad077d54a54bbf43b5be2b593b3/tox.ini |
|
I have no problem with any of this. |
|
@mcdonc jenkins will have to be fixed to use nosetests-py27.xml From what I have read at some point you can actually have jenkins use multiple xml files to combine the results, but I cannot verify this. |
update tox/travis to check code coverage on py3
|
Jenkins did break. I tweaked its config to look for |
Coverage is combined between py2 and py3 for an aggregate coverage
metric. This means we can stop putting "no cover" gates around py3 code
and ignoring whether it's ever tested.
This is the best way I could find to do this based on several resources [1,2]. I'd be ecstatic if someone has a cleaner way. The only negative I see of this approach is running the tests one final time to dump xunit and coverage.xml files.
Fixes #1470.
[1] http://blog.ionelmc.ro/2014/05/25/python-packaging/
[2] https://github.com/tuskar/python-tuskarclient/blob/master/tox.ini