Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
Add test for loading the various wptrunner products #9496
2 times, most recently
Feb 13, 2018
jgraham left a comment
Seems OK I guess. I see the point given we missed some regressions here, although the generally crazy import stuff seems to make the tests a bit delicate.
The tox stuff really needs documentation because it's far from obvious what you're doing or why.
So per my comments on IRC, I don't really undrestand all the complexity here. I don't understand why there have to be three different environment variables or why we run all the tests but then ignore the results sometimes. I think this should either be simplified or more thoroughly commented to make it clear why this is the simplest possible approach.
@jgraham What three environment variables? The only environment variables here are
I don't know how to explain the issue further…
Fundamentally the problem is that we have:
We run both of these in the same Travis job, with the same TOXENV set to the installed Python version (so
Except, as you can see, wptrunner doesn't define a tox environment with any of those names, but rather many of them with suffixes, and we want to run all of them (because e.g.
As for running all the tests but ignoring the results sometimes, how do you suggest we do this? We probably shouldn't break running the tests outside of tox, but within tox we could potentially try to more aggressively skip them before running them for other jobs?