Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.Sign up
nixosTests: Reraise exceptions in subtests so they stop the test #79328
Motivation for this change
While porting the
I changed this passage of the test driver so that it gives extra-info in which subtest the problem occured and then leave it to the global exception handling to fail the whole test.
This basically depends on whether we want to use subtests to describe individual independent things / testcases on machines that don't necessarily depend on each other (A), or just use them as a way to group / give a heading to a bunch of statements (B).
However, I think we mostly do B in many tests, even though we probably shouldn't. WDYT?
I don't think we should change semantics here with the perl driver still around.
It generally makes sense to enable the user to decide which things should make tests fail and which should not.
True, but i observed this behavior to do no good in the many many tests that i have ported.
The vast majority of tests that i have seen assumes behavior B. Behavior B is also a good default setting in my opinion.
We could longterm give users some wrapper that is similar to
I agree that it is not cool to have different semantics in different implementations that share the same names.
But we have a lot of things in subtests that are necessary prerequisites to their next steps, but won't blow up visibly, leading to tests that time out (e.g. something must be done, but that fails, and then a test loops on
Regarding the minority of still existing perl tests that remain untouched and working until they are ported (and people will see that they get more fails instead of less then, which is always better than the other way around), i suggest putting this in.
I stumbled more than once into python tests failing in a previous subtests, while I was staring at the later (still running) output.
As this doesn't change semantics of passing tests, and as tests run isolated, I don't have any strong objections against this.