Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Tests fail silently if they have the same name as other tests #447
We have a suite which has a number of tests named "Happy Path".
We recently discovered that if some of them fail, then the test run still exits with a zero exit code, and our CI build passes.
This seems to be a result of this code-
I'd recommend doing something a bit stricter than purely name matching in this case (possibly name + file, or generating a unique id for a suite/test) so that it is not possible for test failures to be ignored.
Happy to PR if you have a preferred approach.