Lower score when combining test suites?

[karfau]

Something happened that might totally break my understanding of how stryker works:

(I really hope this makes no sense and is some config error, but just to make sure I have the right assumptions:)

There is some legacy test suite that reaches the following score:
58.12% ( 1104 ✔️ / 835 👽 / 55 ⏳ )

and there is a new jest based test suite that reaches the following score(when being executed with stryker without the legacy one):
99.70% ( 1958 ✔️ / 6 👽 / 35 ⏳ )

When I tell stryker to execute both of them together, I would expect at least to reach the score of only executing the jest based one, right?

In my case the combined run instead resulted in this score:
64.73% ( 1243 ✔️ / 705 👽 / 51 ⏳ )

(All three cases are using the command runner and version 3.3.1 of all stryker dependencies involved.)

Any help/ideas appriciated.

[karfau]

Here is my "train of thoughts" on combining the two test suites:

For stryker a mutant "survives" if all tests pass (return value 0).
=> When either of the test suites fail (return code 1) the mutant was "killed"
npm run test:legacy && npm run test:jest:

• if test:legacy fails, the expression should result in return code 1: mutant "killed". (test:jest is not executed).
• if test:legacy passes, test:jest is executed
  • if test:jest fails (which it does for 97% when it's the only test suite in place) the mutant is "killed".
  • if test:jest passes the mutant "survived"

So combining them using the && operator seems a valid approach.

[brodybits]

What if you would use npm-run-all?

[karfau]

What if you would use npm-run-all?

It yields the same result as combining with &&, see xmldom/xmldom#112 (comment)

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.