Decide WCT runner's fate

[nicojs]

For some time, Stryker supports the web component tester (WCT) as a test runner (@stryker-mutator/wct-runner). I've added it because I was working on a project with the WCT test runner at the time. However, we've never used it in that particular project 🤷‍♂️.

WCT is pretty much a dead test runner IMHO. The current version on npm was published over 2 years ago, not for a lack of issues. The support in Stryker is also pretty bare-bones, because a browser is started for each mutant to test, this is very slow. I tried to get WCT maintainer's attention, but failed at it. The @stryker-mutator/wct-runner package is not popular on npm with 55 weekly downloads. (although it is 55 more than 0, of course).

Maintaining the WCT runner takes some time, even though it doesn't update often. Every so often the tests fail for some reason. It also depends on Java for some reason. Not the biggest problem, but not ideal either. It is also not yet migrated to mutation switching. This can be done, with a couple of hours effort, but that are hours I cannot spent again.

I would personally like to focus on different test runners that actually are used, like node-tap support #325, or AVA support #243. I would be happy to help anyone maintain a community version of the WCT runner.

Although, this is just my personal opinion. I'm hoping to get feedback from users of @stryker-mutator/wct-runner. Please explain where you're using it and if you want us to keep supporting it.

I also did a poll on twitter, which got one vote from someone claiming to use it. https://twitter.com/stryker_mutator/status/1293080396851220480

[Lakitna]

I feel like maintaining the WTC runner, with all it's scaling issues, is not the best use of your time.

Runtime is the biggest enemy of mutation testing. I'm slowly starting to notice runtime things in my current setup using the Mocha runner and about 3000 mutants. I can only imagine how to slow it would be with a new browser for every mutant. That would be over 3000 browser instances in my case! This makes the WTC runner a bad choice for mutation testing IMO.

Add to that the fact that there are many more interesting things that we could use your eyes on. Things that would take others many hours of getting up to speed with the Stryker codebase.

I feel like it's worth it to deprecatedrop the WTC runner based on these two points alone. Sometimes, keeping features is just not worth it.

Now, I'm all for it if someone else wants to do this. More runners is a good thing. But I fear that most people will stay away unless the scaling issue has been solved.

[nicojs]

Thanks @Lakitna . Just to be clear, I'm not talking about deprecating, instead I just want to drop support entirely in the new Stryker release.

Deprecating it would mean updating to the new test runner API (which is needed because of mutation switching).

[Lakitna]

Yeah, that makes sense. I used deprecating too loosely here. Let me rephrase:

I feel like it's worth it to drop the WTC runner based on these two points alone. Sometimes, keeping features is just not worth it.

[bartekleon]

I agree with that. Assuming WCT is pretty much a dead test runner IMHO. The current version on npm was published over 2 years ago, not for a lack of issues. The support in Stryker is also pretty bare-bones, because a browser is started for each mutant to test, this is very slow. it is just not optimal to both maintain and use it. While mutation testing is good, it makes no sense to run several hours tests, especially if someone wanted to integrate it into CI.

[nicojs]

Note: if we remove the WCT runner, we shouldn't forget to also remove the acorn dependency (I remember adding it because of wct)