feat(schema): add Pending status to JSON schema

[xandervedder]

Do tell if more needs to be changed or tested.

closes #2424

BREAKING CHANGE: Pending is now also a valid mutant status

[hugo-vrijswijk]

Could you also add the status to metrics-scala?

[nicojs]

Seems like a good start. We should also update mutant-states-and-metrics.md, but that can be done in a separate PR.

I have a small remark. I'm curious what others think.

ALSO: you should add the following to your PR:

BREAKING CHANGE: 'pending' is no also a valid mutant status

[nicojs]

@maks-rafalko f.y.i. we're adding a pending state to a mutant as we're adding support for real-time updates to the HTML report. It's technically a breaking change since mutation testing reports that use the status won't be valid according to the v1 schema.

[maks-rafalko]

thanks for letting me know. We, at Infection, don't use Pending status so I assume we are not affected by this BC.

Looking forward to this feature, to migrate to Infection.

[xandervedder]

@nicojs I'm not so sure if this is a breaking change, since I tested my (concept) implementation with Stryker.NET. It appears that the mutation report accepts a status that is not defined in the schema, in my case this was the NotRun status that Stryker.NET uses internally.

Please correct me if I'm wrong.

Edit:
It appears the validation is only done in the Stryker dashboard, so I guess it technically is a breaking change?

[rouke-broersma]

@xandervedder if the html report is not actually validating the received data, and instead the validation is done in the dashboard, then this is not a breaking change for the webcomponents but it is a breaking change for the schema, since the schema does enforce the status as an enum with known values. The dashboard uses the schema for validation so that makes this a breaking change in the schema. Unfortunately I think that does make this a breaking change after all. The webcomponents don't have their own versioning strategy, they are coupled to the schema.

[nicojs]

F.y.i. this versioning thing is just a way to communicate to the outside world. It doesn't actually matter for us. So don't feel 'bad' about a breaking change. It's fine ☺️

[xandervedder]

Is there anything else I need to change for this to go through? Happy to change anything if needed 😃

[hugo-vrijswijk]

I'm thinking about the metrics. How do we want 'pending' to be calculated? I think it probably should be in the 'total' calculation

[xandervedder]

I think it should because otherwise the total amount of mutants would "grow".

Also, should pending be added to the columns in the mte-metrics-table?

[rouke-broersma]

I don't think pending should be in the table because it's only interesting for a very short time, and we can better use the progress bar to display this temporary state.

[xandervedder]

Should I also update mutant-states-and-metrics.md? Might as well do that since it's a small change.

cc @nicojs

[nicojs]

We can also document it in a separate PR IMO. Merge this? Or do a poll to decide the Emoji on the Stryker slack?

[xandervedder]

I'll make a poll in the Stryker slack (#general).

Documentation will be updated in a different PR (I'll also make an issue for that).