New selector to optimize for both coverage and PIT/descartes

[vmassol]

I've tested with the jacoco selector in the hope to get a better coverage increase than with the default PIT selector.

Unfortunately I hit #586. I’ve manually fixed the tests and remove those that didn’t pass (2 that didn't pass). And I got an increase of +0.18% jacoco coverage increase and -2% descartes mutation score…

There are 2 problems here:

1. I got +0.216% of coverage increase when testing with PIT/Gregor selector... I wasn't expecting this
2. It decreased the mutation score which isn't good...

Could we imagine a selector that is a combination of both descartes and jacoco for example so that we would never get a decrease in coverage nor mutation score? Does it even make sense?

Thanks

[danglotb]

It decreased the mutation score which isn't good...

Did you verify the number of errors?
Did you run the original test suite + the amplified test suite?

Could we imagine a selector that is a combination of both descartes and jacoco for example so that we would never get a decrease in coverage nor mutation score? Does it even make sense?

We can imagine in fact. But I see two major problems:

1. It transforms the problem into a multi-objective and so we have to deal with the problem of maximizing them (compute the pareto front for instance).
2. It will require to run even more the amplified tests, and so increase the execution time

[danglotb]

I close this issue since It has not been updated in two months.

For the decrease of the mutation score, I'm pretty sure that comes from the instability of PIT. Please, double-check.

For the multi-objective test-criteria, one can implement a selector that keeps newly covered mutants AND new mutants killed.