Pitest seems to ignore or never trigger Mockito's UnnecessaryStubbingException

[daniel-frak]

Given a mutation, if a test mutated this way would normally fail with org.mockito.exceptions.misusing.UnnecessaryStubbingException, Pitest will report it as having survived.

It seems that Pitest never triggers Mockito's UnnecessaryStubbing checks.

I have encountered this issue more than once, where I was doing the same modifications as Pitest but getting different results, before realising that Pitest behaves as if all stubbings were lenient().

[hcoles]

Which test framework are you using (JUnit 3/4/5, testng etc)

[daniel-frak]

I'm using Junit 5.

I'm unsure of what other information could be useful, so let me know if I can provide any more info.

[hcoles]

If you could put together a minimal project that replicates the issue, that would be helpful.

[daniel-frak]

I whipped up a quick minimal repro:
https://github.com/daniel-frak/pitest-mockito-stubbing-issue-example

[hcoles]

Been having a look at this, but not made much progress. It seems that the unnecessary stubbing check never triggers when running against a mutant (the tests fail correctly when run through pitest if the mutant is present when the test is first run).

Unless I've missed something, this will require some digging into the internals of mockito.

[hcoles]

Hi @daniel-frak,

Finally got chance to take a closer look at this today.

After a bit of debugging I confirmed that stubbing exception are propagating as they should do. The actual issue is the usual difficultly understanding exactly what bytecode mutations look like when translated back to source, which tripped us both up.

The mutation "removed conditional - replaced equality check with false"

Mutates this

private boolean conditionOneNotMet(TestObject2 testObject2) {
     return !condition1Checker.checkCondition1(testObject2);
}

To

private boolean conditionOneNotMet(TestObject2 testObject2) {
     condition1Checker.checkCondition1(testObject2);
     return false;
}

Retaining the method call.

If you recreate this mutant manually, you'll see that all tests pass.

So I believe everything is working as expected.