Using 'Verify' and 'It.Is<>' with a method call inside cases the same verify the run for all invocations

[AntonVasserman]

Hi moq folks,

So far we have been using Mock<>.Verify with It.Is<> inside and always there was a single evaluation. (meaning we only checked one of the invocation to match, which allowed us to write two Verifies over the same method, with different inputs and verify two different invocations)

Today I have noticed that isn't the case if inside the It.Is<> match Func we call another method.
In that case the Verify method actually runs a verification against all invocations of that method.

Here is a simple reproduction:

using Moq;
using Xunit;

namespace ConsoleAppNet7;

public class Program
{
    public static void Main(string[] args)
    {
        Mock<ISomeInterface> mock = new();
        SomeClass someClass = new(mock.Object);
        SomeModel someModel1 = new() { Value = 1 };
        SomeModel someModel2 = new() { Value = 2 };
        List<SomeModel> list = new List<SomeModel> { someModel1, someModel2 };
        someClass.BatchedMethod(list);

        // This first call actually verifies both invocations
        mock.Verify(m => m.SomeMethod(It.Is<SomeModel>(actualModel => ValidateModel(actualModel, someModel1))), Times.Once());
        // This second call is never even called since the first one fails on its second run due to 'actualModel' being 'someModel2'...
        mock.Verify(m => m.SomeMethod(It.Is<SomeModel>(actualModel => ValidateModel(actualModel, someModel2))), Times.Once());

        Console.WriteLine("Finished");
    }

    private static bool ValidateModel(SomeModel actualModel, SomeModel originalModel)
    {
        Assert.Equal(actualModel.Value, originalModel.Value);
        return true;
    }
}

public class SomeModel { public int Value { get; set; } }
public interface ISomeInterface { void SomeMethod(SomeModel model); }
public class SomeClass
{
    private readonly ISomeInterface _someInterface;
    public SomeClass(ISomeInterface someInterface) => _someInterface = someInterface;
    public void BatchedMethod(List<SomeModel> models)
    {
        foreach (SomeModel model in models)
        {
            _someInterface.SomeMethod(model);
        }
    }
}

Is it a bug or by design?
If it's by design how can we properly verify the two invocations? Currently we are resulting in not using a method call which make the code a bit cumbersome.

[stakx]

Hi @AntonVasserman. I haven't run your code, but by just looking at it, I suspect the problem doesn't lie with Moq, but with your usage of Assert.Equal inside ValidateModel. Assert.Equal may throw exceptions. The predicate function passed to It.Is<> OTOH should never to throw, but simply return a bool indicating whether a positive match was made or not. That is, your ValidateModel method should convert thrown exceptions to a return value of false (or even more simply, just use object.Equals instead of Assert.Equal); then the rest of your code should work as expected.

(Also, if it isn't self-evident: mock.Verify has to run the predicate against all invocations to figure out how many of them match the predicate; that is, you should expect that your predicate gets run against invocations that may not match it. That's why the use of a possibly-throwing Assert.Equal inside the predicate is an error.)

[AntonVasserman]

@stakx No I don't think this is related.
The only reason ValidateModel throws is because there is a second execution of the It.Is<> for some reason.
I have changed the code to this:

public class Program
{
    private static int _count = 0;

    public static void Main(string[] args)
    {
        Mock<ISomeInterface> mock = new();
        SomeClass someClass = new(mock.Object);
        SomeModel someModel1 = new() { Value = 1 };
        SomeModel someModel2 = new() { Value = 2 };
        List<SomeModel> list = new List<SomeModel> { someModel1, someModel2 };
        someClass.BatchedMethod(list);

        // This first call actually verifies both invocations
        mock.Verify(m => m.SomeMethod(It.Is<SomeModel>(actualModel => ValidateModel(actualModel, someModel1))), Times.Exactly(2));
        // This second call is never even called since the first one fails on its second run due to 'actualModel' being 'someModel2'...
        // mock.Verify(m => m.SomeMethod(It.Is<SomeModel>(actualModel => ValidateModel(actualModel, someModel2))), Times.Once());

        Console.WriteLine($"Finished: {_count}");
    }

    private static bool ValidateModel(SomeModel actualModel, SomeModel originalModel)
    {
        // Assert.Equal(actualModel.Value, originalModel.Value);
        if (originalModel.Value == 1) _count++;
        return true;
    }
}

And this is the value I get:

This is even though we, supposedly, call mock.Verify(m => m.SomeMethod(It.Is<SomeModel>(actualModel => ValidateModel(actualModel, someModel1))), Times.Exactly(2)); once, so it clearly means that the method is called twice for some reason.
(Note I had to change Times.Once to Times.Exactly(2) so it won't throw...)

[stakx]

You have two invocations, so your predicate runs twice (once per invocation) per Verify call. Am I still misunderstanding?

Verify cannot magically know which invocations will match the predicate and which ones won't. It uses the predicate to determine that, which is why the predicate runs once per invocation.

[stakx]

Note I had to change Times.Once to Times.Exactly(2) so it won't throw...

This doesn't match with your code comments in the example shown:

// This first call actually verifies both invocations
mock.Verify(m => m.SomeMethod(It.Is<SomeModel>(actualModel => ValidateModel(actualModel, someModel1))), Times.Exactly(2));
// This second call is never even called since the first one fails on its second run due to 'actualModel' being 'someModel2'...
// mock.Verify(m => m.SomeMethod(It.Is<SomeModel>(actualModel => ValidateModel(actualModel, someModel2))), Times.Once());

[AntonVasserman]

You have two invocations, so your predicate runs twice (once per invocation) per Verify call. Am I still misunderstanding?

Verify cannot magically know which invocations will match the predicate and which ones won't. It uses the predicate to determine that, which is why the predicate runs once per invocation.

Okay that now makes total sense... The issue is indeed with the Assert throwing.
Since we indeed have to go through all the invocations to validate the predicate, once we execute the validation with the wrong input the Assert will throw.

Thank you very much for the quick help and responses, and for clarifying the case here!

[stakx]

You're welcome. Glad I could help with clearing things up.