Add built-in support to exit an Assert.Multiple() block early

[stevenaw]

As discussed in #4451

It may be helpful to have a built-in way to exit an Assert.Multiple() block early. Currently the only ways are:

• Throw an exception
  • This will cause the multiple to fail, which may not be desired
• Explicit return statement in the lambda
  • This has the obvious limitation it must be in the lambda body directly
  • This will not exit nested Assert.Multiple() in a single command

Some ideas discussed in #4451 are:

• Adding a new method such as Assert.MultipleExit() or Assert.FailFast()
• Adding a parameter to an existing method
• Something else?
  • A MultipleAssertionContext passed into the lambda by the framework as an argument for the lambda

[manfred-brands]

One more thing to consider, it the FailFast exiting only one or all levels of a nested Assert.Multiple().
I suggest only one level as the code at that point doesn't know about higher levels.

[stevenaw]

True. If we use a method-based approach, I wonder if we'd want to allow that to be configurable.

Assert.FailFast(AssertMultipleScope.Immediate);
Assert.FailFast(AssertMultipleScope.All);

To me the name FailFast() implies it should do all scopes but I agree that we'd likely want to default behaviour of this feature to only be one scope so perhaps that's another argument against the name FailFast() for this

[stevenaw]

I find the idea of something like this intriguing too:

public class MultipleAssertContext {
  public MultipleAssertContext? ParentContext { get; }
  public int FailureCount { get; }
  public int TotalFailureCount { get; }
  public void Exit();
}

and then:

Assert.Multiple((context) => {
  // Lots of complicated assertions

  if (arbitraryCondition)
  {
    context.Exit();
  }

  Assert.Fail("This doesn't run");
});

[manfred-brands]

I don't think we should add a parameter, potentially we can add something to TestExecutionContext instead,

There is currently no hierarchy of MultipleAssertContext, there is only an (internal) MultipleAssertLevel property.

I don't see many test themselves care about FailureCount.
This is already available as TestContext.CurrentContext.CurrentResult.PendingFailures

[stevenaw]

I could see TestExecutionContext making sense for this. Did you have anything in particular in mind?

[stevenaw]

I think the introduction of the AssertionScope also makes this a bit easier to exit the test method early based on arbitrary criteria as the individual assertions no longer need to occur in a callback.

@manfred-brands Reaching out as you're the other to have been involved in discussion here. I'm inclined to close this, but feel free to reopen if you feel otherwise

[manfred-brands]

@stevenaw I agree on closing this. If early exit is wanted, why are they combined in a single Assert.Multiple?
It is easy to split them if the 2nd is not independent.