Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expectation verification functionality #318

Closed
YevgeniyShunevych opened this issue Oct 11, 2019 · 0 comments
Closed

Expectation verification functionality #318

YevgeniyShunevych opened this issue Oct 11, 2019 · 0 comments
Assignees
Labels
Milestone

Comments

@YevgeniyShunevych
Copy link
Member

@YevgeniyShunevych YevgeniyShunevych commented Oct 11, 2019

The new functionality is here to provide warnings. This functionality is similar to the meaning of "soft assertions" term.

If the test you are writing is quite complex and you need to do several assertions during the test flow without breaking the test with assertion exception when the first assertion fails, the expectation functionality is for the help.

Usage

The usage is similar to Atata's Should assertions. Just use ExpectTo instead of Should:

Component.ExpectTo.Equal("...")

It doesn't throw an exception on failure, but write the warning assertion result to the log and adds it to AtataContext.Current.PendingFailureAssertionResults collection. When on TearDown AtataContext.Current?.CleanUp() is invoked, after the actual cleaning it will throw aggregate assertion exception with all found ExpectTo errors. When using NUnit (AtataContextBuilder.UseNUnitWarningReportStrategy()), it will record every expectation failure as NUnit warning, which is similar. It is also absolutely valid to use expectations inside AggregateAssert sections.

Configuration

The following methods are added to AtataContextBuilder for configuring expectation verification functionality:

public AtataContextBuilder UseWarningReportStrategy(IWarningReportStrategy strategy);

public AtataContextBuilder UseNUnitWarningReportStrategy();

When using NUnit, it is available to invoke UseNUnitWarningReportStrategy() during AtataContext configuration in order to record warnings as NUnit's built-in warnings. For other testing frameworks (xUnit, MSTest) the native Atata expectation functionality will work well by default.

NOTE that not all test runners support/recognize NUnit warnings (for example Visual Studio 2019 test explorer does not for now, while 2015 and 2017 do).

Implementation

ExpectationVerificationStrategy

ExpectationVerificationStrategy is a core part of expectation verification functionality. Its ReportFailure method builds warning details, appends a warning into the log, adds assertion result to AtataContext.Current.AssertionResults and finally reports warning details to AtataContext.Current.WarningReportStrategy.

AtataWarningReportStrategy

Represents the native/default Atata strategy for warning assertion reporting. Adds AssertionResult object of warning kind to AtataContext.Current.PendingFailureAssertionResults collection.

NUnitWarningReportStrategy

Represents the NUnit strategy for warning assertion reporting. Invokes TestExecutionContext.CurrentContext.CurrentResult.RecordAssertion(AssertionStatus.Warning, message, stackTrace).

Add ExpectTo properties

ExpectTo property should be added to all classes that have Should property.

Additions to AtataContext

CleanUp method of AtataContext as the last action should check PendingFailureAssertionResults collection and throw AggregateAssertionException when there are some.

@YevgeniyShunevych YevgeniyShunevych added this to the 1.4.0 milestone Oct 11, 2019
@YevgeniyShunevych YevgeniyShunevych self-assigned this Oct 11, 2019
YevgeniyShunevych added a commit that referenced this issue Oct 11, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
1 participant
You can’t perform that action at this time.