NullReferenceException using Is.EqualTo on two unequal strings

[MarcusGast]

I'm writing some tests and wanted to test the equality of two strings. I didn't realise until runtime that they would be unequal. I expected the Assert to fail but not with a null reference exception.

Text comparison as been shortened for ease of reading but the result is the same.

       'string errorText = ScenarioContext.Current.Get<FlightSearchPod>().errorOnScreen;

        string requiredText = "You can only book one infant per adult online.";

        Assert.That(errorText, Does.Contain(requiredText));`

At runtime errorText = "Oops!\r\nYou can only book one infant per adult online.".

If I change requiredText to "Oops!\r\nYou can only book one infant per adult online." it works without an exception.
This Exception is only thrown on v3.6.0, If I try the same Assert on earlier versions it works fine so I'm trying to determine if it's a feature or an issue.

I can get around the issue by using Does.Contain but have been asked by my organisation to see if we can identify the cause of this unexpected exception.

Let me know if I need to provide any more information, I'm fairly new to automation testing.

[JustinRChou]

ScenarioContext.Current.Get<FlightSearchPod>().errorOnScreen;

It seems that it is more likely that somewhere in your chain is a null.

string errorText = "Oops!\r\nYou can only book one infant per adult online.";
string requiredText = "Oops!\r\nYou can only book one infant per adult online.";

Assert.That(errorText,Is.EqualTo(requiredText));

That is working for me.

[MarcusGast]

Yeah the code snippet you posted works fine it only throws the NullReferenceException when the strings don't match and only on 3.6.0.
The reason I put the string retrieval into variables before comparison is to prove that they are what I expect before comparison, not sure what else could be null.

[JustinRChou]

What nunit adapter version are you using?
I am on NUnit Adapter 3.7.0.0

[MarcusGast]

Hmmm I'm not using the NUnit Adapter package just the framework package, do I need the adapter package?

[MarcusGast]

FYI I just got the same issue again but this time testing equality of two doubles. Seems it happens whenever the objects aren't equal.

[CharliePoole]

No need for the adapter if that's not how you are running your tests. How are you running them?

Is this happening in an async method? Or in a thread created by your test?

[MarcusGast]

Using Specflow and Specrunner to run the tests.

I'm not creating an Async method so presumably it's a thread created by my test.

[CharliePoole]

Presumably? Either you have code to create a thread or you don't. 😄

Can you run the tests with nunit3-console?

[MarcusGast]

Sorry I've been writing automated tests for two weeks so a lot of the concepts are still new to me. I'm not explicitly creating threads. Tests are just being run via specflow feature files, I don't know how specflow handles threads.

As for the nunit3 console, I don't know how, will need to look into it.

[CharliePoole]

OK, as another approach to this, can you post the stack trace from the exception? I'm suspecting this is related to another bug report.

[CharliePoole]

Unfortunately, most of us who write NUnit know even less about. SpecFlow than you. 😄

[MarcusGast]

Here you go, changed the filepath for client confidentiality reasons but everything is as is:

at NUnit.Framework.Assert.ReportFailure(String message)
at NUnit.Framework.Assert.ReportFailure(ConstraintResult result, String message, Object[] args)
at NUnit.Framework.Assert.That[TActual](TActual actual, IResolveConstraint expression, String message, Object[] args)
at NUnit.Framework.Assert.That[TActual](TActual actual, IResolveConstraint expression)
at Specification.FlightSearchSteps.ThenIShouldBeInformedThisIsAnInvalidScenario() in C:\Projects\Test\Specification\FlightSearchSteps.cs:line 144
at lambda_method(Closure , IContextManager )

[CharliePoole]

Looks very much like #1965 and #1952 stack traces. I'll check it out in the morning.

[JustinRChou]

Does SpecFlow+ work with NUnit 3.6?

I am looking at the free version and it installs something like version 3.0.0 and it seems to generate the test.dll and runs it through the nunit-console.

At the moment, I can't seem to find how do they write the test fixtures.

[JustinRChou]

SpecFlow Generated Test Fixture.txt

The TestRunner runs on it's own thread.

[CharliePoole]

I think this was fixed along with #2035 and #2050. Can you retest using one of the builds pointed to from those issues?

[MarcusGast]

Recreated the issue again in v3.6.0 and then retested in v3.6.1 of the NUnit framework with the same code and it is now working as expected. Thanks for the assistance Charlie.