[Suggestion] Improve TextMessageWriter output for numeric values

[nzain]

When comparing double values it is sometimes difficult to see why the test failed. Example:

        [Test]
        public void CompareDoubleValues()
        {
            double actual = -25.148640873240467;
            double expected = -24.685244717451383;
            const double tolerance = 0.4;
            Assert.That(actual, Is.EqualTo(expected).Within(tolerance));
        }

results in the following message:

  Expected: -24.685244717451383d +/- 0.40000000000000002d
  But was:  -25.148640873240467d

It is immediately obvious, that the numbers differ. And you trust nunit to properly handle the tolerance. However, to fix the issue you have to understand what's wrong with my number? Is my number too large? Oh wait, they are signed. It's actually too small. Was my tolerance reasonable? Well, what is the actual difference?

Proposed Message
If we have two values and a tolerance, it would be great to see the actual difference as well. In my oppinion, that would improve the readability significantly for weird numbers or tiny tolerances.

  Expected: -24.685244717451383d +/- 0.40000000000000002d
  But was:  -25.148640873240467d  (differs by -0.463396155789084)

Of course I can output the difference on my own, e.g. using nunit's message argument. However, I think that extra work could be done once in nunit instead of every time on the developer side.

Remarks
nunit 3.12 (latest at the time of writing)
Visual Studio 2017 professional + ReSharper to run the tests
or dotnet core with dotnet test

I wanted to create a PR, but VS2017 cannot compile the solution.
The relevant parts of the code should be

nunit/src/NUnitFramework/framework/Constraints/EqualConstraintResult.cs

Line 96 in fd63471

writer.DisplayDifferences(expected, actual, tolerance);

and

nunit/src/NUnitFramework/framework/Internal/Execution/TextMessageWriter.cs

Lines 166 to 175 in fd63471

	public override void DisplayDifferences(object expected, object actual, Tolerance tolerance)
	{
	if (expected != null && actual != null && expected.GetType() != actual.GetType() && MsgUtils.FormatValue(expected) == MsgUtils.FormatValue(actual))
	{
	_sameValDiffTypes = true;
	ResolveTypeNameDifference(expected, actual, out _expectedType, out _actualType);
	}
	WriteExpectedLine(expected, tolerance);
	WriteActualLine(actual);
	}

but I might be wrong, since I couldn't step into the code.

Detection of numerical values is probably required to work with object arguments - there is nice code available in Numerics.cs. This class could be extended with methods like GetDifference(object expected, object actual, Tolerance tolerance).

PS: Thanks for this awesome test framework!

[ChrisMaddock]

I think this would be a nice improvement. 🙂 It would be specific to cases with a tolerance, right? Is this something you'd be willing to contribute? I think it'll be reasonably approachable.

In terms of implementation, take a look at ConstraintResult.WriteAdditionalLinesTo(). That's the method which is intended to be used for cases like this.

If you do want to go ahead with this, probably best just to hold on for a couple of other team members to comment first, before getting stuck into the code.

[nzain]

@ChrisMaddock yes, only makes sense with tolerance. I'm working on 3D geometry software/data... 90% double values :-)

I'm not sure how to contribute, my VS2017 cannot compile nunit. Is it possible to compile & test with VS Code? To be honest, I didn't consider that.

[nzain]

I just found out that NUnit allows me to change the way a tolerance is interpreted, e.g. using

   Assert.That(987654, Is.EqualTo(987651).Within(1).Percent);

On the one hand, this complicates things quite a bit, since the absolute difference is not interesting in that case - you would want to see the difference in percent as well. On the other hand, in that scenario it is even more important to display something meaningful to improve result readability.

[ChrisMaddock]

I’d recommend VS 2019 community edition personally, but vs code should work as well. 😊

[nzain]

I cannot install VS2019 without breaking some small things (e.g. msbuild locator) for some of our projects. VS Code doesn't work: dotnet build complains about missing net35 SDK, which is actually installed (I'm facing dotnet/msbuild#1333 or similar again). The build scripts don't work, they depend on latest msbuild (VS2017 comes with msbuild 15.9).

I would like to contribute, but too many obstacles are blocking my way.

[ChrisMaddock]

@nzain Sorry to hear this. Will leave this issue open, in case anyone else wishes to contribute. 🙂

[Antash]

@ChrisMaddock Hi, I've noticed this issue. Seems to be very nice to have so I've created a PR.

[ChrisMaddock]

Great! Afraid I don't work on this part of the project any more, but I'm sure the team that do will take a look at the pull requests soon - thanks for doing them! 😄