Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Has.All.../Has.None... - show non-matching items in error message #2854

Closed
Dreamescaper opened this issue May 11, 2018 · 7 comments
Closed
Assignees
Milestone

Comments

@Dreamescaper
Copy link
Member

Dreamescaper commented May 11, 2018

Currently NUnit only logs actual collection in error message. That's fine if we have 3 items, but not really great for 200, especially due to that only several first items are displayed.
Would be really great to log non-matching items as well, something like:

Expected: all items instance of <System.Char>
Actual: < 'a', \"b\", 'c' >
Non-Matching: < \"b\" > 

That would require to traverse through all collection though, even if the first checked item does not match, but I suppose that's OK, considering that it might happen only for failing test.

@CharliePoole
Copy link
Member

Other errors often only show the first failure. For example, when comparing two arrays for equality, we show the index of the mismatch as well as the values. Seems as if that would work here as well and avoids re-examining the collection.

I do think we need to take a broad look at various error messages applied to collections.

@Dreamescaper
Copy link
Member Author

Do you think that's important?
As I mentioned, in order test to pass it will need to traverse the whole collection anyway.

The only case I can think of is to use NotConstraint with AllItemsConstraint, but it makes no sense considering that we have separate SomeItemConstraint.

If you still think we should break on the first non-mathcing item, what message would you suggest?
Non-Matching: "b" ?

@Dreamescaper
Copy link
Member Author

Dreamescaper commented May 12, 2018

Or something like Non-matching item at index [1]: "b" ?

@Dreamescaper
Copy link
Member Author

Decided to go with the last one so far, created PR for that.

@CharliePoole
Copy link
Member

Have you looked at the output of comparing two arrays for equality? The location of the mismatch is handled in one line, followed by the precise failure. Maybe that's the general model to use.

@Dreamescaper
Copy link
Member Author

Dreamescaper commented May 12, 2018

Not sure I understand your suggestion. What should be on the following line (precise failure)?
In EqualConstraintResult it's simple expected-actual text, with special cases handling (like strings, enumerables etc.), as far as I can see.

Not sure that applies here, because:

a) actual result is same for all elements, therefore it will be simply duplicated:

Expected: all items instance of <System.Char>
Actual: < 'a', "b", 'c' >
Non-matching item at index [1]: 
	Expected: instance of <System.Char>
	Actual: "b"

b) in EqualConstraint we know that we need to check specifically for equality, but here it could be any condition, therefore I don't think we can handle any special cases. We could provide BaseConstraints message for that purpose, but see a) - that will simply duplicate first Expected line, won't it?

@jnm2
Copy link
Contributor

jnm2 commented May 27, 2018

I like this, especially because we may decide to trim the number of items printed:

Expected: all items instance of <System.Char>
Actual: < 'a', 'b', 'c', 'd', 'e', ...>
Non-matching item at index [25]: "z"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants