Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add *.received.* to MSTest's results attachments #1183

Closed

Conversation

MattKotsenas
Copy link
Contributor

Fixes #1182

This change adds any .received. files from failing diffs to MSTest's results attachments. Attaching the received file makes it easy to download from a CI server's test results viewer without needing to modify the CI pipeline or YAML files to collect received files and upload them as build artifacts.

To accomplish that I:

  1. Updated the VerifyException to track the FilePairs for mismatches
  2. Wrap the inner verify to catch the exception and add the FilePair to the TestContext

In order to test the behavior, the logic is as follows:

  1. Create a test where the .received. will not match the .verified. (Foo and Bar in this case)
  2. Disable diffing so that unit test runs don't pop a diff tool
  3. Create a custom [TestMethod] attribute, as that's the only way I know of to access the attachments
  4. As part of ReceivedFileRequiredTestMethodAttribute's execute, verify each test has a .received. file, and if not set the test result to failed

@MattKotsenas
Copy link
Contributor Author

Opening this more to start a conversation to decide if this feature has merit. If yes, then I'm open to implementation alternatives. The current PR uses a decent amount of LINQ, but since it's in failure cases I opted for clarity to start.

If you have any questions / concerns, please let me know. Thanks!

@MattKotsenas
Copy link
Contributor Author

One alternative to possibly consider is doing the attachment in a custom test attribute instead of in the "core" verify call.

However I've found that custom test attributes:

  • tend to have low discoverability
  • don't compose well with other custom attributes

@SimonCropp
Copy link
Member

i am currently thinking of taking a diff approach https://github.com/VerifyTests/Verify/pull/1186/files

@MattKotsenas
Copy link
Contributor Author

i am currently thinking of taking a diff approach https://github.com/VerifyTests/Verify/pull/1186/files

No problem! I'm in no way married to the specific approach I took.

I'll abandon this; we can always reactivate if we need to for some reason.

Thanks!

@MattKotsenas MattKotsenas deleted the feature/mstest-attachment branch April 12, 2024 20:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

[Feature Request] Attach .received. files to test context for MSTest
2 participants