Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TRX Reporter for MultiNode tests #3904

Merged
merged 19 commits into from Oct 8, 2019

Conversation

valdisz
Copy link
Contributor

@valdisz valdisz commented Sep 5, 2019

Test reporter in TRX format for MulitNode tests. This will allow Azure DevOps and other tools, who understands TRX, to see actual test outcome.

It is possible to use either TeamCity or TRX reporter depending on parameter value:
-Dmulitnode.reporter=teamcity or -Dmulitnode.reporter=trx

-Dmultinode.teamcity switch is removed and build scripts are updated accordingly.

PR is related to #3855

@valdisz
Copy link
Contributor Author

valdisz commented Sep 5, 2019

Right now PR exists to verify that something will be reported to DevOps during a build.

Please do not look into it yet :)

@valdisz
Copy link
Contributor Author

valdisz commented Sep 5, 2019

@Aaronontheweb Incrementalist is failing :( Can you direct into a possible issue? Before I will spend some hours on it.

@Aaronontheweb
Copy link
Member

Aaronontheweb commented Sep 5, 2019 via email

@valdisz
Copy link
Contributor Author

valdisz commented Sep 6, 2019

@Aaronontheweb seems that build related token does not allow to publish test results. I can verify that this is really so, then we have 2 possibilities:

  1. Create access token with needed permissions
  2. Generate xunit test report style instead of direct reporting to devops

What you will choose?

@Aaronontheweb
Copy link
Member

@valdisz I'd just emit an XUnit-style report and I can come along later and output those TRX files up to Azure DevOps, which is what we do here:

https://github.com/akkadotnet/akka.net/blob/dev/build.fsx#L232

This forces XUnit to emit a TRX file, which can be read and understood by Azure DevOps.

And here is where we tell Azure DevOps how to consume it:

- task: PublishTestResults@2
inputs:
testRunner: VSTest
testResultsFiles: '**/*.trx' #TestResults folder usually
testRunTitle: ${{ parameters.name }}
mergeTestResults: true

So all we really need is just a way to output a .TRX file - does that make sense?

@valdisz valdisz changed the title MultiNode test reporter for Azure DevOps TRX Reporter for MultiNode tests Sep 22, 2019
@valdisz valdisz marked this pull request as ready for review September 22, 2019 19:48
@valdisz
Copy link
Contributor Author

valdisz commented Sep 22, 2019

@Aaronontheweb you can look on this.

@Aaronontheweb
Copy link
Member

@valdisz will do

Copy link
Member

@Aaronontheweb Aaronontheweb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have some changes.

I don't think this could should ship in separate projects - could you please put all of it under the Akka.MultiNodeTestRunner.Shared package instead? In its own namespace?

@Aaronontheweb
Copy link
Member

@valdisz thanks for answering my question about the test structure - will review this again once the other changes are implemented.

Copy link
Member

@Aaronontheweb Aaronontheweb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Aaronontheweb Aaronontheweb merged commit 3345b1d into akkadotnet:dev Oct 8, 2019
@valdisz valdisz deleted the feature/mntr-test-output branch October 21, 2019 16:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants