Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support publishing conformance test results to Azure DevOps #1064

Closed

Conversation

CodeMonkeyLeet
Copy link
Contributor

@CodeMonkeyLeet CodeMonkeyLeet commented Aug 9, 2021

Description

  • Update conformance test to output junit xml reports that can be consumed by Azure DevOps test analytics.
  • Add conformance test workflow job to trigger pipeline in Azure DevOps that publishes the test results.
  • Add Azure pipeline definition for publishing test results from a specified GitHub workflow run to Azure DevOps.

⚠ There are a number of maintainer actions necessary for this to be useful:

  • An ADO project to publish test results into.
    • A new PAT from a Dapr service account in ADO (AZURE_DEVOPS_TOKEN) with Read and Execute build permissions for GitHub to trigger the test result publishing workflow.
    • A new build pipeline in the ADO project named components-contrib.publish-conformance-results based on the .azuredevops/pipelines/publish-test-results.yml definition from the dapr/components-contrib project after this change is merged.
  • A new PAT from a Dapr service account (GITHUB_TOKEN) with public_repo permissions for ADO to access test logs

Issue reference

POC using components-contrib conformance tests to address test dashboard requirement suggested in dapr/dapr#3316

Checklist

Please make sure you've completed the relevant tasks for this PR, out of the following list:

  • Code compiles correctly
  • Created/updated tests
  • Extended the documentation / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[issue number]

@CodeMonkeyLeet CodeMonkeyLeet requested review from a team as code owners August 9, 2021 17:47
- Update conformance test to output junit xml reports that can be
  consumed by Azure DevOps test analytics.
- Add conformance test workflow job to trigger pipeline in Azure
  DevOps that publishes the test results.
- Add Azure pipeline definition for publishing test results from a
  specified GitHub workflow run to Azure DevOps.
@CodeMonkeyLeet
Copy link
Contributor Author

@wcs1only @artursouza Putting this up so that any maintainer tasks can happen in parallel to code reviews.

@codecov
Copy link

codecov bot commented Aug 9, 2021

Codecov Report

Merging #1064 (a9cc6ae) into master (253ef85) will increase coverage by 0.00%.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #1064   +/-   ##
=======================================
  Coverage   34.53%   34.54%           
=======================================
  Files         132      132           
  Lines       10870    10870           
=======================================
+ Hits         3754     3755    +1     
+ Misses       6736     6734    -2     
- Partials      380      381    +1     
Impacted Files Coverage Δ
state/cloudstate/cloudstate_crdt.go 9.09% <0.00%> (+0.41%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 253ef85...a9cc6ae. Read the comment docs.

displayName: 'Install Python tools'

- task: PythonScript@0
displayName: 'Download test artifacts'

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shot in the dark, but I'd be curious if you could just use the Download Pipeline/Build Artifacts AzDO task: https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/utility/download-pipeline-artifact?view=azure-devops or https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/utility/download-build-artifacts?view=azure-devops

The only reason I suggest that is that both github actions and azure devops artifacts use the same backend, I'd be curious if it just works. Though I don't know what the "pipeline" name would be.

Besides that, I can't think of anything else that would work better. Maybe reaching out to someone who works on Azure DevOps may be the next best step.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the suggestion! I took a look at the task definition for that and it's a wrapper for the TFS HTTP endpoint, so it doesn't look like it's possible to get through that to the underlying shared backend with GitHub. I'll see if I can find anyone on AzDevOps who might be able to help.

@CodeMonkeyLeet
Copy link
Contributor Author

Per sync with maintainers, the desire is to switch to a pull-only model from the test dashboard service to remove the need for a GitHub action trigger. Closing this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants