Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Azure Pipelines] Run pytest tests (and publish results) on macOS #14133

Merged
merged 2 commits into from Nov 20, 2018

Conversation

foolip
Copy link
Member

@foolip foolip commented Nov 19, 2018

Two changes in this PR:

The decision job running in a Linux VM will decide whether or not jobs
running on other OSes start at all. Most templates are now only used
once, but most ought to be reused when running affected tests.

And run pytest tests and public results using this nifty task:
https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/test/publish-test-results

@foolip foolip changed the title [Azure Pipelines] Introduce a decision job and lots of templates [Azure Pipelines] Run pytest tests (and publish results) on macOS Nov 19, 2018
@foolip
Copy link
Member Author

foolip commented Nov 19, 2018

Pardon the rename, I hit buttons too fast. Also want to see Azure Pipelines go green for the first commit before I push the next.

@foolip
Copy link
Member Author

foolip commented Nov 19, 2018

See https://dev.azure.com/web-platform-tests/wpt/_build/results?buildId=848&view=logs for proof that the first commit works. (I intend for this PR to be rebased with separate commits.)

@foolip
Copy link
Member Author

foolip commented Nov 19, 2018

https://dev.azure.com/web-platform-tests/wpt/_build/results?buildId=850&view=logs shows the intended outcome of this. I missed the important part in the first push, it's all in the fixup :)

@foolip
Copy link
Member Author

foolip commented Nov 19, 2018

@Hexcles @lukebjerring requesting your review since it's probably a good idea for more of us to understand this setup when I add affected tests to it.

@lukebjerring
Copy link
Contributor

Can confirm I read through, but any review would merely be a rubber stamp.

Is that publish results tool flexible enough for custom results ( WPT reports ) in future?

@foolip foolip force-pushed the foolip/azure-pipelines-pytest branch from 12fc45d to 5641708 Compare November 19, 2018 23:34
@foolip
Copy link
Member Author

foolip commented Nov 19, 2018

Is that publish results tool flexible enough for custom results ( WPT reports ) in future?

If we could convert our results into the JUnit XML format then we could see something cool for ./wpt run as well. However, I suspect we'd run into some trouble with the assumption that each test is a method and we just have a bunch of strings as our test names. Totally worth experimenting with though, there might be workarounds.

.azure-pipelines.yml Show resolved Hide resolved
tools/ci/azure/install_certs.yml Show resolved Hide resolved
Copy link
Member

@Hexcles Hexcles left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM % some minor nits & questions

tools/ci/azure/tox_pytest.yml Show resolved Hide resolved
tools/ci/azure/pip_install.yml Show resolved Hide resolved
tools/ci/azure/tox_pytest.yml Outdated Show resolved Hide resolved
The decision job running in a Linux VM will decide whether or not jobs
running on other OSes start at all. Most templates are now only used
once, but most ought to be reused when running affected tests.
@foolip foolip force-pushed the foolip/azure-pipelines-pytest branch from cbf9dbb to 12b87d6 Compare November 20, 2018 20:45
@foolip foolip merged commit e340f91 into master Nov 20, 2018
@foolip foolip deleted the foolip/azure-pipelines-pytest branch November 20, 2018 21:20
@foolip foolip added this to In progress in Azure Pipelines via automation Nov 20, 2018
@foolip foolip moved this from In progress to Reviewer approved in Azure Pipelines Nov 20, 2018
@foolip foolip moved this from Reviewer approved to Done in Azure Pipelines Nov 20, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

None yet

5 participants