ci(new workflow): Run tests on master every 12 hours #13334
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We recently started capturing coverage data for Pull Requests again but we omitted tracking for master. Part of the reason for this is that the our CI setup uses the same binaries that were used for the end-to-end tests as part of the release. And we don't want to upload the coverage-instrumented binaries for a release. This is fixable, but instead of making our CI set up even more complex, this introduces a new workflow to test master periodically. This workflow will run every 12 hours, running all tests on master, with acceptance tests on both Windows and macOS, track coverage for all of this, and post that to codecov. We'll be able to track code coverage trends on master in Codecov.
|
Current dependencies on/for this PR: This comment was auto-generated by Graphite. |
abhinav
added
the
impact/no-changelog-required
Changelog[uncommitted] (2023-06-29) |
justinvp
approved these changes
Jun 29, 2023
|
bors merge |
|
Build succeeded! The publicly hosted instance of bors-ng is deprecated and will go away soon. If you want to self-host your own instance, instructions are here. If you want to switch to GitHub's built-in merge queue, visit their help page. |
abhinav
added a commit
to pulumi/pulumi-yaml
that referenced
this pull request
Aug 1, 2023
Adds a `make test_cover` target to the Makefile which builds pulumi-language-yaml with coverage instrumentation and runs all tests with coverage tracking. Note that right now, there are no tests that invoke pulumi-language-yaml so integration test data is empty. For CI, adds an coverage option that, when enabled, will run `make test_cover` instead of `make test`, and upload the results to codecov. This option is only enabled for tests invoked for PRs and by the `/run-acceptance-tests` command. Additionally, similarly to pulumi/pulumi#13334, this adds a workflow that runs tests on master with coverage tracking every 12 hours. Resolves #464
abhinav
added a commit
to pulumi/pulumi-yaml
that referenced
this pull request
Aug 1, 2023
Adds a `make test_cover` target to the Makefile which builds pulumi-language-yaml with coverage instrumentation and runs all tests with coverage tracking. Note that right now, there are no tests that invoke pulumi-language-yaml so integration test data is empty. For CI, adds an coverage option that, when enabled, will run `make test_cover` instead of `make test`, and upload the results to codecov. This option is only enabled for tests invoked for PRs and by the `/run-acceptance-tests` command. Additionally, similarly to pulumi/pulumi#13334, this adds a workflow that runs tests on master with coverage tracking every 12 hours. Resolves #464
abhinav
added a commit
to pulumi/pulumi-yaml
that referenced
this pull request
Aug 1, 2023
Adds a `make test_cover` target to the Makefile which builds pulumi-language-yaml with coverage instrumentation and runs all tests with coverage tracking. Note that right now, there are no tests that invoke pulumi-language-yaml so integration test data is empty. For CI, adds an coverage option that, when enabled, will run `make test_cover` instead of `make test`, and upload the results to codecov. This option is only enabled for tests invoked for PRs and by the `/run-acceptance-tests` command. Additionally, similarly to pulumi/pulumi#13334, this adds a workflow that runs tests on master with coverage tracking every 12 hours. Resolves #464
abhinav
added a commit
to pulumi/pulumi-yaml
that referenced
this pull request
Aug 1, 2023
Adds a `make test_cover` target to the Makefile which builds pulumi-language-yaml with coverage instrumentation and runs all tests with coverage tracking. Note that right now, there are no tests that invoke pulumi-language-yaml so integration test data is empty. For CI, adds an coverage option that, when enabled, will run `make test_cover` instead of `make test`, and upload the results to codecov. This option is only enabled for tests invoked for PRs and by the `/run-acceptance-tests` command. Additionally, similarly to pulumi/pulumi#13334, this adds a workflow that runs tests on master with coverage tracking every 12 hours. Resolves #464
abhinav
added a commit
to pulumi/pulumi-yaml
that referenced
this pull request
Aug 1, 2023
Adds a `make test_cover` target to the Makefile which builds pulumi-language-yaml with coverage instrumentation and runs all tests with coverage tracking. Note that right now, there are no tests that invoke pulumi-language-yaml so integration test data is empty. For CI, adds an coverage option that, when enabled, will run `make test_cover` instead of `make test`, and upload the results to codecov. This option is only enabled for tests invoked for PRs and by the `/run-acceptance-tests` command. Additionally, similarly to pulumi/pulumi#13334, this adds a workflow that runs tests on master with coverage tracking every 12 hours. Resolves #464
abhinav
added a commit
to pulumi/pulumi-yaml
that referenced
this pull request
Aug 1, 2023
Adds a `make test_cover` target to the Makefile which builds pulumi-language-yaml with coverage instrumentation and runs all tests with coverage tracking. Note that right now, there are no tests that invoke pulumi-language-yaml so integration test data is empty. For CI, adds an coverage option that, when enabled, will run `make test_cover` instead of `make test`, and upload the results to codecov. This option is only enabled for tests invoked for PRs and by the `/run-acceptance-tests` command. Additionally, similarly to pulumi/pulumi#13334, this adds a workflow that runs tests on master with coverage tracking every 12 hours. Resolves #464
abhinav
added a commit
to pulumi/pulumi-yaml
that referenced
this pull request
Aug 1, 2023
Adds a `make test_cover` target to the Makefile which builds pulumi-language-yaml with coverage instrumentation and runs all tests with coverage tracking. Note that right now, there are no tests that invoke pulumi-language-yaml so integration test data is empty. For CI, adds an coverage option that, when enabled, will run `make test_cover` instead of `make test`, and upload the results to codecov. This option is only enabled for tests invoked for PRs and by the `/run-acceptance-tests` command. Additionally, similarly to pulumi/pulumi#13334, this adds a workflow that runs tests on master with coverage tracking every 12 hours. Resolves #464
abhinav
added a commit
to pulumi/pulumi-yaml
that referenced
this pull request
Aug 1, 2023
Adds a `make test_cover` target to the Makefile which builds pulumi-language-yaml with coverage instrumentation and runs all tests with coverage tracking. Note that right now, there are no tests that invoke pulumi-language-yaml so integration test data is empty. For CI, adds an coverage option that, when enabled, will run `make test_cover` instead of `make test`, and upload the results to codecov. This option is only enabled for tests invoked for PRs and by the `/run-acceptance-tests` command. Additionally, similarly to pulumi/pulumi#13334, this adds a workflow that runs tests on master with coverage tracking every 12 hours. Resolves #464
abhinav
added a commit
to pulumi/pulumi-yaml
that referenced
this pull request
Aug 2, 2023
Adds a `make test_cover` target to the Makefile which builds pulumi-language-yaml with coverage instrumentation and runs all tests with coverage tracking. Note that right now, there are no tests that invoke pulumi-language-yaml so integration test data is empty. For CI, adds an coverage option that, when enabled, will run `make test_cover` instead of `make test`, and upload the results to codecov. This option is only enabled for tests invoked for PRs and by the `/run-acceptance-tests` command. Additionally, similarly to pulumi/pulumi#13334, this adds a workflow that runs tests on master with coverage tracking every 12 hours. Resolves #464
abhinav
added a commit
to pulumi/pulumi-yaml
that referenced
this pull request
Aug 2, 2023
Adds a `make test_cover` target to the Makefile which builds pulumi-language-yaml with coverage instrumentation and runs all tests with coverage tracking. Note that right now, there are no tests that invoke pulumi-language-yaml so integration test data is empty. For CI, adds an coverage option that, when enabled, will run `make test_cover` instead of `make test`, and upload the results to codecov. This option is only enabled for tests invoked for PRs and by the `/run-acceptance-tests` command. Additionally, similarly to pulumi/pulumi#13334, this adds a workflow that runs tests on master with coverage tracking every 12 hours. Resolves #464
abhinav
added a commit
to pulumi/pulumi-yaml
that referenced
this pull request
Aug 7, 2023
Adds a `make test_cover` target to the Makefile which builds pulumi-language-yaml with coverage instrumentation and runs all tests with coverage tracking. Note that right now, there are no tests that invoke pulumi-language-yaml so integration test data is empty. For CI, adds an coverage option that, when enabled, will run `make test_cover` instead of `make test`, and upload the results to codecov. This option is only enabled for tests invoked for PRs and by the `/run-acceptance-tests` command. Additionally, similarly to pulumi/pulumi#13334, this adds a workflow that runs tests on master with coverage tracking every 12 hours. Resolves #464
abhinav
added a commit
to pulumi/pulumi-yaml
that referenced
this pull request
Aug 8, 2023
Adds a `make test_cover` target to the Makefile which builds pulumi-language-yaml with coverage instrumentation and runs all tests with coverage tracking. Note that right now, there are no tests that invoke pulumi-language-yaml so integration test data is empty. For CI, adds an coverage option that, when enabled, will run `make test_cover` instead of `make test`, and upload the results to codecov. This option is only enabled for tests invoked for PRs and by the `/run-acceptance-tests` command. Additionally, similarly to pulumi/pulumi#13334, this adds a workflow that runs tests on master with coverage tracking every 12 hours. Resolves #464
abhinav
added a commit
to pulumi/pulumi-yaml
that referenced
this pull request
Aug 8, 2023
Adds a `make test_cover` target to the Makefile which builds pulumi-language-yaml with coverage instrumentation and runs all tests with coverage tracking. Note that right now, there are no tests that invoke pulumi-language-yaml so integration test data is empty. For CI, adds an coverage option that, when enabled, will run `make test_cover` instead of `make test`, and upload the results to codecov. This option is only enabled for tests invoked for PRs and by the `/run-acceptance-tests` command. Additionally, similarly to pulumi/pulumi#13334, this adds a workflow that runs tests on master with coverage tracking every 12 hours. Resolves #464
abhinav
added a commit
to pulumi/pulumi-yaml
that referenced
this pull request
Aug 8, 2023
Adds a `make test_cover` target to the Makefile which builds pulumi-language-yaml with coverage instrumentation and runs all tests with coverage tracking. Note that right now, there are no tests that invoke pulumi-language-yaml so integration test data is empty. For CI, adds an coverage option that, when enabled, will run `make test_cover` instead of `make test`, and upload the results to codecov. This option is only enabled for tests invoked for PRs and by the `/run-acceptance-tests` command. Additionally, similarly to pulumi/pulumi#13334, this adds a workflow that runs tests on master with coverage tracking every 12 hours. Resolves #464
abhinav
added a commit
to pulumi/pulumi-yaml
that referenced
this pull request
Aug 9, 2023
Adds a `make test_cover` target to the Makefile which builds pulumi-language-yaml with coverage instrumentation and runs all tests with coverage tracking. Note that right now, there are no tests that invoke pulumi-language-yaml so integration test data is empty. For CI, adds an coverage option that, when enabled, will run `make test_cover` instead of `make test`, and upload the results to codecov. This option is only enabled for tests invoked for PRs and by the `/run-acceptance-tests` command. Additionally, similarly to pulumi/pulumi#13334, this adds a workflow that runs tests on master with coverage tracking every 12 hours. Resolves #464
abhinav
added a commit
to pulumi/pulumi-yaml
that referenced
this pull request
Aug 28, 2023
Adds a `make test_cover` target to the Makefile which builds pulumi-language-yaml with coverage instrumentation and runs all tests with coverage tracking. Note that right now, there are no tests that invoke pulumi-language-yaml so integration test data is empty. For CI, adds an coverage option that, when enabled, will run `make test_cover` instead of `make test`, and upload the results to codecov. This option is only enabled for tests invoked for PRs and by the `/run-acceptance-tests` command. Additionally, similarly to pulumi/pulumi#13334, this adds a workflow that runs tests on master with coverage tracking every 12 hours. Resolves #464
abhinav
added a commit
to pulumi/pulumi-yaml
that referenced
this pull request
Aug 28, 2023
Adds a `make test_cover` target to the Makefile which builds pulumi-language-yaml with coverage instrumentation and runs all tests with coverage tracking. Note that right now, there are no tests that invoke pulumi-language-yaml so integration test data is empty. For CI, adds an coverage option that, when enabled, will run `make test_cover` instead of `make test`, and upload the results to codecov. This option is only enabled for tests invoked for PRs and by the `/run-acceptance-tests` command. Additionally, similarly to pulumi/pulumi#13334, this adds a workflow that runs tests on master with coverage tracking every 12 hours. Resolves #464
abhinav
added a commit
to pulumi/pulumi-yaml
that referenced
this pull request
Aug 29, 2023
Adds a `make test_cover` target to the Makefile which builds pulumi-language-yaml with coverage instrumentation and runs all tests with coverage tracking. Note that right now, there are no tests that invoke pulumi-language-yaml so integration test data is empty. For CI, adds an coverage option that, when enabled, will run `make test_cover` instead of `make test`, and upload the results to codecov. This option is only enabled for tests invoked for PRs and by the `/run-acceptance-tests` command. Additionally, similarly to pulumi/pulumi#13334, this adds a workflow that runs tests on master with coverage tracking every 12 hours. Resolves #464
justinvp
pushed a commit
to pulumi/pulumi-yaml
that referenced
this pull request
Sep 4, 2023
Adds a `make test_cover` target to the Makefile which builds pulumi-language-yaml with coverage instrumentation and runs all tests with coverage tracking. Note that right now, there are no tests that invoke pulumi-language-yaml so integration test data is empty. For CI, adds an coverage option that, when enabled, will run `make test_cover` instead of `make test`, and upload the results to codecov. This option is only enabled for tests invoked for PRs and by the `/run-acceptance-tests` command. Additionally, similarly to pulumi/pulumi#13334, this adds a workflow that runs tests on master with coverage tracking every 12 hours. Resolves #464
justinvp
pushed a commit
to pulumi/pulumi-yaml
that referenced
this pull request
Sep 4, 2023
* ci: Add coverage tracking Adds a `make test_cover` target to the Makefile which builds pulumi-language-yaml with coverage instrumentation and runs all tests with coverage tracking. Note that right now, there are no tests that invoke pulumi-language-yaml so integration test data is empty. For CI, adds an coverage option that, when enabled, will run `make test_cover` instead of `make test`, and upload the results to codecov. This option is only enabled for tests invoked for PRs and by the `/run-acceptance-tests` command. Additionally, similarly to pulumi/pulumi#13334, this adds a workflow that runs tests on master with coverage tracking every 12 hours. Resolves #464 * Add gocov tool Adds an experimental tool that runs `go test -c` and then runs the tests for each package separately. This will provide combined coverage data for unit and integration tests. * gocv: Support test2json Adds a flag to use test2json to run the test binary, enabling compatibility with gotestsum. * gocov: Add -race flag
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We recently started capturing coverage data for Pull Requests again
but we omitted tracking for master.
Part of the reason for this is that the our CI setup uses the same
binaries that were used for the end-to-end tests as part of the release.
And we don't want to upload the coverage-instrumented binaries for a
release.
This is fixable, but instead of making our CI set up even more complex,
this introduces a new workflow to test master periodically.
This workflow will run every 12 hours, running all tests on master,
with acceptance tests on both Windows and macOS,
track coverage for all of this, and post that to codecov.
We'll be able to track code coverage trends on master in Codecov.