Lint and clean up templates, update Makefile for linting #451
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
AaronFriel
force-pushed
the
friel/lint-workflows
branch
from
29ec4c5
to
873b8b5
Compare
guineveresaenger
approved these changes
Jun 9, 2023
| @@ -33,18 +33,15 @@ The following tools are required for generating and deploying GitHub Actions wor | |||
| After checking out the code, run the following command: | |||
|
|
|||
| ```bash | |||
| cd provider-ci && make | |||
| cd provider-ci && make clean && make -j | |||
There was a problem hiding this comment.
Is it strictly necessary to use the -j flag?
There was a problem hiding this comment.
Not necessarily, but it will run a bit faster and I think folks will prefer that :)
| ACTIONLINT_VERSION := 1.6.24 | ||
| ACTIONLINT := bin/actionlint-$(ACTIONLINT_VERSION) | ||
|
|
||
| all: ensure test format providers providers lint-providers |
There was a problem hiding this comment.
I think I would prefer running clean on every all recipe so I'm curious about the tradeoffs here!
There was a problem hiding this comment.
I had to ask GPT (and confirmed via stackoverflow, locally) about this after clean in all did some unexpected things.
Because clean is a PHONY target, it always runs. But it will delete files like bin/provider-ci or other outputs.
However make will only check which dependencies need to build when you initially run it, so clean will delete bin/provider-ci, and then not build it again.
The suggested approach is to instead do make clean && make, because the second make is given a clean directory to determine how up to date every rule is.
| ``` | ||
|
|
||
| Common commands: | ||
|
|
||
| - `make gen`: Generate all code | ||
| - `make providers`: Generate code for all providers | ||
| - `make`: Generate all code and check the output. |
There was a problem hiding this comment.
should this say to run make clean first?
There was a problem hiding this comment.
It should probably be part of the list.
Seeking to resolve the CI disparity between ecosystem managed repositories and pulumi/pulumi-std#29, it quickly became apparent that the style of GitHub Actions - using reusable workflows - in that repository and in pulumi-command would require a major rewrite of the templating in this repository. Making large changes to the templating in this repository is more difficult as it requires modifying the Node scripts and the type-safe "generator" to reverse engineer CI workflows into a known form. Directly generating text templates, we skip the intermediate steps and can more naturally write the intended GitHub Actions YAML. The Go utility in this repository can be run and a diff with the current repository scripts generated via: ``` cd ./package-ci ./scripts/test.sh ``` The committed template, "bridged-provider", is intended to be bit-for-bit identical in output to the current scripts to the extent possible. As a result, there is no semantic diff with the current output. The two cases where the output was not identical are in the New Relic provider, and the diff is inconsequential wrapping behavior in the current Node scripts that is not present in the Go text template, both appearing in `.goreleaser` files. In this case, the wrapping has no effect on Goreleaser: ```diff diff --git a/../provider-ci/providers/newrelic/repo/.goreleaser.prerelease.yml b/./test-output/providers/newrelic/repo/.goreleaser.prerelease.yml index 439adbf7a..a5b009761 100644 --- a/../provider-ci/providers/newrelic/repo/.goreleaser.prerelease.yml +++ b/./test-output/providers/newrelic/repo/.goreleaser.prerelease.yml @@ -29,8 +29,7 @@ builds: ignore: [] ldflags: - -X github.com/pulumi/pulumi-newrelic/provider/v5/pkg/version.Version={{.Tag}} - - -X - github.com/newrelic/terraform-provider-newrelic/v2/main.UserAgentServiceName=pulumi + - -X github.com/newrelic/terraform-provider-newrelic/v2/main.UserAgentServiceName=pulumi main: ./cmd/pulumi-resource-newrelic/ ```
This commit deletes the provider CI folder and replaces it with package-ci, implementing #446, updating the Makefile to conform to the same interface. Two shared TypeScript files are moved to native-provider-ci: * provider-ci/src/shared-workflows.ts * provider-ci/src/action-versions.ts
Builds on #446 and #450 by adding linting. The Makefile is updated to make for a fast inner dev loop on changing templates. Using a tool such as `watchexec`, we can run: ```sh watchexec -i providers -- NAME=databricks make lint-provider ``` When running in a loop like this, `watchexec` should trigger on changes to templates, rebuild `bin/provider-ci`, and regenerate the output. When changing config files, the `provider-ci` binary should remain unchanged: ``` $ make bin/provider-ci make: 'bin/provider-ci' is up to date. ``` The `provider` and `providers` targets now run linting by default.
AaronFriel
force-pushed
the
friel/lint-workflows
branch
from
873b8b5
to
d98d4a5
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Lint generated output with actionlint and shellcheck
Builds on #446 and #450 by adding linting.
The Makefile is updated to make for a fast inner dev loop on changing templates. Using a tool such as
watchexec, we can run:When running in a loop like this,
watchexecshould trigger on changes to templates, rebuildbin/provider-ci, and regenerate the output. When changing config files, theprovider-cibinary should remain unchanged: