Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace provider-ci with Go text templating #450

Merged
merged 7 commits into from
Jun 9, 2023

Conversation

AaronFriel
Copy link
Contributor

@AaronFriel AaronFriel commented Jun 8, 2023

Replace provider-ci with package-ci

This commit deletes the provider CI folder and replaces it with package-ci, applying #446, updating the Makefile to conform to the same interface.

Two shared TypeScript files are moved to native-provider-ci:

  • provider-ci/src/shared-workflows.ts
  • provider-ci/src/action-versions.ts

This is a stacked PR - the last commit is the only commit added in this PR.

@AaronFriel AaronFriel force-pushed the friel/package-ci-rollout branch 3 times, most recently from ea25daa to adaed51 Compare June 8, 2023 01:25
AaronFriel added a commit that referenced this pull request Jun 8, 2023
Builds on #446 and #450 by adding linting.

The Makefile is updated to make for a fast inner dev loop on changing templates.
Using a tool such as `watchexec`, we can run:

```sh
watchexec -i providers -- NAME=databricks make lint-provider
```

When running in a loop like this, `watchexec` should trigger on changes to
templates, rebuild `bin/provider-ci`, and regenerate the output. When changing
config files, the `provider-ci` binary should remain unchanged:

```
$ make bin/provider-ci
make: 'bin/provider-ci' is up to date.
```

The `provider` and `providers` targets now run linting by default.
AaronFriel added a commit that referenced this pull request Jun 8, 2023
Builds on #446 and #450 by adding linting.

The Makefile is updated to make for a fast inner dev loop on changing templates.
Using a tool such as `watchexec`, we can run:

```sh
watchexec -i providers -- NAME=databricks make lint-provider
```

When running in a loop like this, `watchexec` should trigger on changes to
templates, rebuild `bin/provider-ci`, and regenerate the output. When changing
config files, the `provider-ci` binary should remain unchanged:

```
$ make bin/provider-ci
make: 'bin/provider-ci' is up to date.
```

The `provider` and `providers` targets now run linting by default.
AaronFriel added a commit that referenced this pull request Jun 8, 2023
Builds on #446 and #450 by adding linting.

The Makefile is updated to make for a fast inner dev loop on changing templates.
Using a tool such as `watchexec`, we can run:

```sh
watchexec -i providers -- NAME=databricks make lint-provider
```

When running in a loop like this, `watchexec` should trigger on changes to
templates, rebuild `bin/provider-ci`, and regenerate the output. When changing
config files, the `provider-ci` binary should remain unchanged:

```
$ make bin/provider-ci
make: 'bin/provider-ci' is up to date.
```

The `provider` and `providers` targets now run linting by default.
@aq17 aq17 self-requested a review June 9, 2023 00:16
Copy link
Contributor

@aq17 aq17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Seeking to resolve the CI disparity between ecosystem managed repositories and
pulumi/pulumi-std#29, it quickly became apparent that the style of GitHub
Actions - using reusable workflows - in that repository and in pulumi-command
would require a major rewrite of the templating in this repository.

Making large changes to the templating in this repository is more difficult as
it requires modifying the Node scripts and the type-safe "generator" to reverse
engineer CI workflows into a known form.

Directly generating text templates, we skip the intermediate steps and can more
naturally write the intended GitHub Actions YAML.

The Go utility in this repository can be run and a diff with the current
repository scripts generated via:

```
cd ./package-ci
./scripts/test.sh
```

The committed template, "bridged-provider", is intended to be bit-for-bit
identical in output to the current scripts to the extent possible. As a result,
there is no semantic diff with the current output.

The two cases where the output was not identical are in the New Relic provider,
and the diff is inconsequential wrapping behavior in the current Node scripts
that is not present in the Go text template, both appearing in `.goreleaser`
files. In this case, the wrapping has no effect on Goreleaser:

```diff
diff --git a/../provider-ci/providers/newrelic/repo/.goreleaser.prerelease.yml b/./test-output/providers/newrelic/repo/.goreleaser.prerelease.yml
index 439adbf7a..a5b009761 100644
--- a/../provider-ci/providers/newrelic/repo/.goreleaser.prerelease.yml
+++ b/./test-output/providers/newrelic/repo/.goreleaser.prerelease.yml
@@ -29,8 +29,7 @@ builds:
   ignore: []
   ldflags:
   - -X github.com/pulumi/pulumi-newrelic/provider/v5/pkg/version.Version={{.Tag}}
-  - -X
-    github.com/newrelic/terraform-provider-newrelic/v2/main.UserAgentServiceName=pulumi
+  - -X github.com/newrelic/terraform-provider-newrelic/v2/main.UserAgentServiceName=pulumi
   main: ./cmd/pulumi-resource-newrelic/
```
This commit deletes the provider CI folder and replaces it with package-ci,
implementing #446, updating the Makefile to conform to the same interface.

Two shared TypeScript files are moved to native-provider-ci:
* provider-ci/src/shared-workflows.ts
* provider-ci/src/action-versions.ts
AaronFriel added a commit that referenced this pull request Jun 9, 2023
Builds on #446 and #450 by adding linting.

The Makefile is updated to make for a fast inner dev loop on changing templates.
Using a tool such as `watchexec`, we can run:

```sh
watchexec -i providers -- NAME=databricks make lint-provider
```

When running in a loop like this, `watchexec` should trigger on changes to
templates, rebuild `bin/provider-ci`, and regenerate the output. When changing
config files, the `provider-ci` binary should remain unchanged:

```
$ make bin/provider-ci
make: 'bin/provider-ci' is up to date.
```

The `provider` and `providers` targets now run linting by default.
@AaronFriel AaronFriel merged commit 473e766 into master Jun 9, 2023
@AaronFriel AaronFriel deleted the friel/package-ci-rollout branch June 9, 2023 22:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants