Schema loader made to respect PULUMI_DEBUG_PROVIDERS #15526

t0yv0 · 2024-02-27T22:11:52Z

Description

With this change pulumi-yaml can pick up local provider and attach to it from PULUMI_DEBUG_PROVIDERS for the purposes of schema resolution, which enables using non-existent test-only providers. Before the change it would fail hard trying to download it.

Checklist

I have run make tidy to update any new dependencies
I have run make lint to verify my code passes the lint check
- I have formatted my code using gofumpt

I have added tests that prove my fix is effective or that my feature works

I have run make changelog and committed the changelog/pending/<file> documenting my change

Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version

pulumi-bot · 2024-02-27T22:12:40Z

Changelog

[uncommitted] (2024-03-04)

Features

[pkg] Make schema.NewPluginLoader respect PULUMI_DEBUG_PROVIDERS, which enables Pulumi YAML programs to work correctly with this feature
#15526

pkg/codegen/schema/loader.go

sdk/go/common/resource/plugin/provider_plugin.go

pkg/codegen/schema/loader.go

Frassle · 2024-02-27T22:34:13Z

pkg/codegen/schema/loader.go

+	if err != nil {
+		return nil, nil, err
+	}
+	// If PULUMI_DEBUG_PROVIDERS requested an attach port, skip caching and workspace


t0yv0 · 2024-03-04T20:36:04Z

"Prime Go Cache" job flaked with

##[debug]Dropping file value '/home/go/pkg/mod/github.com/!proton!mail/go-crypto@v0.0.0-20230828082145-3c4c8a2d2371/openpgp/internal/ecc/curve25519.go'. Path does not exist
Error: ../../../../go/pkg/mod/github.com/!proton!mail/go-crypto@v0.0.0-20230828082145-3c4c8a2d2371/openpgp/internal/ecc/curve25519.go:9:2: github.com/cloudflare/circl@v1.3.7: read "https://proxy.golang.org/github.com/cloudflare/circl/@v/v1.3.7.zip": stream error: stream ID 339; INTERNAL_ERROR; received from peer
##[debug]Dropping file value '/home/go/pkg/mod/github.com/!proton!mail/go-crypto@v0.0.0-20230828082145-3c4c8a2d2371/openpgp/internal/ecc/x448.go'. Path does not exist
Error: ../../../../go/pkg/mod/github.com/!proton!mail/go-crypto@v0.0.0-20230828082145-3c4c8a2d2371/openpgp/internal/ecc/x448.go:9:2: github.com/cloudflare/circl@v1.3.7: read "https://proxy.golang.org/github.com/cloudflare/circl/@v/v1.3.7.zip": stream error: stream ID 339; INTERNAL_ERROR; received from peer
##[debug]Dropping file value '/home/go/pkg/mod/github.com/!proton!mail/go-crypto@v0.0.0-20230828082145-3c4c8a2d2371/openpgp/internal/ecc/ed25519.go'. Path does not exist
Error: ../../../../go/pkg/mod/github.com/!proton!mail/go-crypto@v0.0.0-20230828082145-3c4c8a2d2371/openpgp/internal/ecc/ed25519.go:9:2: github.com/cloudflare/circl@v1.3.7: read "https://proxy.golang.org/github.com/cloudflare/circl/@v/v1.3.7.zip": stream error: stream ID 339; INTERNAL_ERROR; received from peer
##[debug]Dropping file value '/home/go/pkg/mod/github.com/!proton!mail/go-crypto@v0.0.0-20230828082145-3c4c8a2d2371/openpgp/internal/ecc/ed448.go'. Path does not exist
Error: ../../../../go/pkg/mod/github.com/!proton!mail/go-crypto@v0.0.0-20230828082145-3c4c8a2d2371/openpgp/internal/ecc/ed448.go:9:2: github.com/cloudflare/circl@v1.3.7: read "https://proxy.golang.org/github.com/cloudflare/circl/@v/v1.3.7.zip": stream error: stream ID 339; INTERNAL_ERROR; received from peer
Error: Process completed with exit code 1.
##[debug]Finishing: Prime Go cache

Frassle · 2024-03-04T20:48:14Z

pkg/codegen/schema/loader.go

+
+		if version == nil {
+			info, _ := provider.GetPluginInfo() // nonfatal error
+			version = info.Version


Use contract.IgnoreError()

Co-authored-by: Thomas Gummerer <t.gummerer@gmail.com>

### Features - [pkg] Make schema.NewPluginLoader respect PULUMI_DEBUG_PROVIDERS, which enables Pulumi YAML programs to work correctly with this feature [#15526](#15526) - [sdk/python] Add support for asynchronous invokes via a new `invoke_async` function [#15602](#15602) - [sdkgen/dotnet] Support for non-overlay components in codegen for pulumi-kubernetes provider [#15490](#15490) ### Bug Fixes - [backend/service] Make decrypt/encrypt network calls retryable to help work around network hiccups [#15600](#15600) - [cli/new] Strip credentials and query strings from template URLs saved to project [#15586](#15586) - [engine] Fix an issue where snapshots could become invalid when doing a targeted up [#15476](#15476) - [sdk/python] Fix determining plugins for old packages in the Python language host [#15576](#15576) - [pkg/testing] Make ProgramTest use a temporary PULUMI_HOME for each test [#15568](#15568) - [sdkgen/dotnet] Codegen fix for resources without constant input properties [#15488](#15488)

Draft changelog: ### Features - [auto/{go,nodejs,python}] Add support for suppress progress and suppress outputs parameters in the Automation API [#15596](#15596) - [pkg] Make schema.NewPluginLoader respect PULUMI_DEBUG_PROVIDERS, which enables Pulumi YAML programs to work correctly with this feature [#15526](#15526) - [sdk/dotnet] Update dotnet language host to 3.60.0 [#15609](#15609) - [sdk/nodejs] Add experimental support to the NodeJS SDK for the new transforms system. [#15532](#15532) - [sdk/python] Add support for asynchronous invokes via a new `invoke_async` function [#15602](#15602) - [sdkgen/dotnet] Support for non-overlay components in codegen for pulumi-kubernetes provider [#15490](#15490) ### Bug Fixes - [cli] Fix a panic when the secrets provider is missing from the deployment snapshot [#15599](#15599) - [backend/service] Make decrypt/encrypt network calls retryable to help work around network hiccups [#15600](#15600) - [cli/new] Strip credentials and query strings from template URLs saved to project [#15586](#15586) - [engine] Fix an issue where snapshots could become invalid when doing a targeted up [#15476](#15476) - [pkg/testing] Make ProgramTest use a temporary PULUMI_HOME for each test [#15568](#15568) - [sdkgen/dotnet] Codegen fix for resources without constant input properties [#15488](#15488) - [sdk/nodejs] Properly capture node:crypto and global.crypto in node 19+ - [sdk/python] Fix determining plugins for old packages in the Python language host [#15576](#15576)

t0yv0 requested a review from a team as a code owner February 27, 2024 22:11

t0yv0 requested review from Frassle and Zaid-Ajaj February 27, 2024 22:11

pulumi-bot mentioned this pull request Feb 27, 2024

Test: Upgrade pulumi/{pkg,sdk} to cbe2157ef5809032bc8710af2ecf8719af6499a3 pulumi/pulumi-azuread#843

Closed

t0yv0 commented Feb 27, 2024

View reviewed changes

pkg/codegen/schema/loader.go Outdated Show resolved Hide resolved

pulumi-bot mentioned this pull request Feb 27, 2024

Upgrade pulumi-terraform-bridge to v3.76.0 pulumi/pulumi-aws#3500

Closed

Frassle reviewed Feb 27, 2024

View reviewed changes

pulumi-bot mentioned this pull request Feb 27, 2024

Test: Upgrade pulumi/{pkg,sdk} to cbe2157ef5809032bc8710af2ecf8719af6499a3 pulumi/pulumi-azure#1767

Closed

t0yv0 mentioned this pull request Feb 28, 2024

Fix attach to enable PULUMI_DEBUG_PROVIDERS pulumi/pulumi-terraform-bridge#1716

Merged

pulumi-bot mentioned this pull request Mar 1, 2024

Test: Upgrade pulumi/{pkg,sdk} to 8672250a59eb02e6d2bd7e93afc780ce950d10d2 pulumi/pulumi-aws#3547

Closed

t0yv0 mentioned this pull request Mar 4, 2024

Allow the engine to infer additional secret properties of resources from their schema when being registered #15395

Open

6 tasks

t0yv0 requested a review from tgummerer March 4, 2024 17:08

t0yv0 force-pushed the t0yv0/schema-loader-respects-debug-providers branch from 8672250 to ce8ffce Compare March 4, 2024 17:09

Frassle approved these changes Mar 4, 2024

View reviewed changes

Support PULUMI_DEBUG_PROVIDERS in schema/loader

ddb43d5

Co-authored-by: Thomas Gummerer <t.gummerer@gmail.com>

t0yv0 force-pushed the t0yv0/schema-loader-respects-debug-providers branch from ce8ffce to ddb43d5 Compare March 4, 2024 21:38

t0yv0 enabled auto-merge March 4, 2024 21:39

t0yv0 added this pull request to the merge queue Mar 4, 2024

Merged via the queue into master with commit ee17f80 Mar 4, 2024
53 of 55 checks passed

t0yv0 deleted the t0yv0/schema-loader-respects-debug-providers branch March 4, 2024 22:55

t0yv0 mentioned this pull request Mar 5, 2024

CLI attempts to download plugin when Attach is set #10572

Open

justinvp mentioned this pull request Mar 6, 2024

Prepare for v3.109.0 release #15610

Merged

justinvp mentioned this pull request Mar 7, 2024

Freeze v3.109.0 #15620

Merged

This was referenced Mar 8, 2024

Update pulumi/pkg and pulumi/sdk dependencies to v3.109.0 pulumi/pulumi-yaml#553

Merged

Table-driven diff tests pulumi/pulumi-terraform-bridge#1282

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Schema loader made to respect PULUMI_DEBUG_PROVIDERS #15526

Schema loader made to respect PULUMI_DEBUG_PROVIDERS #15526

t0yv0 commented Feb 27, 2024 •

edited by Frassle

pulumi-bot commented Feb 27, 2024 •

edited

Frassle Feb 27, 2024

t0yv0 commented Mar 4, 2024

Frassle Mar 4, 2024

Schema loader made to respect PULUMI_DEBUG_PROVIDERS #15526

Schema loader made to respect PULUMI_DEBUG_PROVIDERS #15526

Conversation

t0yv0 commented Feb 27, 2024 • edited by Frassle

Description

Checklist

pulumi-bot commented Feb 27, 2024 • edited

Changelog

[uncommitted] (2024-03-04)

Features

Frassle Feb 27, 2024

Choose a reason for hiding this comment

t0yv0 commented Mar 4, 2024

Frassle Mar 4, 2024

Choose a reason for hiding this comment

t0yv0 commented Feb 27, 2024 •

edited by Frassle

pulumi-bot commented Feb 27, 2024 •

edited