Allow Node.js dynamic providers to capture secrets #13329
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This allows the function serialization used by dynamic providers to capture secrets, and if so, marks the resulting input value as a secret so that the `__provider` property will be a secret in the state. Fixes #8265.
Changelog[uncommitted] (2023-07-07)Bug Fixes
|
lukehoban
changed the title
6 tasks
|
bors try |
tryBuild failed: |
|
bors merge |
|
Build succeeded! The publicly hosted instance of bors-ng is deprecated and will go away soon. If you want to self-host your own instance, instructions are here. If you want to switch to GitHub's built-in merge queue, visit their help page.
|
github-merge-queue bot
pushed a commit
that referenced
this pull request
Aug 3, 2024
As of #13315 (which shipped in [v3.75.0](https://github.com/pulumi/pulumi/releases/tag/v3.75.0)), `__provider` (the serialized provider string) is _always_ set to a secret in the state. This can lead to poor performance when there are a lot of dynamic resources and the serialized provider does not actually have any secrets. This change does two things: 1. Provides a way to opt-out of always serializing the provider as a secret. 2. Allows Outputs to be captured during serialization of the provider, which wasn't previously possible. ## 1. Opt-out of always serializing as secret A new attribute, `serialize_as_secret_always`, can be set on a subclass of `ResourceProvider` to opt-out of always serializing the provider as a secret. If you know you don't have any secrets being serialized into the provider and want to avoid the encryption overhead, you can set this attribute to `False`. ```python class MyProvider(ResourceProvider): serialize_as_secret_always = False def create(self, props): # Doesn't have any secrets that need encrypting ... ``` ## 2. Allow Outputs to be captured If you currently try to capture an `Output`, it fails when serializing the provider with: ``` TypeError: cannot pickle '_asyncio.Future' object ``` This change allows Outputs to be captured/serialized for dynamic providers, including secret Outputs. This aligns Python dynamic providers with [Node.js](#13329). ```python import pulumi from pulumi.dynamic import CreateResult, Resource, ResourceProvider config = pulumi.Config() password = config.require_secret("password") class SimpleProvider(ResourceProvider): def create(self, props): # Need to use `password.get()` to get the underlying value of the secret from within the serialized code. # This simulates using this as a credential to talk to an external system. return CreateResult("0", { "authenticated": "200" if password.get() == "s3cret" else "401" }) class SimpleResource(Resource): authenticated: pulumi.Output[str] def __init__(self, name): super().__init__(SimpleProvider(), name, { "authenticated": None }) r = SimpleResource("foo") pulumi.export("out", r.authenticated) ``` Note: In the above example, we didn't have to specify `serialize_as_secret_always` since the default behavior is to always serialize the provider as a secret. If we wanted to, we could have specified `serialize_as_secret_always = False` and it still would have serialized the provider as a secret, since it captured `password` which is a secret Output. If `serialize_as_secret_always = False` was specified and no secrets were captured, then the provider would not be serialized as a secret. We plan to recommend this approach to capturing secrets for both Node.js and Python dynamic providers after this change. Fixes #15539
Zaid-Ajaj
pushed a commit
that referenced
this pull request
Aug 9, 2024
As of #13315 (which shipped in [v3.75.0](https://github.com/pulumi/pulumi/releases/tag/v3.75.0)), `__provider` (the serialized provider string) is _always_ set to a secret in the state. This can lead to poor performance when there are a lot of dynamic resources and the serialized provider does not actually have any secrets. This change does two things: 1. Provides a way to opt-out of always serializing the provider as a secret. 2. Allows Outputs to be captured during serialization of the provider, which wasn't previously possible. ## 1. Opt-out of always serializing as secret A new attribute, `serialize_as_secret_always`, can be set on a subclass of `ResourceProvider` to opt-out of always serializing the provider as a secret. If you know you don't have any secrets being serialized into the provider and want to avoid the encryption overhead, you can set this attribute to `False`. ```python class MyProvider(ResourceProvider): serialize_as_secret_always = False def create(self, props): # Doesn't have any secrets that need encrypting ... ``` ## 2. Allow Outputs to be captured If you currently try to capture an `Output`, it fails when serializing the provider with: ``` TypeError: cannot pickle '_asyncio.Future' object ``` This change allows Outputs to be captured/serialized for dynamic providers, including secret Outputs. This aligns Python dynamic providers with [Node.js](#13329). ```python import pulumi from pulumi.dynamic import CreateResult, Resource, ResourceProvider config = pulumi.Config() password = config.require_secret("password") class SimpleProvider(ResourceProvider): def create(self, props): # Need to use `password.get()` to get the underlying value of the secret from within the serialized code. # This simulates using this as a credential to talk to an external system. return CreateResult("0", { "authenticated": "200" if password.get() == "s3cret" else "401" }) class SimpleResource(Resource): authenticated: pulumi.Output[str] def __init__(self, name): super().__init__(SimpleProvider(), name, { "authenticated": None }) r = SimpleResource("foo") pulumi.export("out", r.authenticated) ``` Note: In the above example, we didn't have to specify `serialize_as_secret_always` since the default behavior is to always serialize the provider as a secret. If we wanted to, we could have specified `serialize_as_secret_always = False` and it still would have serialized the provider as a secret, since it captured `password` which is a secret Output. If `serialize_as_secret_always = False` was specified and no secrets were captured, then the provider would not be serialized as a secret. We plan to recommend this approach to capturing secrets for both Node.js and Python dynamic providers after this change. Fixes #15539
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This allows the function serialization used by dynamic providers to capture secrets, and if so, marks the resulting input value as a secret so that the
__providerproperty will be a secret in the state.Fixes #8265.