[sdkgen/python] Fix error calling _configure when the value is None #14014
Conversation
This test has a Provider with an optional `certmanager: ProviderCertmanagerArgs` argument, which itself has two required properties: `mtls_cert_pem` and `mtls_key_pem`. When creating a new `Provider` without specifying a `certmanager`, we get an error: ``` TypeError: ProviderCertmanagerArgs._configure() missing 2 required positional arguments: 'mtls_cert_pem' and 'mtls_key_pem' ```
In order to set default values on passed-in dictionaries for object args, we check to see if the value isn't an instance of the args class, and then proceed with treating it as a dictionary. However, if the value is optional, it will be `None` and therefore not an instance of the args class, so we proceed with calling configure on it. If the object itself has any required args, this will then fail, because we'd be passing an empty dict to `_configure`. The fix is to check both the value is not None and it is not an instance of the args class, before treating it as a dict.
Changelog[uncommitted] (2023-09-22)Bug Fixes
|
| fmt.Fprintf(w, " if not isinstance(%s, %s):\n", InitParamName(prop.Name), expectedObjType) | ||
| fmt.Fprintf(w, " if %[1]s is not None and not isinstance(%[1]s, %s):\n", | ||
| InitParamName(prop.Name), expectedObjType) |
There was a problem hiding this comment.
I understand the original implementation of this was if isinstance(%s, dict), which would have avoided this error, but the code review feedback was to flip the check around to check for the args class instead, to be more permissive about the types of dictionaries that could be supported. #13825 (comment).
I still wonder if we should switch this back to checking for a dictionary, but rather than checking for dict specifically, check if it is an instance of Mapping (if isinstance(%s, Mapping)), which would be permissive in the types of dictionaries it accepts.
I've chosen not to do that for this fix, but wanted to mention it.
| _setter: Callable[[Any, Any], None], | ||
| mtls_cert_pem: pulumi.Input[str], | ||
| mtls_key_pem: pulumi.Input[str], | ||
| opts: Optional[pulumi.ResourceOptions]=None): |
There was a problem hiding this comment.
@dixler, I kind of get why ProviderArgs._configure has an opts parameter, but I don't understand why any other type like this needs it. Seems like it's not necessary here.
There was a problem hiding this comment.
if it's being supplied via **kwargs, we need to catch it iirc.
There was a problem hiding this comment.
if it's being supplied via **kwargs, we need to catch it iirc.
I get it for top-level resource args.
But this is a nested object. I'm not seeing how opts would get passed to that.
|
Change looks good to me. Did #13825 add a test that specifically target its new behavior, or was it leaning on existing tests? Do we have tests that validate this at runtime? |
#13825 did add a basic runtime test, but it didn't cover the case where the nested object is optional but has required fields. This PR includes an additional runtime regression test for this case (https://github.com/pulumi/pulumi/pull/14014/files#diff-cbd72a99dacdd14d7f9d0c055f2bf7a06126c2fcf710aa11d9246f41fee4ec02). The postmortem discussion should touch on ensuring we're covering all cases with runtime tests. We may still want to add additional runtime tests as a follow-up from that. |
Reopens #12546 This removes the `_configure()` ResourceArgs helper method as it has caused a number of issues (linked below). `_configure()` was added in order to support initializing default values sdk side for python. This has led to the following PRs being merged to address unexpected bugs. - #14321 - #14318 - #14281 - #14235 - #14014 - #13825
Fixes #14418 Reopens #12546 This removes the `_configure()` ResourceArgs helper method as it has caused a number of issues (linked below). `_configure()` was added in order to support initializing default values sdk side for python. This has led to the following PRs being merged to address unexpected bugs. - #14321 - #14318 - #14281 - #14235 - #14014 - #13825
<!--- Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation. --> # Description <!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. --> Fixes #14418 Reopens #12546 This removes the `_configure()` ResourceArgs helper method as it has caused a number of issues (linked below). `_configure()` was added in order to support initializing default values sdk side for python. This has led to the following PRs being merged to address unexpected bugs. ## Overview of fixes: It will be quite hard to demonstrate without examples, so I will give an example of the code and describe its short comings: --- - #14418 ```python args = MyArgs() # fails as required argument foo not provided args.foo = "Hello, World!" ``` --- - #14235 Supporting `imageName` and `image_name` ```python def _configure(... image_name, # required ...): ... # This should not error, but errors. _configure(imageName="debian") ``` --- - #14281 ```python def _configure(... image_name, # required ... **kwargs): ... # This should not fail, but fails as `image_name` is not provided _configure(imageName="debian") ``` --- - #14014 ```python class Thing: def __init__(self, ... # Optional certmanager=None, ...): ... Thing._configure( ... certmanager=None, ...) ... def _configure(...): ... # This block runs when certmanager = None, but should not. if not isinstance(certmanager, ProviderCertmanagerArgs): certmanager = certmanager or {} def _setter(key, value): certmanager[key] = value ProviderCertmanagerArgs._configure(_setter, **certmanager) ... Provider() ``` --- - #14321 ```python registry_info=accessToken.apply(get_registry_info) # Build and publish the image. image = Image( 'my-image', build=DockerBuildArgs( context='app', ), image_name=image_name, # Note that this is an Output. registry=registry_info, ) # registry is not None and it is not an instance of RegistryArgs, so we fall into the if, thinking it is a dict, but it is an Output. if registry is not None and not isinstance(registry, RegistryArgs): registry = registry or {} def _setter(key, value): registry[key] = value RegistryArgs._configure(_setter, **registry) __props__.__dict__["registry"] = registry ``` --- - #14318 ```python # foo.core.v1.PodArgs may be an external type and may not be upgraded to have _configure() and will fail, but should not. if pod is not None and not isinstance(pod, foo.core.v1.PodArgs): pod = pod or {} def _setter(key, value): pod[key] = value pulumi_kubernetes.core.v1.PodArgs._configure(_setter, **pod) ``` ## Checklist - [x] I have run `make tidy` to update any new dependencies - [x] I have run `make lint` to verify my code passes the lint check - [x] I have formatted my code using `gofumpt` <!--- Please provide details if the checkbox below is to be left unchecked. --> - [ ] I have added tests that prove my fix is effective or that my feature works <!--- User-facing changes require a CHANGELOG entry. --> - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change <!-- If the change(s) in this PR is a modification of an existing call to the Pulumi Cloud, then the service should honor older versions of the CLI where this change would not exist. You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add it to the service. --> - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version <!-- @pulumi employees: If yes, you must submit corresponding changes in the service repo. -->
We recently fixed an issue where defaults weren't set for nested objects when the nested objects are passed as dicts (#13825). Unfortunately, this introduced a regression when the nested object is optional, but it itself has required fields, and the nested object is not specified. In that case, an unintended error is raised.
Consider a
Providerresource with an optionalcertmanager: ProviderCertmanagerArgsargument, which itself has two required properties:mtls_cert_pemandmtls_key_pem.When creating a new
Providerwithout specifying acertmanager, we get an error:The source of the problem is this check in the generated
Provider's constructor:When
certmanageris not specified, its value isNone, which is also not an instance ofProviderCertmanagerArgs. So the code inside theifexecutes.ProviderCertmanagerArgs._configureis called on an empty dict, and the error is raised because there are two required positional arguments toProviderCertmanagerArgs._configure.The fix is to add an additional check to ensure the value is not
None.Fixes #14012