Component option propagation (.NET SDK) (#2720)

### Proposed changes Epic: #2254 Fixes: #1214 Fixes: #2345 This PR standardizes the option propagation logic for the component resources in the pulumi-kubernetes .NET SDK. The general approach is: 1. In the component resource constructor, compute the child options to be propagated to any children. The child options consist of the component as parent, and with `version` and `pluginDownloadURL` if specified. 2. Compute the invoke options by copying the child options. ### Specification The component resource is responsible for computing sub-options for invokes and for child resource declarations. This table outlines the expected behavior for each [resource option](https://www.pulumi.com/docs/concepts/options/) when presented to a component resource. | | Propagated | Remarks | |---|---|---| | `additionalSecretOutputs` | no | "does not apply to component resources" | | `aliases` | no | Inherited via parent-child relationship. | | `customTimeouts` | no | "does not apply to component resources" | | `deleteBeforeReplace` | no | | | `deletedWith` | no | | | `dependsOn` | no | The children implicitly wait for the dependency. | | `ignoreChanges` | no | Nonsensical to apply directly to children (see [discussion](pulumi/pulumi#8969)). | | `import` | no | | | `parent` | **yes** | The component becomes the parent. | | `protect` | no | Inherited. | | `provider` | no | Combined into providers map, then inherited via parent-child relationship. | | `providers` | no | Inherited. | | `replaceOnChanges` | no | "does not apply to component resources" | | `retainOnDelete` | no | "does not apply to component resources" | | `transformations` | no | Inherited. | | `version` | **yes** | Influences default provider selection logic during invokes.<br/>Should propagate when child resource is from the same provider type. | | `pluginDownloadURL` | **yes** | Influences default provider selection logic during invokes.<br/>Should propagate when child resource is from the same provider type. | ### Testing A new test case is provided ([test case](https://github.com/pulumi/pulumi-kubernetes/blob/db3cd71f3e5780bfe0bf4584d667b10e446761f7/tests/sdk/dotnet/dotnet_test.go#L316), [test program](https://github.com/pulumi/pulumi-kubernetes/blob/db3cd71f3e5780bfe0bf4584d667b10e446761f7/tests/sdk/dotnet/options/Program.cs)) that exercises option propagation across the component resources: - [kubernetes.helm.sh.v3.Chart](https://www.pulumi.com/registry/packages/kubernetes/api-docs/helm/v3/chart/) - [kubernetes.kustomize.Directory](https://www.pulumi.com/registry/packages/kubernetes/api-docs/kustomize/directory/) - [kubernetes.yaml.ConfigGroup](https://www.pulumi.com/registry/packages/kubernetes/api-docs/yaml/configgroup/) - [kubernetes.yaml.ConfigFile](https://www.pulumi.com/registry/packages/kubernetes/api-docs/yaml/configfile/) Upgrade testing must be done manually, with an emphasis on avoiding replacement due to reparenting. ### Upgrade Considerations In previous versions, the `ConfigFile` and `ConfigGroup` component resources inadvertently assigned the wrong parent to the child resource(s). This would happen when the component resource itself had a parent; the child would be assigned that same parent. This also had the effect of disregarding the component resource's provider in favor of the parent's provider. For example, here's a before/after look at the component hierarchy: Before: ``` ├─ pkg:index:MyComponent parent │ ├─ kubernetes:core/v1:ConfigMap cg-options-cg-options-cm-1 │ ├─ kubernetes:yaml:ConfigFile cg-options-testdata/options/configgroup/manifest.yaml │ ├─ kubernetes:core/v1:ConfigMap cg-options-configgroup-cm-1 │ ├─ kubernetes:yaml:ConfigFile cg-options-testdata/options/configgroup/empty.yaml │ └─ kubernetes:yaml:ConfigGroup cg-options ``` After: ``` └─ pkg:index:MyComponent parent └─ kubernetes:yaml:ConfigGroup cg-options ├─ kubernetes:yaml:ConfigFile cg-options-testdata/options/configgroup/manifest.yaml │ └─ kubernetes:core/v1:ConfigMap cg-options-configgroup-cm-1 └─ kubernetes:core/v1:ConfigMap cg-options-cg-options-cm-1 ``` This PR addresses this issue and attempts to heal existing stacks using aliases. This is effective at avoiding a replacement except in the case where the child was created with the wrong provider. In this case, **Pulumi will suggest a replacement of the child resource(s)**, such that they use the correct provider. ### Related issues (optional)
pulumi · Jan 4, 2024 · 8d724be · 8d724be
1 parent afe1f1b
commit 8d724be
Show file tree

Hide file tree

Showing 20 changed files with 1,164 additions and 169 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,4 +1,5 @@
 ## Unreleased
+- Fix option propagation in component resources (.NET SDK) (https://github.com/pulumi/pulumi-kubernetes/pull/2720)
 - Fix option propagation in component resources (NodeJS SDK) (https://github.com/pulumi/pulumi-kubernetes/pull/2713)
 - Fix option propagation in component resources (Go SDK) (https://github.com/pulumi/pulumi-kubernetes/pull/2709)
 

diff --git a/provider/pkg/gen/dotnet-templates/helm/ChartBase.cs b/provider/pkg/gen/dotnet-templates/helm/ChartBase.cs
@@ -139,7 +139,7 @@ protected ChartBase(string releaseName, Union<ChartArgs, LocalChartArgs> args, C
 
                     var yaml = ExecuteCommand("helm", flags);
                     return ParseTemplate(
-                        yaml, cfgBase.Transformations, cfgBase.ResourcePrefix, dependencies, cfgBase.Namespace, options?.Provider);
+                        yaml, cfgBase.Transformations, cfgBase.ResourcePrefix, dependencies, cfgBase.Namespace, options);
                 }
                 catch (Exception e)
                 {
@@ -261,20 +261,20 @@ private void Fetch(string chart, ChartFetchArgsUnwrap opts)
 
         private Output<ImmutableDictionary<string, KubernetesResource>> ParseTemplate(string text,
             List<TransformationAction> transformations, string? resourcePrefix, ImmutableHashSet<Pu.Resource> dependsOn,
-            string? defaultNamespace, Pu.ProviderResource provider)
+            string? defaultNamespace, ComponentResourceOptions? options)
         {
+            var childOpts = GetChildOptions(this, dependsOn.ToArray(), options);
+            var invokeOpts = GetInvokeOptions(childOpts);
             return Yaml.Invokes
-                .YamlDecode(new YamlDecodeArgs { Text = text, DefaultNamespace = defaultNamespace }, new InvokeOptions { Provider = provider })
+                .YamlDecode(new YamlDecodeArgs { Text = text, DefaultNamespace = defaultNamespace }, invokeOpts)
                 .Apply(objs =>
                 {
-                    var args = new ConfigGroupArgs
+                    return Parser.ParseYamlDocument(new ParseArgs
                     {
-                        ResourcePrefix = resourcePrefix,
                         Objs = objs,
-                        Transformations = transformations
-                    };
-                    var opts = new ComponentResourceOptions { Parent = this, DependsOn = dependsOn.ToArray(), Provider = provider };
-                    return Parser.Parse(args, opts);
+                        Transformations = transformations,
+                        ResourcePrefix = resourcePrefix
+                    }, childOpts);
                 });
         }
 

diff --git a/provider/pkg/gen/dotnet-templates/helm/v3/Chart.cs b/provider/pkg/gen/dotnet-templates/helm/v3/Chart.cs
@@ -361,23 +361,23 @@ private Output<ImmutableDictionary<string, KubernetesResource>> ParseTemplate(Un
             }
             jsonOptsString = JsonSerializer.Serialize(jsonOpts, serializeOptions);
 
+            var childOpts = GetChildOptions(this, dependsOn.ToArray(), options);
+            var invokeOpts = GetInvokeOptions(childOpts);
             return Invokes
-                .HelmTemplate(new HelmTemplateArgs { JsonOpts = jsonOptsString }, new InvokeOptions { Provider = options?.Provider })
+                .HelmTemplate(new HelmTemplateArgs { JsonOpts = jsonOptsString }, invokeOpts)
                 .Apply(objs =>
                 {
                     var transformations = cfgBase.Transformations;
                     if (cfgBase.SkipAwait == true)
                     {
                         transformations = transformations.Append(Parser.SkipAwait).ToList();
                     }
-                    var args = new ConfigGroupArgs
+                    return Parser.ParseYamlDocument(new ParseArgs
                     {
-                        ResourcePrefix = cfgBase.ResourcePrefix,
                         Objs = objs,
-                        Transformations = transformations
-                    };
-                    var opts = new ComponentResourceOptions { Parent = this, DependsOn = dependsOn.ToArray(), Provider = options?.Provider };
-                    return Parser.Parse(args, opts);
+                        Transformations = transformations,
+                        ResourcePrefix = cfgBase.ResourcePrefix
+                    }, childOpts);
                 });
         }
 

diff --git a/provider/pkg/gen/dotnet-templates/kustomize/Directory.cs b/provider/pkg/gen/dotnet-templates/kustomize/Directory.cs
@@ -141,27 +141,19 @@ public sealed class Directory : CollectionComponentResource
         public Directory(string name, DirectoryArgs args, ComponentResourceOptions? options = null)
             : base("kubernetes:kustomize:Directory", MakeName(args, name), options)
         {
-            name = GetName(args, name);
-            var objs = Invokes.KustomizeDirectory(new KustomizeDirectoryArgs { Directory = args.Directory }, new InvokeOptions  { Provider = options?.Provider });
-            var configGroupArgs = new ConfigGroupArgs
-            {
-                ResourcePrefix = args.ResourcePrefix,
-                Objs = objs,
-                Transformations = args.Transformations
-            };
-            var opts = ComponentResourceOptions.Merge(options, new ComponentResourceOptions { Parent = this });
-            var resources = Parser.Parse(configGroupArgs, opts);
+            var childOpts = GetChildOptions(this, null, options);
+            var invokeOpts = GetInvokeOptions(childOpts);
+            var objs = Invokes.KustomizeDirectory(new KustomizeDirectoryArgs { Directory = args.Directory }, invokeOpts);
+            var resources = Parser.ParseYamlDocument(new ParseArgs
+                {
+                    Objs = objs,
+                    Transformations = args.Transformations,
+                    ResourcePrefix = args.ResourcePrefix
+                }, childOpts);
             RegisterResources(resources);
         }
         private static string MakeName(DirectoryArgs? args, string name)
             => args?.ResourcePrefix != null ? $"{args.ResourcePrefix}-{name}" : name;
-
-        private static string GetName(DirectoryArgs config, string releaseName)
-        {
-            var prefix = config.ResourcePrefix;
-            return string.IsNullOrEmpty(prefix) ? releaseName : $"{prefix}-{releaseName}";
-        }
-
     }
 
     /// <summary>

diff --git a/provider/pkg/gen/dotnet-templates/yaml/ConfigFile.cs b/provider/pkg/gen/dotnet-templates/yaml/ConfigFile.cs
@@ -122,11 +122,21 @@ public sealed class ConfigFile : CollectionComponentResource
         /// <param name="args">The arguments used to populate this resource's properties</param>
         /// <param name="options">A bag of options that control this resource's behavior</param>
         public ConfigFile(string name, ConfigFileArgs? args = null, ComponentResourceOptions? options = null)
+            : this(name, args, options, options?.Parent)
+        {
+        }
+
+        internal ConfigFile(string name, ConfigFileArgs? args = null, ComponentResourceOptions? options = null, Pulumi.Resource? aliasParent = null)
             : base("kubernetes:yaml:ConfigFile", MakeName(args, name), options)
         {
             name = MakeName(args, name);
-            options ??= new ComponentResourceOptions();
-            options.Parent ??= this;
+            var childOpts = GetChildOptions(this, null, options);
+            // https://github.com/pulumi/pulumi-kubernetes/issues/1214
+            if (aliasParent is not null) {
+                childOpts.ResourceTransformations ??= new List<ResourceTransformation>();
+                childOpts.ResourceTransformations.Add(Aliased(this, aliasParent));
+            }
+            var invokeOpts = GetInvokeOptions(childOpts);
 
 			var transformations = args?.Transformations ?? new List<TransformationAction>();
 			if (args?.SkipAwait == true)
@@ -153,15 +163,15 @@ public ConfigFile(string name, ConfigFileArgs? args = null, ComponentResourceOpt
             }).Apply(text =>
                 Parser.ParseYamlDocument(new ParseArgs
                 {
-                    Objs = Invokes.YamlDecode(new YamlDecodeArgs { Text = text }, new InvokeOptions { Provider = options?.Provider }),
+                    Objs = Invokes.YamlDecode(new YamlDecodeArgs { Text = text }, invokeOpts),
                     Transformations = transformations,
                     ResourcePrefix = args?.ResourcePrefix
-                }, options));
+                }, childOpts));
 
             RegisterResources(resources);
         }
 
-        private static string MakeName(ConfigFileArgs? args, string name)
+        internal static string MakeName(ConfigFileArgs? args, string name)
             => args?.ResourcePrefix != null ? $"{args.ResourcePrefix}-{name}" : name;
     }
 

diff --git a/provider/pkg/gen/dotnet-templates/yaml/ConfigGroup.cs b/provider/pkg/gen/dotnet-templates/yaml/ConfigGroup.cs
@@ -15,6 +15,7 @@
 // *** WARNING: this file was generated by pulumigen. ***
 // *** Do not edit by hand unless you're certain you know what you are doing! ***
 
+using System;
 using System.Collections.Generic;
 using System.Collections.Immutable;
 
@@ -205,9 +206,15 @@ public sealed class ConfigGroup : CollectionComponentResource
         public ConfigGroup(string name, ConfigGroupArgs config, ComponentResourceOptions? options = null)
             : base("kubernetes:yaml:ConfigGroup", name, options)
         {
-            options ??= new ComponentResourceOptions();
-            options.Parent ??= this;
-            RegisterResources(Parser.Parse(config, options));
+            var childOpts = GetChildOptions(this, null, options);
+
+            // https://github.com/pulumi/pulumi-kubernetes/issues/1214
+            if (options?.Parent is not null) {
+                childOpts.ResourceTransformations ??= new List<ResourceTransformation>();
+                childOpts.ResourceTransformations.Add(Aliased(this, options.Parent));
+            }
+
+            RegisterResources(Parser.Parse(config, childOpts, options?.Parent ?? this));
         }
     }
 

diff --git a/provider/pkg/gen/dotnet-templates/yaml/yaml.tmpl b/provider/pkg/gen/dotnet-templates/yaml/yaml.tmpl
@@ -100,11 +100,82 @@ namespace Pulumi.Kubernetes.Yaml
             var id = namespaceName != null ? $"{namespaceName}/{name}" : name;
             return Resources.Apply(r => (CustomResource)r[$"{groupVersionKind}::{id}"]);
         }
+
+        internal static CustomResourceOptions GetChildOptions(Pu.Resource parent, InputList<Pu.Resource>? extraDependsOn, ComponentResourceOptions? options)
+        {
+            // Create resource options based on component resource options.
+            var dependsOn = new InputList<Pu.Resource>();
+            if (extraDependsOn is not null)
+                dependsOn.AddRange(extraDependsOn);
+            return new CustomResourceOptions
+            {
+                Parent = parent,
+                DependsOn = dependsOn,
+                Version = options?.Version,
+                PluginDownloadURL = options?.PluginDownloadURL,
+            };
+        }
+
+        internal static ComponentResourceOptions ConvertChildOptions(CustomResourceOptions options)
+        {
+            var dependsOn = new InputList<Pu.Resource>();
+            if (options is not null)
+                dependsOn.AddRange(options.DependsOn);
+            return new ComponentResourceOptions
+            {
+                Aliases = options.Aliases.ToList(),
+                DependsOn = dependsOn,
+                Parent = options.Parent,
+                ResourceTransformations = options.ResourceTransformations.ToList(),
+                Version = options.Version,
+                PluginDownloadURL = options.PluginDownloadURL,
+            };
+        }
+
+        internal static InvokeOptions GetInvokeOptions(CustomResourceOptions? options)
+        {
+            return new InvokeOptions { 
+                Parent = options?.Parent,
+                Provider = options?.Provider,
+                Version = options?.Version,
+                PluginDownloadURL = options?.PluginDownloadURL,
+            }.WithDefaults();
+        }
+
+        internal static ResourceTransformation Aliased(Pulumi.Resource parent, Pulumi.Resource? oldParent = null) {
+            return new ResourceTransformation((args) => {
+                if (!Object.ReferenceEquals(args.Options?.Parent, parent)) {
+                    return null;
+                }
+
+                var alias = new Alias { 
+                    Parent = oldParent,
+                    Name = args.Resource.GetResourceName(),
+                    Type = args.Resource.GetResourceType(),
+                };
+                if (args.Options is ComponentResourceOptions options1)
+                {
+                    var options = ComponentResourceOptions.Merge(
+                        options1,
+                        new ComponentResourceOptions { Aliases = {alias} });
+                    return new ResourceTransformationResult(args.Args, options);
+                }
+                if (args.Options is CustomResourceOptions options2)
+                {
+                    var options = CustomResourceOptions.Merge(
+                        options2,
+                        new CustomResourceOptions { Aliases = {alias} });
+                    return new ResourceTransformationResult(args.Args, options);
+                }
+                return null;
+            });
+        }
     }
 
     internal static class Parser
     {
-        public static Output<ImmutableDictionary<string, KubernetesResource>> Parse(ConfigGroupArgs config, ComponentResourceOptions? options)
+        public static Output<ImmutableDictionary<string, KubernetesResource>> Parse(ConfigGroupArgs config, CustomResourceOptions options,
+            Pulumi.Resource? aliasParent = null)
         {
             var resources = Output.Create(ImmutableDictionary.Create<string, KubernetesResource>());
 
@@ -127,28 +198,29 @@ namespace Pulumi.Kubernetes.Yaml
 
                 foreach (var file in files)
                 {
-                    var cf = new ConfigFile(
-                        file,
-                        new ConfigFileArgs
-                        {
-                            File = file,
-                            Transformations = transformations,
-                            ResourcePrefix = config.ResourcePrefix
-                        },
-                        options);
+                    var cfOptions = CollectionComponentResource.ConvertChildOptions(options);
+                    var cfArgs = new ConfigFileArgs
+                    {
+                        File = file,
+                        Transformations = transformations,
+                        ResourcePrefix = config.ResourcePrefix
+                    };
+
+                    var cf = new ConfigFile(file, cfArgs, cfOptions, aliasParent);
                     resources = Output.Tuple(resources, cf.Resources).Apply(vs => vs.Item1.AddRange(vs.Item2));
                 }
             }
 
             if (config.Yaml != null)
             {
+                var invokeOpts = CollectionComponentResource.GetInvokeOptions(options);
                 var yamlResources = config.Yaml.ToOutput().Apply(texts =>
                 {
                     var yamls = texts
                         .Select(text =>
                             ParseYamlDocument(new ParseArgs
                             {
-                                Objs = Invokes.YamlDecode(new YamlDecodeArgs { Text = text }, new InvokeOptions { Provider = options?.Provider }),
+                                Objs = Invokes.YamlDecode(new YamlDecodeArgs { Text = text }, invokeOpts),
                                 Transformations = transformations,
                                 ResourcePrefix = config.ResourcePrefix
                             }, options))
@@ -207,7 +279,7 @@ namespace Pulumi.Kubernetes.Yaml
             => s.StartsWith("http://", StringComparison.Ordinal) || s.StartsWith("https://", StringComparison.Ordinal);
 
         internal static Output<ImmutableDictionary<string, KubernetesResource>> ParseYamlDocument(ParseArgs config,
-            ComponentResourceOptions? options = null)
+            CustomResourceOptions? options = null)
         {
             return config.Objs.ToOutput().Apply(objs =>
             {
@@ -225,21 +297,13 @@ namespace Pulumi.Kubernetes.Yaml
         }
 
         private static Output<(string, KubernetesResource)>[] ParseYamlObject(ImmutableDictionary<string, object> obj,
-            List<TransformationAction>? transformations, string? resourcePrefix, ComponentResourceOptions? options = null)
+            List<TransformationAction>? transformations, string? resourcePrefix, CustomResourceOptions? options = null)
         {
             if (obj == null || obj.Count == 0)
                 return new Output<(string, KubernetesResource)>[0];
 
-            // Create custom resource options based on component resource options.
-            var opts = new CustomResourceOptions
-            {
-                Parent = options?.Parent,
-                DependsOn = options?.DependsOn ?? new InputList<Pu.Resource>(),
-                IgnoreChanges = options?.IgnoreChanges ?? new List<string>(),
-                Version = options?.Version,
-                Provider = options?.Provider,
-                CustomTimeouts = options?.CustomTimeouts
-            };
+            // Create a copy of options to pass into potentially mutating transforms that will be applied to this resource.
+            var opts = CustomResourceOptions.Merge(null, options);
 
             // Allow users to change API objects before any validation.
             if (transformations != null)
@@ -281,7 +345,7 @@ namespace Pulumi.Kubernetes.Yaml
                 if (obj["items"] is IEnumerable<ImmutableDictionary<string, object>> items)
                 {
                     foreach (var item in items)
-                        objs.AddRange(Parser.ParseYamlObject(item, transformations, resourcePrefix));
+                        objs.AddRange(Parser.ParseYamlObject(item, transformations, resourcePrefix, opts));
                 }
                 return objs.ToArray();
             }

diff --git a/sdk/dotnet/Helm/ChartBase.cs b/sdk/dotnet/Helm/ChartBase.cs
@@ -139,7 +139,7 @@ protected ChartBase(string releaseName, Union<ChartArgs, LocalChartArgs> args, C
 
                     var yaml = ExecuteCommand("helm", flags);
                     return ParseTemplate(
-                        yaml, cfgBase.Transformations, cfgBase.ResourcePrefix, dependencies, cfgBase.Namespace, options?.Provider);
+                        yaml, cfgBase.Transformations, cfgBase.ResourcePrefix, dependencies, cfgBase.Namespace, options);
                 }
                 catch (Exception e)
                 {
@@ -261,20 +261,20 @@ private void Fetch(string chart, ChartFetchArgsUnwrap opts)
 
         private Output<ImmutableDictionary<string, KubernetesResource>> ParseTemplate(string text,
             List<TransformationAction> transformations, string? resourcePrefix, ImmutableHashSet<Pu.Resource> dependsOn,
-            string? defaultNamespace, Pu.ProviderResource provider)
+            string? defaultNamespace, ComponentResourceOptions? options)
         {
+            var childOpts = GetChildOptions(this, dependsOn.ToArray(), options);
+            var invokeOpts = GetInvokeOptions(childOpts);
             return Yaml.Invokes
-                .YamlDecode(new YamlDecodeArgs { Text = text, DefaultNamespace = defaultNamespace }, new InvokeOptions { Provider = provider })
+                .YamlDecode(new YamlDecodeArgs { Text = text, DefaultNamespace = defaultNamespace }, invokeOpts)
                 .Apply(objs =>
                 {
-                    var args = new ConfigGroupArgs
+                    return Parser.ParseYamlDocument(new ParseArgs
                     {
-                        ResourcePrefix = resourcePrefix,
                         Objs = objs,
-                        Transformations = transformations
-                    };
-                    var opts = new ComponentResourceOptions { Parent = this, DependsOn = dependsOn.ToArray(), Provider = provider };
-                    return Parser.Parse(args, opts);
+                        Transformations = transformations,
+                        ResourcePrefix = resourcePrefix
+                    }, childOpts);
                 });
         }