Random Helm Errors

[sam-cogan]

Problem description

I am attempting to use Pulumi to deploy the Nginx Ingress Helm Chart and I am encountering an issue where it seems to somewhat randomly fail on certain parts. Some times it will run absolutely fine, but sometimes it will fail on a particular resource, throwing the object reference not set error below. This seems to be random in what resource it fails on, I have seen it on a service account, role, PSP. It also seems to be random whether the issue happens in the preview, or in the actual create.

• Other:

Errors & Logs

ServiceAccount

Diagnostics:
  pulumi:pulumi:Stack (apps-apps.scdev):
    error: Running program 'C:\repos\pulumi-examples\aks-deployment\apps\bin\Debug\netcoreapp3.1\apps.dll' failed with an unhandled exception:
    System.NullReferenceException: Object reference not set to an instance of an object.
       at System.Collections.Generic.HashSet`1.AddIfNotPresent(T value)
       at System.Collections.Generic.HashSet`1.Add(T item)
       at Pulumi.Resource..ctor(String type, String name, Boolean custom, ResourceArgs args, ResourceOptions options)
       at Pulumi.CustomResource..ctor(String type, String name, ResourceArgs args, CustomResourceOptions options)
       at Pulumi.Kubernetes.KubernetesResource..ctor(String type, String name, DictionaryResourceArgs args, CustomResourceOptions options)
       at Pulumi.Kubernetes.Core.V1.ServiceAccount..ctor(String name, ImmutableDictionary`2 dictionary, CustomResourceOptions options)
       at Pulumi.Kubernetes.Yaml.Parser.<>c__DisplayClass3_0.<ParseYamlObject>b__58(String id)
       at Pulumi.Output`1.<>c__DisplayClass12_0`1.<Apply>b__0(T t)
       at Pulumi.Output`1.ApplyHelperAsync[U](Task`1 dataTask, Func`2 func)
       at Pulumi.Output`1.AllHelperAsync(ImmutableArray`1 inputs)
       at Pulumi.Output`1.ApplyHelperAsync[U](Task`1 dataTask, Func`2 func)
       at Pulumi.Output`1.ApplyHelperAsync[U](Task`1 dataTask, Func`2 func)
       at Pulumi.Output`1.<TupleHelperAsync>g__GetData|20_0[T1,T2,T3,T4,T5,T6,T7,T8,X](Input`1 input)
       at Pulumi.Output`1.TupleHelperAsync[T1,T2,T3,T4,T5,T6,T7,T8](Input`1 item1, Input`1 item2, Input`1 item3, Input`1 item4, Input`1 item5, Input`1 item6, Input`1 item7, Input`1 item8)
       at Pulumi.Output`1.ApplyHelperAsync[U](Task`1 dataTask, Func`2 func)
       at Pulumi.Output`1.ApplyHelperAsync[U](Task`1 dataTask, Func`2 func)
       at Pulumi.Output`1.ApplyHelperAsync[U](Task`1 dataTask, Func`2 func)
       at Pulumi.Output`1.ApplyHelperAsync[U](Task`1 dataTask, Func`2 func)
       at Pulumi.Output`1.Pulumi.IOutput.GetDataAsync()
       at Pulumi.Serialization.Serializer.SerializeAsync(String ctx, Object prop)
       at Pulumi.Deployment.SerializeFilteredPropertiesAsync(String label, IDictionary`2 args, Predicate`1 acceptKey)
       at Pulumi.Deployment.SerializeAllPropertiesAsync(String label, IDictionary`2 args)
       at Pulumi.Deployment.RegisterResourceOutputsAsync(Resource resource, Output`1 outputs)
       at Pulumi.Deployment.Runner.<>c__DisplayClass7_0.<<WhileRunningAsync>g__HandleCompletion|0>d.MoveNext()
    --- End of stack trace from previous location where exception was thrown ---
       at Pulumi.Deployment.Runner.WhileRunningAsync()

PSP:

Diagnostics:
  pulumi:pulumi:Stack (apps-apps.scdev):
    error: Running program 'C:\repos\pulumi-examples\aks-deployment\apps\bin\Debug\netcoreapp3.1\apps.dll' failed with an unhandled exception:
    System.NullReferenceException: Object reference not set to an instance of an object.
       at System.Collections.Generic.HashSet`1.AddIfNotPresent(T value)
       at Pulumi.Resource..ctor(String type, String name, Boolean custom, ResourceArgs args, ResourceOptions options)
       at Pulumi.CustomResource..ctor(String type, String name, ResourceArgs args, CustomResourceOptions options)
       at Pulumi.Kubernetes.KubernetesResource..ctor(String type, String name, DictionaryResourceArgs args, CustomResourceOptions options)
       at Pulumi.Kubernetes.Policy.V1Beta1.PodSecurityPolicy..ctor(String name, ImmutableDictionary`2 dictionary, CustomResourceOptions options)
       at Pulumi.Kubernetes.Yaml.Parser.<>c__DisplayClass3_0.<ParseYamlObject>b__79(String id)
       at Pulumi.Output`1.<>c__DisplayClass12_0`1.<Apply>b__0(T t)
       at Pulumi.Output`1.ApplyHelperAsync[U](Task`1 dataTask, Func`2 func)
       at Pulumi.Output`1.AllHelperAsync(ImmutableArray`1 inputs)
       at Pulumi.Output`1.ApplyHelperAsync[U](Task`1 dataTask, Func`2 func)
       at Pulumi.Output`1.ApplyHelperAsync[U](Task`1 dataTask, Func`2 func)
       at Pulumi.Output`1.<TupleHelperAsync>g__GetData|20_0[T1,T2,T3,T4,T5,T6,T7,T8,X](Input`1 input)
       at Pulumi.Output`1.TupleHelperAsync[T1,T2,T3,T4,T5,T6,T7,T8](Input`1 item1, Input`1 item2, Input`1 item3, Input`1 item4, Input`1 item5, Input`1 item6, Input`1 item7, Input`1 item8)
       at Pulumi.Output`1.ApplyHelperAsync[U](Task`1 dataTask, Func`2 func)
       at Pulumi.Output`1.ApplyHelperAsync[U](Task`1 dataTask, Func`2 func)
       at Pulumi.Output`1.ApplyHelperAsync[U](Task`1 dataTask, Func`2 func)
       at Pulumi.Output`1.ApplyHelperAsync[U](Task`1 dataTask, Func`2 func)
       at Pulumi.Output`1.Pulumi.IOutput.GetDataAsync()
       at Pulumi.Serialization.Serializer.SerializeAsync(String ctx, Object prop)
       at Pulumi.Deployment.SerializeFilteredPropertiesAsync(String label, IDictionary`2 args, Predicate`1 acceptKey)
       at Pulumi.Deployment.SerializeAllPropertiesAsync(String label, IDictionary`2 args)
       at Pulumi.Deployment.RegisterResourceOutputsAsync(Resource resource, Output`1 outputs)
       at Pulumi.Deployment.Runner.<>c__DisplayClass7_0.<<WhileRunningAsync>g__HandleCompletion|0>d.MoveNext()
    --- End of stack trace from previous location where exception was thrown ---
       at Pulumi.Deployment.Runner.WhileRunningAsync()

Affected product version(s)

Pulumi 2.13.2

Reproducing the issue

Attempt to deploy the Nginx Ingress Helm Chart. Example configuration:

            _ = new Pulumi.Kubernetes.Helm.V3.Chart(name + $"ingress",
                new ChartArgs()
                {
                    Namespace = ingressNameSpace.Metadata.Apply(x => x.Name),
                    Chart = "ingress-nginx",
                    Version = args.ChartVersion,
                    Repo = args.ChartContainerRegistryName,
                    Values = ingressValues ,
                    Transformations ={
                        SetLbIp
                    }
                }, opts);

[mikhailshilkov]

This looks like a dupe of #1311 and pulumi/pulumi#4821. The fix is already merged and should show up in the next release of Pulumi.Kubernetes. You could also try the latest dev build to check that it helps https://www.nuget.org/packages/Pulumi.Kubernetes/2.7.0-alpha.1605049460

[joeduffy]

@mikhailshilkov @lblackstone It looks like that was fixed ~a month ago? Any reason we haven't shipped a new Kubernetes release with the fix?

[sam-cogan]

Thanks @mikhailshilkov, looks like I had even commented on that earlier issue! Using the dev build seems to resolve the issue, so be good to get that released.

[lblackstone]

@mikhailshilkov @lblackstone It looks like that was fixed ~a month ago? Any reason we haven't shipped a new Kubernetes release with the fix?

CI was broken due to preview changes, so I've been fixing that up over the past few days. I'll cut a new release today or tomorrow.

[mikhailshilkov]

@joeduffy Yeah, I did not communicate the fix properly, so I believe @lblackstone didn't have it on the radar. I will open a PR to the changelog, at least. Let's try to release it soon.

[joeduffy]

Thanks @mikhailshilkov!

[lblackstone]

https://github.com/pulumi/pulumi-kubernetes/releases/tag/v2.7.0 is out now