[python] Pulumi Import Codegen Fails

[phillipedwards]

What happened?

When running a pulumi import ... command, the importing of the resource succeeds, however, the codegen part fails with an obscure error.

error: anonymous.pp:24,22-25,14: cannot assign expression of type (null) to location of type   list(  { onDeregistration: output(string) | string, onUnhealthy: output(string) | string }
  | output({ onDeregistration: string, onUnhealthy: string }))
| output(list({ onDeregistration: string, onUnhealthy: string }))?: ; 

Expected Behavior

Pulumi import succeeds and correctly generates the required code definition.

Steps to reproduce

Create pulumi python program on pulumi and generate the appropriate code definition.

pulumi import ....

See error shown above.

Output of pulumi about

CLI          
Version      3.66.0
Go Version   go1.20.3
Go Compiler  gc

Plugins
NAME    VERSION
aws     5.40.0
python  unknown

Host     
OS       ubuntu
Version  22.04
Arch     x86_64

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[phillipedwards]

Repro for issue:

• Create a vanilla Pulumi python program pulumi new aws-python and pulumi up
• Create an isolated AWS Target Group via the AWS Console; do not attach it to any load balancer (not sure if this impacts the codegen part)
• Run pulumi import aws:lb/targetGroup:TargetGroup {your_name} {target_group_arn}

[Zaid-Ajaj]

I've been able to reproduce the issue locally, looking into it now

[Zaid-Ajaj]

Found out the main root cause of the issue. Pulumi import reads the state from the AWS provider, then it writes out a PCL block of text from that state. Finally it gives the PCL text to one of our code generators to emit the final piece of code from that resource.

The problem here is that AWS provider is reading the state from the cloud incorrectly returning the following:

Resource input state

{
  "urn": "urn:pulumi:stack::project::aws:lb/targetGroup:TargetGroup::test-target-group",
  "custom": true,
  "type": "aws:lb/targetGroup:TargetGroup",
  "inputs": {
    "__defaults": [
      
    ],
    "deregistrationDelay": 300,
    "healthCheck": {
      "healthyThreshold": 5,
      "matcher": "200",
      "path": "/",
      "timeout": 5,
      "unhealthyThreshold": 2
    },
    "ipAddressType": "ipv4",
    "loadBalancingAlgorithmType": "round_robin",
    "loadBalancingCrossZoneEnabled": "use_load_balancer_configuration",
    "name": "test-target-group",
    "port": 80,
    "protocol": "HTTP",
    "protocolVersion": "HTTP1",
    "stickiness": {
      "enabled": false,
      "type": "lb_cookie"
    },
    "targetFailovers": [
      null
    ],
    "vpcId": "<redacted>"
  },
  "parent": "urn:pulumi:dev::pythonimportissue::pulumi:pulumi:Stack::pythonimportissue-dev",
  "protect": true,
  "provider": "<redacted>"
}

Notice here the input for property targetFailovers:

"targetFailovers": [
  null
]

has recorded a single element during Read but that element is null. This does not match with the schema of the property targetFailovers which is

array<{ 
  onDeregistration: string, 
  onUnhealthy: string 
}>

The code generators expect such elements to be non-nullable which is why the PCL binder fails when type-checking the resulting PCL text

Generated PCL text block

resource test-target-group "aws:lb/targetGroup:TargetGroup" {
    deregistrationDelay = 300
    healthCheck ={
        healthyThreshold = 5,
        matcher = "200",
        path = "/",
        timeout = 5,
        unhealthyThreshold = 2
    }
    ipAddressType = "ipv4"
    loadBalancingAlgorithmType = "round_robin"
    loadBalancingCrossZoneEnabled = "use_load_balancer_configuration"
    name = "test-target-group"
    port = 80
    protocol = "HTTP"
    protocolVersion = "HTTP1"
    stickiness ={
        enabled = false,
        type = "lb_cookie"
    }
    targetFailovers =[
        null]
    vpcId = "<redacted>"
    options {
      protect =true
    }
}

Will move the issue to pulumi/pulumi-aws since the root cause is in the implementation of Read in the AWS provider where it should return inputs for the resource state that match up with the schema of that resource.

[danielrbradley]

Looking into the read implementation, the logic looks slightly questionable .. if there's no attributes at all then an empty array is returned. If neither target_failover.on_deregistration or target_failover.on_unhealthy are in the attribute set then an array with a single empty map will be returned. My first thought is that this empty map might be getting converted into the null within the bridge. Need to attach a debugger to follow this the rest of the way.

https://github.com/pulumi/terraform-provider-aws/blob/patched-v4.65.0/internal/service/elbv2/target_group.go#L1117C24-L1134

[t0yv0]

Watching this, I'm available to investigate this as a potential bridge bug.

[t0yv0]

Just some more info to help reproduce. This provisions a TargetGroup.

import pulumi
import pulumi_aws as aws

main = aws.ec2.Vpc("main", cidr_block="10.0.0.0/16")
test = aws.lb.TargetGroup("test",
    port=80,
    protocol="HTTP",
    vpc_id=main.id)

pulumi.export('arn', test.arn)

Then I can reproduce with:

pulumi import aws:lb/targetGroup:TargetGroup foo $(pulumi stack output arn)

error: anonymous.pp:27,22-28,14: cannot assign expression of type (null) to location of type   list(  { onDeregistration: output(string) | string, onUnhealthy: output(string) | string }
  | output({ onDeregistration: string, onUnhealthy: string }))
| output(list({ onDeregistration: string, onUnhealthy: string }))?: ; 

[t0yv0]

Found that this line upstream generates a list with an empty map element:

https://github.com/pulumi/terraform-provider-aws/blob/d058da74480f4f9e2faa385eeca82db6915c98a7/internal/service/elbv2/target_group.go#L1133

There is an issue in the bridge that translates the empty map to null. Repro-as-unit-test is available:

pulumi/pulumi-terraform-bridge#1105

We have a potential fix that changes that but there's a lot of testing diligence to roll that out.

I recommend finding some workarounds in the meanwhile so we can reduce urgency from P1.

Perhaps import code could tolerate it somehow?

Could we patch upstream code to return an empty list?

[danielrbradley]

To resolve the P1, I'll do a patch on the upstream implementation to remove the map if it's got zero elements.