custom_resources: parameters are not passed for a custom resource calling ELBv2:DescribeListeners

[vfertig]

Describe the bug

I am unable to retrieve a list of all listeners associated with an Application Load Balancer by ARN using a CDK custom resource. During deployment, CDK generates an error that the assumed role is not authorized to perform the action.

I am able to successfully retrieve the list of listeners using the CLI:
aws elbv2 describe-listeners --load-balancer-arn="arn:aws:elasticloadbalancing:us-east-1:_account_:loadbalancer/app/sandbox-alb/<alb>"
I am using a two-step process which involves a custom resource to pull the current value of an SSM parameter containing the ARN of the load balancer and passing the result as a parameter to DescribeListeners.

Expected Behavior

The AwsCustomResource should successfully retrieve the listeners given the specified load balancer arn. The returned value should be equivalent to the output of the aws elbv2 describe-listeners command shown above.

{
    "Listeners": [
        {
            "ListenerArn": "arn:aws:elasticloadbalancing:us-east-1:<account>:listener/app/sandbox-alb/<listener>",
            "LoadBalancerArn": "arn:aws:elasticloadbalancing:us-east-1:<account>:loadbalancer/app/sandbox-alb/<alb>",
            "Port": 80,
            "Protocol": "HTTP",
            "DefaultActions": [
                {
                    "Type": "fixed-response",
                    "FixedResponseConfig": {
                        "MessageBody": "{\n  \"status\": 501,\n  \"message\": \"Not implemented\"\n}",
                        "StatusCode": "501",
                        "ContentType": "application/json"
                    }
                }
            ]
        }
    ]
}

Current Behavior

The deployment fails:

Deployment failed: Error: The stack named tenant-pool failed to deploy: CREATE_FAILED (The following resource(s) failed to create: [DescribeListenersA7BB02EB]. ): Received response status [FAILED] from custom resource. Message returned: User: arn:aws:sts::_account_:assumed-role/<tenant-pool-role> is not authorized to perform: elasticloadbalancing:DescribeListeners because no identity-based policy allows the elasticloadbalancing:DescribeListeners action (RequestId: <id>)
    at FullCloudFormationDeployment.monitorDeployment (/Applications/IBM/SoftwareDeliveryPlatform/cordova_cli/lib/node_modules/aws-cdk/lib/index.js:443:10232)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Object.deployStack2 [as deployStack] (/Applications/IBM/SoftwareDeliveryPlatform/cordova_cli/lib/node_modules/aws-cdk/lib/index.js:446:153546)
    at async /Applications/IBM/SoftwareDeliveryPlatform/cordova_cli/lib/node_modules/aws-cdk/lib/index.js:446:136809

The stack named tenant-pool failed to deploy: CREATE_FAILED (The following resource(s) failed to create: [DescribeListenersA7BB02EB]. ): Received response status [FAILED] from custom resource. Message returned: User: arn:aws:sts::_account_:assumed-role/_tenant_pool_role_ is not authorized to perform: elasticloadbalancing:DescribeListeners because no identity-based policy allows the elasticloadbalancing:DescribeListeners action (RequestId: _id_)

When I check CloudTrail, the "DescribeListeners" event shows requestParameters as null instead of the load balancer ARN.
custom_resource_event_record.txt
successful_aws_cmd_event_record.txt

Reproduction Steps

Prerequisite:
Store an SSM parameter "alb_arn" : "arn:aws:elasticloadbalancing:us-east-1:account_id:loadbalancer/app/sandbox-alb/"

Create a stack stack1 and retrieve the value of "alb_arn" using a custom resource, attempt to retrieve the listeners using a separate custom resource:

alb_arn_cr = AwsCustomResource.Builder.create(stack, resourceId)
                .onUpdate(AwsSdkCall.builder()
                        .service("SSM")
                        .region(targetRegion)
                        .action("getParameter")
                        .parameters( Map.of("Name", parameterName))
                        .physicalResourceId(PhysicalResourceId.of(new Date().toString()))
                        .build())
        .policy(AwsCustomResourcePolicy.fromSdkCalls(SdkCallsPolicyOptions.builder().resources(Collections.singletonList(ssmArn)).build()))
                .build();

alb_listeners_cr = AwsCustomResource.Builder.create(stack, resourceId)
                .onUpdate(AwsSdkCall.builder()
                        .service("ELBv2")
                        .region(targetRegion)
                        .action("describeListeners")
                        .parameters( Map.of (
                                "LoadBalancerArn", alb_arn_cr.getResponseField("Parameter.Value") ))
                        .physicalResourceId(PhysicalResourceId.of(new Date().toString()))
                        .policy(AwsCustomResourcePolicy.fromSdkCalls(SdkCallsPolicyOptions.builder().resources(Collections.singletonList(permissiveAccessToLb)).build()))
                        .build();

Sample project reproducing the issue
bug-report.zip

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.95.1 (build ae455d8)

Framework Version

No response

Node.js Version

v18.14.2

OS

macOS Ventura version 13.5.2

Language

Java

Language Version

Java (17)

Other information

No response

[peterwoodworth]

I attempted to reproduce this in TypeScript, however the call succeeded. Here is my code:

    const ssmparam = new ssm.StringParameter(this, 'Parameter', { stringValue: 'myloadbalancerarn' });
    const ssmcr = new cr.AwsCustomResource(this, 'SSMCustomResource', { 
      onUpdate: {
        service: 'SSM',
        action: 'getParameter',
        parameters: {
          Name: ssmparam.parameterName
        },
        physicalResourceId: cr.PhysicalResourceId.of(Date.now().toString()),
      },
      policy: cr.AwsCustomResourcePolicy.fromStatements([
        new iam.PolicyStatement({
          actions: ['*'],
          resources: ['*'],
          effect: iam.Effect.ALLOW,
        })
      ]),
    });

    const elbcr = new cr.AwsCustomResource(this, 'ELBCustomResource', { 
      onUpdate: {
        service: 'ELBv2',
        action: 'describeListeners',
        parameters: {
          LoadBalancerArn: ssmcr.getResponseField('Parameter.Value').toString()
        },
        physicalResourceId: cr.PhysicalResourceId.of(Date.now().toString()),
      },
      policy: cr.AwsCustomResourcePolicy.fromStatements([
        new iam.PolicyStatement({
          actions: ['*'],
          resources: ['*'],
          effect: iam.Effect.ALLOW,
        })
      ]),
    });

You mention

When I check CloudTrail, the "DescribeListeners" event shows requestParameters as null instead of the load balancer ARN.

Could you share the synthesized CloudFormation template? You should see something like the following for your LoadBalancerArn

  "ELBCustomResource5A118D7C": {
   "Type": "Custom::AWS",
   "Properties": {
    "ServiceToken": {
     "Fn::GetAtt": [
      "AWS679f53fac002430cb0da5b7982bd22872D164C4C",
      "Arn"
     ]
    },
    "Create": {
     "Fn::Join": [
      "",
      [
       "{\"service\":\"ELBv2\",\"action\":\"describeListeners\",\"parameters\":{\"LoadBalancerArn\":\"",
       {
        "Fn::GetAtt": [
         "SSMCustomResourceFF6A08D8",
         "Parameter.Value"
        ]
       },
       "\"},\"physicalResourceId\":{\"id\":\"1694729238854\"}}"
      ]
     ]
    },

[vfertig]

Hi Peter, Here is what I get in the generated templates (BugReportStack.template.json)
"resourcetwo75B45749": {
"Type": "Custom::AWS",
"Properties": {
"ServiceToken": {
"Fn::GetAtt": [
"AWS679f53fac002430cb0da5b7982bd22872D164C4C",
"Arn"
]
},
"Create": "{"action":"describeListeners","service":"ELBv2","parameters":{"LoadBalancerArn":"arn:aws:elasticloadbalancing:us-east-1::loadbalancer/app/sandbox-alb/6f09204312f7d123"},"physicalResourceId":{"id":"Fri Sep 15 07:44:25 EDT 2023"},"region":"us-east-1"}",
"Update": "{"action":"describeListeners","service":"ELBv2","parameters":{"LoadBalancerArn":"arn:aws:elasticloadbalancing:us-east-1::loadbalancer/app/sandbox-alb/6f09204312f7d123"},"physicalResourceId":{"id":"Fri Sep 15 07:44:25 EDT 2023"},"region":"us-east-1"}",
"InstallLatestAwsSdk": false
},
"DependsOn": [
"resourcetwoCustomResourcePolicy176BB315"
],
"UpdateReplacePolicy": "Delete",
"DeletionPolicy": "Delete",
"Metadata": {
"aws:cdk:path": "BugReportStack/resource two/Resource/Default"
}
},

[vfertig]

I've found the key difference between what you're doing and what I'm doing and it's in the policy statement for the DescribeListeners call.

Yours is:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": "*",
            "Resource": "*",
            "Effect": "Allow"
        }
    ]
}

while mine is:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": "elasticloadbalancingv2:DescribeListeners",
            "Resource": "arn:aws:elasticloadbalancing:us-east-1:<account>:loadbalancer/*",
            "Effect": "Allow"
        }
    ]
}

Once I matched my policy to yours, it worked. The question I have then is: What should the policy have been to allow the parameter to be passed?

[vfertig]

This leads into another issue with my not being able to retrieve the raw response and iterate over it. getResponseField doesn't help unless you know exactly what those response fields are.

I see this is discussed here: #22826

Is there any work on a solution to read the raw response from the Lambda function?

[peterwoodworth]

What should the policy have been to allow the parameter to be passed?

I'm not exactly sure off the top of my head, I'd have to look into that.

This leads into another issue with my not being able to retrieve the raw response and iterate over it. Is there any work on a solution to read the raw response from the Lambda function?

There isn't a way to do that unfortunately. The getResponseField method works by creating a reference in the CloudFormation template to that custom resource's attribute values, like so:

        "Fn::GetAtt": [
         "SSMCustomResourceFF6A08D8",
         "Parameter.Value"
        ]

If you need to loop over an arbitrary list, that requires creating n number of attribute references that look something like this:

        "Fn::GetAtt": [
         "ELBCustomResourceFF6A08D8",
         "Listeners.n.LoadBalancerArn"
        ]

But, we won't know what n is until deployment time when the custom resource runs, which is after the template synthesizes.

My response here addresses what the workaround is. If you'd like more info or a detailed explanation on why this is the recommended solution, please see the "Commit cdk.context.json section on this page.

[rix0rrr]

What should the policy have been to allow the parameter to be passed?

I think it might need to be:

            "Action": "elasticloadbalancing:DescribeListeners",

[github-actions]

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.