custom-resources: AwsCustomResource not using latest value

[alexandervandekleutab]

Describe the bug

I created a custom resource that reads parameters from parameter store in various regions. For some resources it fails to use the latest value.

Specifically in my case, I deleted and re-created a Hosted Zone, exporting the hosted zone name and IDs as parameters to parameter store, which I want to import in other stacks.

Expected Behavior

It should always fetch the latest value from parameter store.

Current Behavior

I can see the associated lambda function running and returning the correct value. However, when the stack deploys, it seems to be trying to use the old value for the hosted zone ID.

Reproduction Steps

I use the following construct:

import { Construct } from '@aws-cdk/core'
import { AwsCustomResource, AwsSdkCall } from '@aws-cdk/custom-resources'
import iam = require('@aws-cdk/aws-iam')

interface SSMParameterReaderProps {
  parameterName: string
  region: string
}

/**
 * https://stackoverflow.com/questions/59774627/cloudformation-cross-region-reference
 */
export class SSMParameterReader extends AwsCustomResource {
  constructor(scope: Construct, name: string, props: SSMParameterReaderProps) {
    const { parameterName, region } = props

    const ssmAwsSdkCall: AwsSdkCall = {
      service: 'SSM',
      action: 'getParameter',
      parameters: {
        Name: parameterName,
      },
      region,
      physicalResourceId: { id: Date.now().toString() }, // Update physical id to always fetch the latest version
    }

    super(scope, name, {
      onUpdate: ssmAwsSdkCall,
      policy: {
        statements: [
          new iam.PolicyStatement({
            resources: ['*'],
            actions: ['ssm:GetParameter'],
            effect: iam.Effect.ALLOW,
          }),
        ],
      },
    })
  }

  public getParameterValue(): string {
    return this.getResponseField('Parameter.Value').toString()
  }
}

Then you can use this construct to import various parameters. Because the physical ID updates each deployment, it always tries to fetch the latest value. The logs show it working correctly:

{
    "Status": "SUCCESS",
    "Reason": "OK",
    "PhysicalResourceId": "1655923819047",
    "StackId": "arn:aws:cloudformation:ca-central-1:<accountId>:<id>",
    "RequestId": "<requestId>",
    "LogicalResourceId": "<logicalId>",
    "NoEcho": false,
    "Data": {
        "apiVersion": null,
        "region": "us-east-1",
        "Parameter.Name": "<prefix>/HostedZoneId",
        "Parameter.Type": "String",
        "Parameter.Value": "<CORRECT_HOSTED_ZONE_ID>",
        "Parameter.Version": 1,
        "Parameter.ARN": "<arn>",
        "Parameter.DataType": "text"
    }
}

The line "Parameter.Value": "<CORRECT_HOSTED_ZONE_ID_HERE>" refers to the correct value, the current hosted zone Id. But when deploying a stack using this construct, I get the error

No hosted zone found with ID: <OLD_INCORRECT_HOSTED_ZONE_ID> (Service: AmazonRoute53; Status Code: 404; Error Code: NoSuchHostedZone; Request ID: <requestId>; Proxy: null)

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

1.124.0

Framework Version

1.144.0

Node.js Version

14.17

OS

macos

Language

Typescript

Language Version

No response

Other information

No response

[alexandervandekleutab]

Note that deleting the stack that fails to use the latest value, and then creating it again, seems to solve the problem.

[rix0rrr]

You need to understand CloudFormation for this -- it will assume that the result of a resource update, any resource update, is idempotent. If the properties of a resource didn't change, it will not perform the update again. So, even if you Lambda would have produced a different output when run again, it will never be run because its input didn't change.

If you can control an input parameter that is not used, and put a value like the current timestamp there, that will invalidate the execution. Or, if you change the logical ID, that will make CFN treat the resource as a different resource which also invalidates it.

[heathers]

It is unclear what you want the requester to do to fix the problem. Here are the aws docs suggesting to set physicalResourceId to Date.now(). What should the requester do differently?

https://docs.aws.amazon.com/cdk/api/v1/docs/custom-resources-readme.html#get-the-latest-version-of-a-secure-ssm-parameter