SSM: Option to disable context caching in ssm.StringParameter.valueFromLookup

[mixtah]

Describe the feature

ssm.StringParameter.valueFromLookup reads a parameter from the parameter store during synthesis, producing a raw value input into a stack/construct as opposed to a deploy-time link. It currently also stores this value in the context cache so that it doesn't need to be resolved in the future.

This feature request is the modification of the caching strategy, adding an optional argument to ssm.StringParameter.valueFromLookup that allow disabling of this caching strategy so that it is forced to be evaluated on each synthesis.

Use Case

While for most stacks, deploy time evaluation of Parameters are suitable, or having these values are cached, I've come across a use case that in which a Parameter must be retrieved during Synthesis and must be up-to-date each time.

While building DockerImageAssets, this occurs after Synthesis and before Deployment and Docker Images may have build
arguments which cannot accept values that are resolved during deployment. External dependencies on libraries or configurations may be sored in an SSM store and may change independently of a stacks deployment, as such, the cached value may be outdated and lead to deploy issues/bugs.

Proposed Solution

Add an optional extra argument to ssm.StringParameter.valueFromLookup to disable using the cache for the value and force it to be evaluated each time. This value should default to having the cache enabled and be optional for backward compatibility.

Other Information

My current workaround is to run cdk context --reset <context-value-id> before each deploy. It is however annoying to do so and adding this feature would be a nice bit of quality of life functionality.

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

CDK version used

2.75.1

Environment details (OS name and version, etc.)

Typescript

[pahud]

Makes perfect sense to me. Thank you for your detailed use case and we look forward to your pull request.

[peterwoodworth]

I'm hesitant on this one - CDK apps should be deterministic, this directly goes against our best practices, so offering this to users could result in enabling this when it shouldn't be enabled.

If you're confident in this design being the best for your CDK app, you could pretty easily work around this - all it would take is an SSM call to fetch the data you need instead of using our provider framework.

[mixtah]

@peterwoodworth That does make sense, especially for the general case. I would say putting this within a construct is bad practice, although directly in a stack it's a bit more understandable.

My specific case may be a little more deterministic however, I have a stack that builds a lambda layer full of shared code. At some point I've hit the lambda function size limit and now using Lambda Images for most of my functions. These don't out of the box support lambda layers, so it is downloaded and built into the docker image, this occurs post-synthesis and before build. I soon faced issues with Docker Image rebuilding, when I update the layer, the docker image will cache the old lambda layer during build. The solution I found is to query the lambda layer version, stored as a SSM Parameter and use this as a docker cache buster.

It's all a bit of a workaround to enable lambda layers in docker image functions and to implement change detection to only build what is necessary in the docker image and not the entire thing each time. I doubt it's exactly best practice, but on the project level it's still a bit more of a deterministic case. What are your thoughts?

[peterwoodworth]

I think this sounds like a safe approach for your use case - if the value being fetched is as controlled as this sounds to be, it should be fine to proceed this way. @pahud do you agree, or know of an alternate solution to this problem?

As for the context this issue was opened in, I would still say that using the context provider for this probably isn't the way to go, since the purpose of it is to be a context provider and not an async operation. But we can keep this FR open for discussion about this 🙂

[emcfins]

We are looking for exactly this solution. We have 2 types of stacks that are launched by our team members in their own accounts - one is a bootstrap stack that sets up some resources and another stack type uses the resource values that are stored in parameter store.

At times when the bootstrap stack needs to be updated or be recreated, the previous values are still cached and the incorrect values are used by the stack type using the values. Having the ability to force lookup for ssm.StringParameter.valueFromLookup would be really helpful for our use case.

[adambiggs]

For what it's worth, I think we have another valid use case for this.

We have multiple stacks in a monorepo that are deployed together, with parameter store values used in place of cross-stack references to decouple the stacks and make working with CloudFormation a little less painful. But the parameters and their values are all defined deterministically in CDK within the same monorepo.

[yaircarel]

My deployment pipeline uses SSM parameters to indicate what version of the container should be deployed in each environment. When we update the SSM parameter we have to remember to tell CDK to reset the cached value to get the new version to deploy. This leads to many issues where if someone forgets to clear their cache the wrong version is deployed which leads to lots of concerns in the environment dependebility. For us the caching creates the opposite effect of a deterministic deploy. In case SSM is not available we would rather have the deploy fail that having issues most of the time. It would be really helpful to be able to tell CDK not to cache certain values. I don't want to build my own solution just to work around CDK design choices. I would rather have the flexibility in the tool to meet my needs.