(custom-resource): node18 doesn't import sdk v2, and framework depends on sdk v2. #26732
Comments
ahammond
added
bug
github-actions
bot
added
the
@aws-cdk/custom-resources
evgenyka
added
p1
needs-review
and removed
needs-triage
|
It does not look like I can install your repro: |
|
Can you clarify a little on the behavior you're seeing?
|
|
@rix0rrr Hmm... the repo is Public. Do you have something like In your This happens for both fresh deploys and upgrades. With upgrades it can be particularly devastating since rollbacks to node12 stuff appear to fail. I think it's either the custom resource lambda or the provider lambda, but I don't know which. Event history (from my sandbox account). |
|
So I think issue is Any custom-resource handler that is vended in aws-cdk-lib that is deployed to Node18 should be using aws-sdk v3. The provider framework runtime included. Looking through @time-loop/cdk-aurora, I'm thinking there may be a regression with aws-lambda-nodejs. Since the default runtime version if not passed by the user is now Node18, if |
|
or it looks like you're explicitly externalizing @rix0rrr should we just revert |
|
Oh! Thank you so much! I couldn’t see that for looking at it!!!
On the topic of making it a feature flag, that would have made a lot of
sense. Horse might have already left the barn there, though.
On Mon, Aug 14, 2023 at 4:47 PM Mitchell Valine ***@***.***> wrote:
or it looks like you're explicitly externalizing aws-sdk here
https://github.com/time-loop/cdk-aurora/blob/main/src/aurora.ts#L416
@rix0rrr <https://github.com/rix0rrr> should we just revert lambda-nodejs's
default runtime to Node16? I think that's the only way to not break with
certain bundling configs like above. We can feature-flag it so newly
created lambda-nodejs functions default to Node18. I guess this is what I'm
leaning towards atm.
—
Reply to this email directly, view it on GitHub
<#26732 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AADM2RFGJIGLPMPPUXXVDR3XVK2J5ANCNFSM6AAAAAA3NPPNJA>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
--
MiniLockID: uX4VrN5FvyFxFCxgTksGxJqvKa16iBhqseYxxA1UkZVJw
GPG: 773A 6BDD 71CE 0AB8 0F5A 1176 8679 A114 FB1A 69BD
|
Maybe, but that feels like it's going to stifle forward progress. We can specifically detect this configuration and do if (!props.runtime && (props?.externalModules ?? []).includes('aws-sdk')) {
Annotations.of(this).addWarning('If you are relying on AWS SDK v2 to be present in the Lambda environment already, please explicitly configure a NodeJS runtime of Node 16 or lower.');
}But yeah, I guess this is technically breaking, so we'll have to. Doesn't |
Previously we changed the default version of the lambda-nodejs Function construct to go from using the `builtInNodeJsCustomResourceRuntime`, a map of regions to available versions, to `lambda.Runtime.NODEJS_18_X`. The default `externalModule` configuration excluded the aws-sdk version based on the runtime passed, excluding v2 for Node16 and under, and v3 for Node18 and up, but users can pass their own bundling configuration excluding `aws-sdk` while not explicitly passing a runtime, which caused their functions to break. Adds a new `lambda.Runtime` value for `NODEJS_LATEST`. This is central reference for the latest version of NodeJS provided by the lamdba service. It also includes a new property `isLatest` which can be used to indicate that the runtime version may change over time. This can used to indicate that relying on packages shipped with the environment may not be relied upon if the version changes. We default to using the `NODEJS_LATEST` runtime only if the feature flag is enabled. If the flag is not enabled, use `NODEJS_16_X` to keep supporting users current bundling configurations. Additionally, add a warning to tell users if they are excluding a package from their bundling that we know doesn't exist within the runtime they are using. IE, if using `NODEJS_18_X` and the exclude list includes `aws-sdk`, warn users that it won't be present. fixes: #26732
Previously we changed the default version of the lambda-nodejs Function construct to go from using the `builtInNodeJsCustomResourceRuntime`, a map of regions to available versions, to `lambda.Runtime.NODEJS_18_X`. The default `externalModule` configuration excluded the aws-sdk version based on the runtime passed, excluding v2 for Node16 and under, and v3 for Node18 and up, but users can pass their own bundling configuration excluding `aws-sdk` while not explicitly passing a runtime, which caused their functions to break. Adds a new `lambda.Runtime` value for `NODEJS_LATEST`. This is central reference for the latest version of NodeJS provided by the lamdba service. It also includes a new property `isLatest` which can be used to indicate that the runtime version may change over time. This can used to indicate that relying on packages shipped with the environment may not be relied upon if the version changes. We default to using the `NODEJS_LATEST` runtime only if the feature flag is enabled. If the flag is not enabled, use `NODEJS_16_X` to keep supporting users current bundling configurations. Additionally, add a warning to tell users if they are excluding a package from their bundling that we know doesn't exist within the runtime they are using. IE, if using `NODEJS_18_X` and the exclude list includes `aws-sdk`, warn users that it won't be present. Fixes #26732 ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
|
Describe the bug
Upgraded to v2.91.0, issue persists (I did some yarn.lock surgery to make sure only v2.91.0 was present). Downgraded to v2.86.0 and managed to unblock by deploying from my laptop.
I assume it's related to #26325
Expected Behavior
I expected to not have to ever think about the custom resource provider framework. Instead... they break when upgraded to node18 because of bundling issues? Why not upgraded them to use sdk v3 before changing them to node18?!?
Current Behavior
Breaks huge swaths of custom resources. Super awesome when combined with older stacks which will fail to roll back because they were previously on node12. I can't emphasize enough how much CustomResources are both essential and yet also horrible when they have issues. Thankfully the only stack I've fully trashed with this was a test system, but wow. Can CDK / Cfn be trusted to deploy resources which store data?
Reproduction Steps
Working: https://github.com/ahammond/repro-node18-bundling-cdk
switch branch to https://github.com/ahammond/repro-node18-bundling-cdk/tree/with-cdk-latest
and it fails.
Possible Solution
No response
Additional Information/Context
No response
CDK CLI Version
2.87.0 breaks things
Framework Version
No response
Node.js Version
v16.20.
OS
MacOS
Language
Typescript
Language Version
5.1.3
Other information
No response
The text was updated successfully, but these errors were encountered: