fix(integ-tests-alpha): optimize assertions handler size #26360
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
bug
aws-cdk-automation
added
the
pr/needs-community-review
mrgrain
changed the title
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
github-actions
bot
added
effort/medium
mergify bot
pushed a commit
that referenced
this pull request
Jul 18, 2023
## Summary Assertions handler is too large and not working. This PR extracts the SDK v2 to v3 features into a separate package. Reduces bundled size from `25.8mb` to `1.9mb` and restores the handlers functionality. ```bash $ esbuild --bundle lib/assertions/providers/lambda-handler/index.ts --target=node14 --platform=node --outfile=lib/assertions/providers/lambda-handler.bundle/index.js lib/assertions/providers/lambda-handler.bundle/index.js 1.9mb⚠️ ``` ## Why need this change? After merged #26271, Assertions lambda function throw error at initialize phase and not working. This reason of problem is maybe to import `aws-cdk-lib/custom-resources` to use aws-sdk v2-v3 translate features. `aws-cdk-lib/custom-resources` exports the some constructs, so when import the package, then bundle size is too large and lambda function is not working. ```log INIT_START Runtime Version: nodejs:18.v8 Runtime Version ARN: arn:aws:lambda:us-east-1::runtime:2755dc322c8dbb64760145d6403d14432af527bf4dd3cf03713aae10e0f8b552 -- 2023-07-13T05:49:25.501Z undefined ERROR Uncaught Exception { "errorType": "TypeError", "errorMessage": "Class extends value undefined is not a constructor or null", "stack": [ "TypeError: Class extends value undefined is not a constructor or null", " at ../../aws-cdk-lib/core/lib/cfn-condition.ts (/var/task/index.js:796267:5)", " at __init (/var/task/index.js:9:56)", " at ../../aws-cdk-lib/core/lib/private/metadata-resource.ts (/var/task/index.js:807546:5)", " at __init (/var/task/index.js:9:56)", " at ../../aws-cdk-lib/core/lib/private/synthesis.ts (/var/task/index.js:814923:5)", " at __init (/var/task/index.js:9:56)", " at ../../aws-cdk-lib/core/lib/app.ts (/var/task/index.js:814951:5)", " at __init (/var/task/index.js:9:56)", " at ../../aws-cdk-lib/core/lib/stack.ts (/var/task/index.js:816555:5)", " at __init (/var/task/index.js:9:56)" ] } START RequestId: b6da27f5-d7ff-4f76-807b-c7f5d1f02d1f Version: $LATEST Unknown application error occurredRuntime.Unknown END RequestId: b6da27f5-d7ff-4f76-807b-c7f5d1f02d1f REPORT RequestId: b6da27f5-d7ff-4f76-807b-c7f5d1f02d1f Duration: 13083.91 ms Billed Duration: 13084 ms Memory Size: 128 MB Max Memory Used: 128 MB ``` ## Solution Extract the SDK v2 to v3 features into a new private package that can be bundled safely. Also required changes to the `aws-custom-resource` v3 handler so it can used the shared code. To make sure that the bundled files are free of start-up errors, I have added them into a post build script. ## Additional changes **Update to Encode/Decode logic** All parameters passed to a Custom Resource will be converted to string when passed to the Lambda Function handler. This is a feature/limitation by CloudFormation. The previous implementation supported special handling for boolean values via a magic string. However with the SDK v3 parameter types on API Calls are more strictly enforced, e.g. we can't pass a string anymore when a number is expected. Therefore the encoding/decoding logic has been updated to indiscriminately encode _all_ parameters with `JSON.stringify()` and reverse the operation inside the assertions provider. **Lambda Invoke Payload** Withe the SDK v3, the type of the Payload returned from a Lambda Invoke operation is now a Buffer. This makes sense from an SDK perspective since the returned payload might be a binary. However assertions can only deal with JSON payloads, so we attempt to convert and parse the payload into a JSON object. If that fails it stays unchanged. ## Acknowledgements Thanks @WinterYukky for the initial investigation! Closes #26359 Replaces #26360 ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
bmoffatt
pushed a commit
to bmoffatt/aws-cdk
that referenced
this pull request
Jul 29, 2023
## Summary Assertions handler is too large and not working. This PR extracts the SDK v2 to v3 features into a separate package. Reduces bundled size from `25.8mb` to `1.9mb` and restores the handlers functionality. ```bash $ esbuild --bundle lib/assertions/providers/lambda-handler/index.ts --target=node14 --platform=node --outfile=lib/assertions/providers/lambda-handler.bundle/index.js lib/assertions/providers/lambda-handler.bundle/index.js 1.9mb⚠️ ``` ## Why need this change? After merged aws#26271, Assertions lambda function throw error at initialize phase and not working. This reason of problem is maybe to import `aws-cdk-lib/custom-resources` to use aws-sdk v2-v3 translate features. `aws-cdk-lib/custom-resources` exports the some constructs, so when import the package, then bundle size is too large and lambda function is not working. ```log INIT_START Runtime Version: nodejs:18.v8 Runtime Version ARN: arn:aws:lambda:us-east-1::runtime:2755dc322c8dbb64760145d6403d14432af527bf4dd3cf03713aae10e0f8b552 -- 2023-07-13T05:49:25.501Z undefined ERROR Uncaught Exception { "errorType": "TypeError", "errorMessage": "Class extends value undefined is not a constructor or null", "stack": [ "TypeError: Class extends value undefined is not a constructor or null", " at ../../aws-cdk-lib/core/lib/cfn-condition.ts (/var/task/index.js:796267:5)", " at __init (/var/task/index.js:9:56)", " at ../../aws-cdk-lib/core/lib/private/metadata-resource.ts (/var/task/index.js:807546:5)", " at __init (/var/task/index.js:9:56)", " at ../../aws-cdk-lib/core/lib/private/synthesis.ts (/var/task/index.js:814923:5)", " at __init (/var/task/index.js:9:56)", " at ../../aws-cdk-lib/core/lib/app.ts (/var/task/index.js:814951:5)", " at __init (/var/task/index.js:9:56)", " at ../../aws-cdk-lib/core/lib/stack.ts (/var/task/index.js:816555:5)", " at __init (/var/task/index.js:9:56)" ] } START RequestId: b6da27f5-d7ff-4f76-807b-c7f5d1f02d1f Version: $LATEST Unknown application error occurredRuntime.Unknown END RequestId: b6da27f5-d7ff-4f76-807b-c7f5d1f02d1f REPORT RequestId: b6da27f5-d7ff-4f76-807b-c7f5d1f02d1f Duration: 13083.91 ms Billed Duration: 13084 ms Memory Size: 128 MB Max Memory Used: 128 MB ``` ## Solution Extract the SDK v2 to v3 features into a new private package that can be bundled safely. Also required changes to the `aws-custom-resource` v3 handler so it can used the shared code. To make sure that the bundled files are free of start-up errors, I have added them into a post build script. ## Additional changes **Update to Encode/Decode logic** All parameters passed to a Custom Resource will be converted to string when passed to the Lambda Function handler. This is a feature/limitation by CloudFormation. The previous implementation supported special handling for boolean values via a magic string. However with the SDK v3 parameter types on API Calls are more strictly enforced, e.g. we can't pass a string anymore when a number is expected. Therefore the encoding/decoding logic has been updated to indiscriminately encode _all_ parameters with `JSON.stringify()` and reverse the operation inside the assertions provider. **Lambda Invoke Payload** Withe the SDK v3, the type of the Payload returned from a Lambda Invoke operation is now a Buffer. This makes sense from an SDK perspective since the returned payload might be a binary. However assertions can only deal with JSON payloads, so we attempt to convert and parse the payload into a JSON object. If that fails it stays unchanged. ## Acknowledgements Thanks @WinterYukky for the initial investigation! Closes aws#26359 Replaces aws#26360 ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Assertions handler is too large and not working so optimize handler size using inject of esbuild (https://esbuild.github.io/api/#inject).
This PR makes bundle size is 25.8mb -> 1.9mb
Why need this change?
After merged #26271, Asserstions lambda function throw error at initialize phase and not working. This reason of problem is maybe to import
aws-cdk-lib/custom-resourcesto use aws-sdk v2-v3 translate features.aws-cdk-lib/custom-resourcesexports the some constructs, so when import the package, then bundle size is too large and lambda function is not working.Other solution
Create new package that include only aws-sdk v2-v3 translate feature. I choosed "inject of esbuild" solution, because I think using "inject of esbuild" solution is easier than to create new package. However, this "inject of esbuild" solution is more obfuscated than "new package" solution, so when we use same solution at also other package, we should use the "new package" solution.
Closes #26359.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license