[Tracking Issue] SAM CLI + CDK Beta release feedback

[praneetap]

We recently announced the public preview of SAM CLI support for local development and testing of CDK projects.
With this preview release, you can use AWS SAM CLI to build, test, and in the future, package applications defined using the AWS CDK.

To install this beta release, follow instructions here.

We would love your feedback on the feature! Please feel free to leave your thoughts/ideas in the comments.

[sonya]

It looks like the homebrew tap for aws-sam-cli-beta-cdk is currently pointing to a tag that no longer exists (1.29.0.dev202108311500), will this be updated soon?

[wchengru]

Thanks for reporting this @sonya ! We have recreated this tag and updated our homebrew config. Can you please try downloading the source, or installing it with homebrew --build-from-source option?

Please find the new release tag here: https://github.com/aws/aws-sam-cli/releases/tag/v1.29.0.dev202108311500

[sonya]

Thanks @wchengru, i was able to install directly with brew install aws-sam-cli-beta-cdk. 🎉

I recently tried out the high level PythonFunction construct in CDK and was impressed by how easily it packaged up all my dependencies. Very excited about the potential of adding SAM to making local testing easier!

I just tried running sam-beta-cdk directly in an existing project that uses CDK to set up an API Gateway and Lambda function in Python 3.8 and ran info a few issues so far (I haven't spent too much time so any of these could be user error):

• My app file (in bin) defines two stacks in two accounts. With regular cdk I can select which stack to build by passing it as a command line parameter. When using sam-beta-cdk I can't seem to make it build only one stack.
• After making my appfile define only one stack, sam-beta-cdk build errors out with Invalid executable for python at /usr/bin/python. When building with CDK, it uses pipenv and doesn't have this issue.
• Lambda invocations attempted via sam-beta-cdk local invoke and sam-beta-cdk local start-api time out, which may or may not be related to the previous issue with building python

[richard-olson]

I've found it unusable at the moment. The CDK L2 constructs sometimes create 'helper' lambdas for configuration, and sam-beta-cdk tries to compile these during the build process. For example, adding a CloudWatch log group as the target of an EventBridge rule in CDK creates a Lambda that creates a Custom::CloudwatchLogResourcePolicy, and SAM attempts to build this Lambda, which fails.

Have raised issue #3192

[iRoachie]

Hey team, overall I'm pretty impressed with the implementation so far. The only thing I find lacking is multiple stages.

With sam, using samconfig.toml, I'd setup different stages:

[staging]
[staging.deploy]
[staging.deploy.parameters]
stack_name = "project-staging"
parameter_overrides = "Environment=\"staging\""

[prod]
[prod.deploy]
[prod.deploy.parameters]
stack_name = "project-prod"
parameter_overrides = "Environment=\"prod\""

And use conditional logic in my template using parameter_overrides. for example:

Parameters:
  Environment:
    Type: String
    AllowedValues:
      - staging
      - prod

Deployments are then easy by specifying the stage e.g sam deploy --config-env staging

This workflow is impossible now since cdk is synthed before SAM evaluates these parameters.

[rupe120]

Why does a Python CDK project initialized with SAM not reference a Virtual Environment like is when initializing a Python CDK project with the CDK cli? Does the sam build manage that aspect for you?

[rupe120]

I just created a bug for sam-beta-cdk build --profile . The switch appears to be ignored.

#3427

Also, related and compounding the problem is the fact that if the environment variables are used to initialize the stack account and region, and the default profile config is empty, the region in the config is never set to unknown-region. So I created a bug for that as well.

#3428

[Ma11hewThomas]

Hey, I'm using M1 Pro on Monterey, brew install aws-sam-cli-beta-cdk returns cannot be found. Strangely it works on my old mac with Mojave.

Warning: No available formula or cask with the name "aws-sam-cli-beta-cdk".

Any advice is greatly appreciated, thanks

[iRoachie]

@matthewthomasgb can try brew install --build-from-source aws-sam-cli-beta-cdk

[Ma11hewThomas]

@iRoachie thanks for getting back to me, unfortunately I get the same result -
homebrew git:(stable) brew install --build-from-source aws-sam-cli-beta-cdk Warning: No available formula or cask with the name "aws-sam-cli-beta-cdk". ==> Searching for similarly named formulae... Error: No similarly named formulae found. ==> Searching for a previously deleted formula (in the last month)... Error: No previously deleted formula found. ==> Searching taps on GitHub... Error: No formulae found in taps.

[moelasmar]

@matthewthomasgb .. thanks for raising this issue .. There is already a requested feature asking for M1 Mac homebrew bottle. This will fix the SAM CLI installation in M1 mac. Please +1 this issue to prioritize it.

[moelasmar]

@sonya .. thanks for sharing your feedback .. can I ask you to give us more info about the sam-beta-cdk local invoke and sam-beta-cdk local start-api issues, if it is possible to refer us to some github project that we can use to reproduce this issue.
Regarding the sam-beta-cdk build issue, we need to know more about your use case of using sam-beta-cdk build to build your CDK project. Was it because you just want to try the beta version, or you was not be able to build your lambda functions using CDK build features (Assets bundling option, and Runtime Lambda Function Constructs Python, nodeJs, go), and you need to make use of sam build features.

[cdanhowell]

First - I absolutely love this and sincerely appreciate the addition of this integration.

I am currently running into an issue that appears to have been resolved for the released versions. I suspect that this beta may be outdated compared to release, but I'm not sure. The referenced issue is #3243.

Secondly, I noticed something interesting. The CDK cli and SAM cli use two different .cdk-out folders. If possible, it would be great if sam-cli would check for a .cdk-out that is in the current working directory before creating its .sam-cli/.cdk-out directory (at least for cdk projects).

Third, I am wondering how well this will play together with CDK deployed apps. I was noticing in my testing that I was getting an invalid ARN for the KMS service (and possibly others, though I didn't check). The setup is this:

• I have cdk app that I have deployed to a dev/staging environment.
• The app creates several resources, including an S3 bucket, an SSM secret, a KMS key, and an RDS database in addition to the lambda function.
• When running sam local invoke [function name], the ARN that is passed for the KMS key does not resolve to a valid resource in my account, thus when accessing the resource to encrypt a string, it fails.

The workaround would be to develop the lambda functionality with sam deploy then use cdk in the pipelines. That is a valid solution for us, though I think having the two CLI's play nicely would be extremely helpful in a number of scenarios.

Thanks again for pushing something like this forward!

[praneetap]

First - I absolutely love this and sincerely appreciate the addition of this integration.

I am currently running into an issue that appears to have been resolved for the released versions. I suspect that this beta may be outdated compared to release, but I'm not sure. The referenced issue is #3243.

Secondly, I noticed something interesting. The CDK cli and SAM cli use two different .cdk-out folders. If possible, it would be great if sam-cli would check for a .cdk-out that is in the current working directory before creating its .sam-cli/.cdk-out directory (at least for cdk projects).

Third, I am wondering how well this will play together with CDK deployed apps. I was noticing in my testing that I was getting an invalid ARN for the KMS service (and possibly others, though I didn't check). The setup is this:

* I have cdk app that I have deployed to a dev/staging environment.

* The app creates several resources, including an S3 bucket, an SSM secret, a KMS key, and an RDS database in addition to the lambda function.

* When running sam local invoke [function name], the ARN that is passed for the KMS key does not resolve to a valid resource in my account, thus when accessing the resource to encrypt a string, it fails.

The workaround would be to develop the lambda functionality with sam deploy then use cdk in the pipelines. That is a valid solution for us, though I think having the two CLI's play nicely would be extremely helpful in a number of scenarios.

Thanks again for pushing something like this forward!

Hey @cdanhowell , thank you so much for the feedback! I would love to get on a call and understand your use cases better, please reach out to me via DM and I will set up a call!

[wazcov]

@iRoachie thanks for getting back to me, unfortunately I get the same result - homebrew git:(stable) brew install --build-from-source aws-sam-cli-beta-cdk Warning: No available formula or cask with the name "aws-sam-cli-beta-cdk". ==> Searching for similarly named formulae... Error: No similarly named formulae found. ==> Searching for a previously deleted formula (in the last month)... Error: No previously deleted formula found. ==> Searching taps on GitHub... Error: No formulae found in taps.

same

[Kelvijn]

@iRoachie thanks for getting back to me, unfortunately I get the same result - homebrew git:(stable) brew install --build-from-source aws-sam-cli-beta-cdk Warning: No available formula or cask with the name "aws-sam-cli-beta-cdk". ==> Searching for similarly named formulae... Error: No similarly named formulae found. ==> Searching for a previously deleted formula (in the last month)... Error: No previously deleted formula found. ==> Searching taps on GitHub... Error: No formulae found in taps.

same

Same, does anyone have a temp solution instead of deploying to aws for each code change? 😓😭

[moelasmar]

@Kelvijn .. is you issue related to install the CDK beta version on M1 Mac? .. I do not get your question about a deploying to aws for each code change?

[jcaito]

@Kelvijn / @wazcov / @matthewthomasdev

Run brew tap aws/tap before running brew install aws-sam-cli-beta-cdk and that should work. You should be able to verify it is working by doing a brew search aws-sam-cli

[moelasmar]

@jcaito .. thanks for your help.

I verified that it is working on Big Sur mac with the following details

administrator@46559 ~ % sysctl -a | grep machdep.cpu
machdep.cpu.brand_string: Apple M1
machdep.cpu.core_count: 8
machdep.cpu.cores_per_package: 8
machdep.cpu.logical_per_package: 8
machdep.cpu.thread_count: 8
administrator@46559 ~ % system_profiler SPSoftwareDataType
Software:

    System Software Overview:

      System Version: macOS 11.6 (20G165)
      Kernel Version: Darwin 20.6.0
      Boot Volume: Macintosh HD
      Boot Mode: Normal
      Computer Name: 46559
      User Name: administrator (administrator)
      Secure Virtual Memory: Enabled
      System Integrity Protection: Enabled
      Time since boot: 6 days 6:37

I just did as explained by @jcaito

administrator@46559 ~ % brew tap aws/tap
==> Tapping aws/tap
Cloning into '/opt/homebrew/Library/Taps/aws/homebrew-tap'...
....
Tapped 15 formulae (48 files, 516.9KB).

administrator@46559 ~ % brew install aws-sam-cli-beta-cdk
==> Downloading https://ghcr.io/v2/homebrew/core/gdbm/manifests/1.22
 ...... 
🍺  /opt/homebrew/Cellar/aws-sam-cli-beta-cdk/202108311500: 6,614 files, 103.4MB, built in 29 seconds

administrator@46559 ~ % sam-beta-cdk --version
SAM CLI, version 1.29.0.dev202108311500

@Kelvijn @wazcov @matthewthomasdev .. could you please confirm if this fixes your issue, or you still have installation issue.

[karl-dau]

Is sam-beta-cdk build supposed to work with awslambdago.NewGoFunction when using awslambda.Runtime_PROVIDED_AL2() ?

I get the following error:

Synthesizing CDK App
<snip>
Running CustomMakeBuilder:CopySource
Running CustomMakeBuilder:MakeBuild

Build Failed
Error: CustomMakeBuilder:MakeBuild - Makefile not found at xxx/.aws-sam/.cdk-out/asset.fb7482be6f42076701659796fc3281b2003f4b30aacb6594b3bee6cfe40ab765/Makefile

Which makes sense, seeing as there is no Makefile required like when using custom runtimes with SAM CLI. Adding a Makefile next to the Go source also does not work, because CustomMakeBulder:CopySource does not actually copy any of the source.

EDIT: Seems the runtime does not matter, then just complains about missing Go files. CustomMakeBuilder:CopySource copies the already built binary.

[goodacres]

Thanks very much to the team for the efforts on SAM+CDK local testing integration to date. My team and I are hoping to migrate from Severless Framework to AWS CDK. Whilst CDK doesn't natively provide local development testing functionality whereas Serverless Framework does, I was hoping to be able to use sam-beta-cdk to fill the gaps for our Lambda deployments.

Something I came across during testing however is that presently sam-beta-cdk only supports local testing of Lambda functions created with the aws_lambda CDK construct:
https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-cdk-testing.html

During refactoring some of our Lambda apps from Serverless Framework -> AWS CDK, I decided that ideally we would use the aws_lambda_nodejs construct as it allows for much simpler dependency management and project structuring of nodejs functions.

I was wondering if anyone would be able to point me in the right direction regarding the reasons for the current limitation and if providing support for other Lambda L2 constructs is on the roadmap?

Thanks in advance!

[rupe120]

Thanks very much to the team for the efforts on SAM+CDK local testing integration to date. My team and I are hoping to migrate from Severless Framework to AWS CDK. Whilst CDK doesn't natively provide local development testing functionality whereas Serverless Framework does, I was hoping to be able to use sam-beta-cdk to fill the gaps for our Lambda deployments.

Something I came across during testing however is that presently sam-beta-cdk only supports local testing of Lambda functions created with the aws_lambda CDK construct: https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-cdk-testing.html

During refactoring some of our Lambda apps from Serverless Framework -> AWS CDK, I decided that ideally we would use the aws_lambda_nodejs construct as it allows for much simpler dependency management and project structuring of nodejs functions.

I was wondering if anyone would be able to point me in the right direction regarding the reasons for the current limitation and if providing support for other Lambda L2 constructs is on the roadmap?

Thanks in advance!

sam build / sam-beta-cdk build manage dependencies based on the runtime specified for the Lambda

[rupe120]

I'm receiving the error Need to perform AWS calls for account ###, but no credentials have been configured when I am trying run build with ec2.Vpc.from_lookup() in a stack.

I tried both the --profile switch and the access key environment variables, and received the same error with both.

How do I provide the credentials?

---

I was able to get it to work by clearing out the existing .aws-sam folder content. Though only the environment variable credentials work, the --profile switch is not actually implemented yet.

[iRoachie]

Is there anyway to update the sam-cli version? Seems sam-beta-cdk is stuck on 1.29.0

[iRoachie]

Hey @praneetap I just saw release 1.37.0. Does this mean that aws-sam-cli-beta-cdk is no longer needed?

[praneetap]

Yes! You should be able to get it on mainline! Let us know what you think :)

[moelasmar]

Thanks all for your feedback on the beta version. The CDK support got released in version 1.37.0.

Please check the CDK support documentation for the updated information about the CDK support usage. If you have any questions, features, or issue, please do not hesitate to open a github issue.

[kenchoong]

I can run my CDK stack locally using SAM by using this guide

My APIgateway need to use Lambda Authorizer. Is that anyway to to test the APIgateway locally which have Authorizer using SAM? If can, please provide me a guide. Thanks

[mikelxc]

Same issue here, cdk and sam are setup and working properly, but when invoking local-api (which calls lambda from another stack), it throws me error of function not found.

[moelasmar]

Thanks @mikelxc for your feedback. Unfortunately SAM CLI currently does not support testing multiple stacks. So, the API gateway resources and Lambda functions have to be in the same stack so sam local start-api can support them.

[moelasmar]

Sorry @kenchoong, unfortunately sam local start-api does not support API Gateway Authorizer right now.

[niluckne]

Can you inform on your planning wrt

and in the future, package applications

That would be great.