Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Example: AWS SAM #716

Closed
0xdevalias opened this issue Sep 14, 2018 · 30 comments
Closed

Example: AWS SAM #716

0xdevalias opened this issue Sep 14, 2018 · 30 comments

Comments

@0xdevalias
Copy link
Contributor

@0xdevalias 0xdevalias commented Sep 14, 2018

I think it could be useful to have a basic canonical example of how to use SAM from CDK (See also: #703, aws/aws-sam-cli#663)

If it would be useful, I could probably knock together what I figured out here into a PR to examples? If so.. what would be the best naming/etc for it? Potentially could choose one of the simpler examples from the following and translate it to CDK:

I found the following, so should be doable:

But when I was exploring the docs, it didn't look like I could do much without dropping into the cloudformation namespace (which is probably fine, as this is 'lower level/direct mapping'):

So I decided to try and make a basic function example from that, just to see how it would work. It took me a little while to figure the intricacies of the types used and their interrelations:

import sam = require('@aws-cdk/aws-serverless');
const helloWorld = new sam.cloudformation.FunctionResource(this, "HelloWorld", {
            functionName: "HelloWorld",
            description: "Greeting the world",
            codeUri: "./target",
            handler: "hello-world",
            runtime: lambda.Runtime.Go1x.name,
            tracing: "active",
            events: {
                CatchAll: {
                    type: "Api",
                    properties: {
                        path: "/{proxy+}",
                        method: "ANY",
                        // restApiId: ""
                    }
                }
            }
        });

Which synthesises to:

Transform: 'AWS::Serverless-2016-10-31'
Resources:
    HelloWorld:
        Type: 'AWS::Serverless::Function'
        Properties:
            CodeUri: ./target
            Handler: hello-world
            Runtime: go1.x
            Description: 'Greeting the world'
            Events:
                CatchAll:
                    Properties:
                        Method: ANY
                        Path: '/{proxy+}'
                    Type: Api
            FunctionName: HelloWorld
            Tracing: active
    CDKMetadata:
        Type: 'AWS::CDK::Metadata'
        Properties:
            Modules: '@aws-cdk/assets=0.9.0,@aws-cdk/aws-cloudwatch=0.9.0,@aws-cdk/aws-codepipeline-api=0.9.0,@aws-cdk/aws-ec2=0.9.0,@aws-cdk/aws-elasticloadbalancing=0.9.0,@aws-cdk/aws-events=0.9.0,@aws-cdk/aws-iam=0.9.0,@aws-cdk/aws-kms=0.9.0,@aws-cdk/aws-lambda=0.9.0,@aws-cdk/aws-s3=0.9.0,@aws-cdk/aws-s3-notifications=0.9.0,@aws-cdk/aws-serverless=0.9.0,@aws-cdk/aws-sqs=0.9.0,@aws-cdk/cdk=0.9.0,@aws-cdk/cx-api=0.9.0,js-base64=2.4.5,poc-aws-cdk=0.1.0'

CDK References:

SAM References:

@eladb
Copy link
Contributor

@eladb eladb commented Sep 16, 2018

Eventually, you should be able to achieve the same results you currently achieve with the SAM resources by simply using the AWS Construct Library.

The CDK already has support for Lambda via the @aws-cdk/aws-lambda module, and we are working on an API Gateway module (see #665) which will allow you to implement the above example like this:

const handler = new lambda.Function(this, 'HelloWorldHandler', {
  description: 'Greeting the world',
  code: apigw.Lambda.directory('./target'),
  handler: 'hello-world',
  runtime: lambda.Runtime.Go1x,
  tracing: lambda.Tracing.Active
});

const api = new apigw.RestApi(this, 'HelloWorldApi');
api.root.onMethod('ANY', handler);

Furthermore, we are looking at ways to allow people to use SAM CLI with CDK Apps, in order to enable local debugging of SAM apps written using the CDK (whether or not they use the SAM resources).

@0xdevalias
Copy link
Contributor Author

@0xdevalias 0xdevalias commented Sep 17, 2018

Eventually, you should be able to achieve the same results you currently achieve with the SAM resources by simply using the AWS Construct Library.

Expected that would be the case based on the wording of how it was described. Until then though, it will be nice to be able to leverage the existing abstractions SAM offers.

The CLI tie in will be nice.

@eladb
Copy link
Contributor

@eladb eladb commented Sep 17, 2018

it will be nice to be able to leverage the existing abstractions SAM offers.

Is the @aws-cdk/aws-sam library what you were looking for or did you look for some richer APIs on top of it?

@0xdevalias
Copy link
Contributor Author

@0xdevalias 0xdevalias commented Sep 18, 2018

As per my example above (probably edited original issue as I worked it all out), it is, but this issue was more about making a canonical example of using it so that others don't have to dig through everything to figure it out.

@jmhummel
Copy link

@jmhummel jmhummel commented Sep 28, 2018

One issue I saw when I attempted to deploy my SAM resources via CDK:

        new FunctionResource(this, 'HelloWorld', {
            codeUri: './src',
            handler: 'handler.handler,
            description: 'Hello World',
            runtime: 'nodejs8.10',
            events: {
                Endpoint: {
                    properties: {
                        path: '/helloworld,
                        method: 'get',
                    },
                    type: 'Api',
                }
            }
        });

I ran cdk synth hello-world > template.yml in order to use SAM
The SAM CLI expects codeUri to be local path. However, this fails at cdk deploy:

Deployment of stack hello-world failed: Error: Failed to create ChangeSet CDK-5bb75637-dc16-4c02-bc7c-2b7bfb218223 on hello-world: FAILED, Transform AWS::Serverless-2016-10-31 failed with: Invalid Serverless Application Specification document. Number of errors found: 1. Resource with id [HelloWorldFBEC9B61] is invalid. 'CodeUri' is not a valid S3 Uri of the form "s3://bucket/key" with optional versionId query parameter.

If I use an asset to upload the code, and pass the s3 information:

codeUri: {
                bucket: codeAsset.s3BucketName,
                key: codeAsset.s3ObjectKey,
            },

It succeeds, but now I can't use template.yml to run sam local start-api as it can't find the handlers

@jmhummel
Copy link

@jmhummel jmhummel commented Sep 29, 2018

My current workaround:

        const createSamTemplate = this.getContext("sam");

        const codeAsset = new ZipDirectoryAsset(this, 'CodeAsset', {
            path: './src',
        });

        new FunctionResource(this, id, {
            codeUri: createSamTemplate ? './src' : {
                bucket: codeAsset.s3BucketName,
                key: codeAsset.s3ObjectKey,
            },

I run cdk synth hello-world -c sam=true > template.yml and pass in the context var to use local src and deploy with cdk deploy

@eladb
Copy link
Contributor

@eladb eladb commented Sep 29, 2018

Hey, thanks for this! We are looking at streamlining this experience. Stay tuned :-)

@aripalo
Copy link

@aripalo aripalo commented Dec 20, 2018

What @eladb said:

Furthermore, we are looking at ways to allow people to use SAM CLI with CDK Apps, in order to enable local debugging of SAM apps written using the CDK (whether or not they use the SAM resources).

That would be super useful. I don't see much purpose on using SAM when using CDK, but currently the biggest pain point by doing so is not having support for SAM CLI (or not having other similar local development toolkit); So basically there isn't a sane local development setup.

Of course one can always build something custom around it, but reinventing the wheel isn't really what I want 😅 … and also as we are using a compiled language (Golang) to write our functions that also makes writing "local invoke wrappers" harder than with - say- NodeJS.

We're currently developing with AppSync, Golang Lambdas and CDK. I know, I know CDK docs say not to use it for production, but it's just exactly the tooling we've been looking for 🙂 and we do vendor in the synthetized CloudFormation templates to Github (just to be on the safe side).

@mipearson
Copy link

@mipearson mipearson commented Jan 13, 2019

Chucking this here as it may be useful to others looking to combine these tools.

The following works just fine:

    const fn = new lambda.Function(this, "Func", {
      ..
    });

    new cdk.Output(this, "FunctionName", { value: fn.functionName });

Then on deploy:

Outputs:
Stack.FunctionName = Stack272B588B-NRQ0Y3ASCFAX

And in your terminal:

sam logs -tn Stack272B588B-NRQ0Y3ASCFAX
@eladb
Copy link
Contributor

@eladb eladb commented Jan 13, 2019

@mipearson thanks for this. Useful for us to learn about this use case. Copying @sam-goodwin

@Doug-AWS
Copy link
Contributor

@Doug-AWS Doug-AWS commented Jan 23, 2019

Let me know if you have some info that we want to put into the guide.

@brettswift
Copy link

@brettswift brettswift commented May 12, 2019

Not sure if there's a better spot for this comment, but with respect to SAM, I think the biggest feature is Lambda safe deployments, and the orchestration it does with CodeDeploy applications and canary cutover.

@eladb in your discussions have there been talk of native support of this in CDK? This is currently the only reason I see to output SAM templates and run a sam deploy. (but I'm also new to SAM so there could be many others). I can obviously implement this myself with or without SAM, but it would help my roadmap to know if you see this in the roadmap (either before or after GA). Thanks!

@eladb
Copy link
Contributor

@eladb eladb commented May 13, 2019

The @aws-cdk/aws-codedeploy package supports AWS Lambda traffic shifting. Can you take a look there and let us know if this is what you were looking for?

We should document this in the @aws-cdk/aws-lambda package

@cortopy
Copy link

@cortopy cortopy commented May 13, 2019

In my opinion @aws-cdk/sam was deprecated too early. There are two major features missing in @aws-cdk/lambda for me:

  • local invoke
  • logs
@eladb
Copy link
Contributor

@eladb eladb commented May 13, 2019

Local invokes and Lambda logs are supported by SAM CLI. We are working on documenting this and @jfuss can provide some details.

@brettswift
Copy link

@brettswift brettswift commented May 13, 2019

To reword my question - cdk doesn't run aws deploy create-deployment.

I found #1938 which seems to have my answer. I'll track that one. Thanks!

@jfuss
Copy link
Contributor

@jfuss jfuss commented May 13, 2019

@cortopy You can use SAM CLI with CDK today. When you author a template with CDK, you can 'sam local [invoke|start-lambda]' or sam logs by doing the following.

$ cd into/cdk/project
$ cdk synth > template.yaml
$ sam local invoke <LogicalIdOfFunction>

Some notes/current gotchas:

  • You will to compile/build your function into where ever you asset directory is.
  • sam local start-api is not yet supported.
  • CDK generates logical Ids and SAM CLI needs you to pass in the Logical Id so it knows what function you want to invoke. So you will need to manually inspect the cdk synth output.
  • You will need CDK version v0.22.0 or greater
  • You will need SAM CLI version v0.11.0 or greater
@cortopy
Copy link

@cortopy cortopy commented May 14, 2019

@jfuss that indeed works and have been using it. The problem is that I don't feel like the cdk can replace any of my usual serverless development flows at this moment.

synth > invoke doesn't pick up changes in local files. It seems to execute the function locally with code that is compiled/packaged for its upload to s3 (i.e.: invoke seems to get code from .cdk.staging)

So I have to rather do (1) tsc (for aws cdk in typescript) > (2) bootstrap > (3) synth > (4) invoke to see my changes. This means that invoking a function after changing code is exasperatingly slow. Even more so because realistically in a node lambda, one would also have webpack/babel, etc. at some point in the pipeline.

I suspect (pure guess at this stage) this is because the output of synth points to the code in the s3 bucket for the lambda function. In a SAM template, there's a codeUri which points to code in local

Also, as a side note, the synth > invoke pipeline doesn't inject the object passed in the environment property of the Lambda construct.

@Doug-AWS
Copy link
Contributor

@Doug-AWS Doug-AWS commented May 14, 2019

FYI: The CDK Developer Guide currently has a short description of using the SAM CLI to test Lambda functions at https://docs.aws.amazon.com/cdk/latest/guide/tools.html#sam.

@SanderKnape
Copy link

@SanderKnape SanderKnape commented May 31, 2019

Hey @cortopy. If you run cdk synth with the --no-staging flag, it will not stage the Lambda code to the .cdk.staging directory. Instead the Metadata in the CloudFormation will contain a reference to the source code you are developing. That way, SAM will directly pick up the changes without the need to run cdk synth again.

The CDK is using a SAM feature for this that is explained here: https://github.com/awslabs/aws-sam-cli/blob/master/designs/resource_metadata_overriding.md

@cpmech
Copy link

@cpmech cpmech commented Jun 26, 2019

@cortopy You can use SAM CLI with CDK today. When you author a template with CDK, you can 'sam local [invoke|start-lambda]' or sam logs by doing the following.

$ cd into/cdk/project
$ cdk synth > template.yaml
$ sam local invoke <LogicalIdOfFunction>

Some notes/current gotchas:

* You will to compile/build your function into where ever you asset directory is.

* `sam local start-api` is not yet supported.

* CDK generates logical Ids and SAM CLI needs you to pass in the Logical Id so it knows what function you want to invoke. So you will need to manually inspect the `cdk synth` output.

* You will need CDK version v0.22.0 or greater

* You will need SAM CLI version v0.11.0 or greater

Hi, I'm just wondering if there is an estimate for when sam local start-api would be supported?

@jfuss
Copy link
Contributor

@jfuss jfuss commented Jun 26, 2019

@cpmech CDK currently only supports generating the raw API Gateway resources. So the two paths to the support would be:

  1. CDK generates Swagger instead of resources for customers
  2. SAM CLI support the raw API Gateway Resources.

I know there is an issue for the first, but I wouldn't expect this to cover all customers, given the current state.

On the second, we are currently under going some work on support API Gateway Resources. Design PR. We plan on supporting RestApi with swagger to start, while we try to understand the full need for support the wider API Gateway Resources (such as AWS::ApiGateway::Method).

@jfuss
Copy link
Contributor

@jfuss jfuss commented Aug 29, 2019

@cpmech Circling back here. We released v0.21.0 earlier this week. This has support for reading and understanding AWS::ApiGateway::Methods, AWS::ApiGateway::RestApi, AWS::ApiGateway::Resource, and AWS::ApiGateway::Stage. This will allow you to do a cdk synth > template.yaml and then sam local start-api to test your functions.

@eladb Might be able to close this out. Could be useful to expand the docs though. I don't have direct bandwidth to tackle a doc update right now though.

@cpmech
Copy link

@cpmech cpmech commented Aug 30, 2019

Fantastic! Thanks a lot!

@NGL321
Copy link
Member

@NGL321 NGL321 commented Sep 5, 2019

An issue feels like a suboptimal way of addressing this, I think this is a great case for having a SAM example in the examples repo.

😸

@NGL321 NGL321 mentioned this issue Sep 5, 2019
1 of 1 task complete
sinistral added a commit to sinistral/aws-mis that referenced this issue Sep 19, 2019
A `--no-staging` syth sets the asset path to the local Lambda deployment
packager, (rather than CDK's asset bundle), which local Lambda honours.

See: aws/aws-cdk#716 (comment)
@Poweranimal
Copy link

@Poweranimal Poweranimal commented Oct 7, 2019

@jfuss Thank you very much for the SAM Local support of the additional AWS::ApiGateway resources.
I noticed that AWS::Lambda::Alias is not yet supported.

@include
Copy link

@include include commented Nov 13, 2019

Nice thread so... sorry for asking this here but couldn't find any reliable place to write my question.

Why would someone use CDK to generate SAM?
I see the point of using CDK which I like but I can only see the local dev vantage of using SAM local.
Is this what CDK >(generate)> SAM approach is trying to leverage? How do you structure such project?

Thank you :)

@BinaryShrub
Copy link

@BinaryShrub BinaryShrub commented Dec 23, 2019

A little rough... but I ended up skipping SAM all-together and wrote a little invoke script like so:

process.env = {
  ...process.env,
  JWT_TOKEN_SECRET_NAME: 'blah',
  JWT_EXPIRATION_TIME: '5m'
}

import { handler } from './src/app'
import { APIGatewayProxyEvent } from 'aws-lambda'

const events: { [key: string]: APIGatewayProxyEvent } = {
  createSession: {
    requestContext: {
      httpMethod: 'POST',
      resourcePath: '/sessions'
    },
    body: JSON.stringify({
      username: 'user',
      password: 'pass'
    })
  } as APIGatewayProxyEvent
}

;(async () => {
    console.log(await handler(events[process.argv[2]]))
})()

package.json

  "scripts": {
    "local": "npx ts-node local.ts $2"
  }

executed with:

$ yarn local createSession
@ranguard
Copy link
Contributor

@ranguard ranguard commented Jan 1, 2020

The setup I use for developing lambda functions locally is: https://gist.github.com/ranguard/3befa8344678c9fe90f34a81e3a5ca8d this is just a small step on from the documented approach. All other resources (Dyn/S3 etc) are deployed and referenced from there.

@nija-at
Copy link
Contributor

@nija-at nija-at commented Mar 10, 2020

Closing this issue since we don't have anything concrete to do here at this time.

In general, better interplay between the CDK and SAM would be to an advantage. However, we don't have anything to offer just yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet