Possibility to get AWS::Serverless::RestApi passed a template variable to Template files with Functions only

We have got a problem where the resource limit of CF is hit when transformed from our SAM Template file.

Given all our Lambdas are behind APIs and we want to maintain all APIs from Single API Gateway, and that we now have to split the SAM Template into multiples to make them independently deploy-able, is there a way that we define one template that has only AWS::Serverless::Api to build out API Gateway with given Swagger file, and then to pass its reference to other template files that will be deploying our AWS::Serverless::Functions.

We simply want to pass the reference of RestApiId to the template files as a CFN Parameter.

Is this possible at this moment? As the error on the attempt says otherwise - RestApiId must be a valid reference to an 'AWS::Serverless::Api' resource in same template, which is quite dis-heartening :-(

You can use CloudFormation export and import to pass the parameters from one stack to a different stack: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-cfn-stack-exports.html

To deal with resource limit issue, CloudFormation recommends to use nested stacks: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-cfn-nested-stacks.html

The caveat here is that nested stack does not support SAM. One way to get around that is to only define the non serverless resources in nested stacks and still leave the serverless resources in the top level SAM template.
Edit: Nested stacks are now supported, starting November 2018

@honglu

@honglu thanks for your reply.

We have been doing what you suggested above and now only serverless resources are left in the SAM when we hit the CF limit after transformation.

There were 3 options we came up with and tried but all failed:

Define API gateway as a separate SAM file, take export of created Gateway. Split Lambdas into multiple Lambda-only SAM files, with all of these taking already created API Gateway as input for their event.properties.RestApiId => That fails due to the limitation that SAM has i.e. the RestApiId must be a valid reference of serverless:api type resource in the same template file.
Define API gateway as a separate SAM file and create Role/Policies such that it is allowed to invoke required functions. Split Lambdas into multiple Lambda-only SAM files but without any Api type Event defined under it. That way we can create our Lambdas and API Gateway independent of each other and integrate using Swagger and arrange for permissions separately => That fails at SAM Local level, upon which we depend a lot for local run and testing, as that now doesn't know against what event a lambda should listen.
Define API gateway as a separate SAM file and create Role/Policies such that it is allowed to invoke required functions. Split Lambdas into multiple Lambda-only SAM files but without RestApiId property for event => That gets even worse as now every lambda creates a new Api Gateway for itself assuming that is what it's asked for.
So at the end we are still blocked in that catch-22 situation due to variety of limits and limitations of SAM and CF.

So at the end we are still blocked in that catch-22 situation due to variety of limits and limitations of SAM and CF.

Is there any other way or ideas that can help us overcome it?

PS: However I also came across another issue - #335 that indicates the option-1 was working until a few days back, which is really interesting and worth watching progressing on that issue.

Hi Folks, #149 seems to imply one cannot split the api resource from the function resources. This issue seems to imply it is indeed possible, without giving template examples. I would be forever grateful if someone who's managed to do this would post an actual example that works. Many thanks in advance.

We managed to split them before, but no longer. It suddenly stopped working so we had to redo all of our stack structure which meant a downtime in production. See #335

Does anyone know if this issue is dead? Or does anyone know if there is a way to use the aws lambda update-function-code CLI method to update the code and roll it out with checkpoints?

It would also be great if RestApiId could be of type AWS::ApiGateway::RestApi instead of just AWS::Serverless::Api. We're currently using CloudFormation and would like to dry up some code by slowly switching to SAM but this prevents us from doing that.

Nested stacks are now supported in SAM, meaning you can nest resources in other stacks and use the outputs as values in the current stack.

Even though they are supported, when I created the AWS::Serverless::Api in a separate stack from an AWS::Serverless::Function I still got the above error (this was two days ago).

Can you post an example that creates this issue? Is it as simple as defining an API in one stack and referencing a Function in that API that is from a nested stack?

Any updates on this?

Seriously guys, we really need to pass down RestAPI to child stacks :D

The number of people and orgs looking for this feature. Is it really that hard to AWS SAM peeps?

So to explain, yes, this has technical limitations getting in the way. Right now, when you create an Api event source, it does two things:

Creates an AWS::Lambda::Permission resource policy.
Modifies the OpenAPI definition to associate a route/verb with a Lambda function, and adding any authorizer/etc properties to that route.

Doing step 2 is the tricky part, because it has to happen right now where the API is defined. You could use AWS::Serverless::Connector to do 1 if you have a fully defined OpenAPI spec. Though it likely would be easier for the Lambda handlers to be inputs into the template with the API definition in that case.

One possible option we could explore if there was high demand for this would be to add some sort of route helpers to the AWS::Serverless::Api resource, which would point to Lambda functions. In that case they could be in separate templates. But with how event sources are defined, it's not likely possible in a backwards compatible or maintainable way.

This doesn't answer the original question exactly (using a single API Gateway), but if you're OK using an API Gateway per project (let's say each project is a microservice made up of multiple lambdas in each project/microservice), then this is how I did it. Posting in case it's useful to anyone. This enables some modularity so you can break your services down across multiple projects. The downside is that you create an API Gateway per project (which may OK for you if you're want to use the "Multiple API Gateway" pattern referenced here: https://docs.aws.amazon.com/prescriptive-guidance/latest/modernization-integrating-microservices/api-gateway-pattern.html#multiple-api-gateways).

Details posted here:
#2703 (comment)

Adding a reproducible example for posterity.

Save the following as template.yaml:

Transform: AWS::Serverless-2016-10-31
Resources:
  MyApi:
    Type: AWS::Serverless::Api
    Properties:
      StageName: prod
  MyFunction:
    Type: AWS::Serverless::Function
    Properties:
      Runtime: python3.8
      Handler: foo
      InlineCode: bar
      Events:
        ApiEvent:
          Type: Api
          Properties:
            Method: get
            Path: /
            RestApiId: !Ref MyApi

Deploy:

sam deploy --region us-west-2 --resolve-s3 --capabilities CAPABILITY_IAM --stack-name test-349 --template template.yaml

It transforms into the following template:

Transformed CloudFormation template

{
  "Resources": {
    "MyFunction": {
      "Type": "AWS::Lambda::Function",
      "Metadata": {
        "SamResourceId": "MyFunction"
      },
      "Properties": {
        "Code": {
          "ZipFile": "bar"
        },
        "Handler": "foo",
        "Role": {
          "Fn::GetAtt": [
            "MyFunctionRole",
            "Arn"
          ]
        },
        "Runtime": "python3.8",
        "Tags": [
          {
            "Key": "lambda:createdBy",
            "Value": "SAM"
          }
        ]
      }
    },
    "MyFunctionApiEventPermissionprod": {
      "Type": "AWS::Lambda::Permission",
      "Properties": {
        "Action": "lambda:InvokeFunction",
        "FunctionName": {
          "Ref": "MyFunction"
        },
        "Principal": "apigateway.amazonaws.com",
        "SourceArn": {
          "Fn::Sub": [
            "arn:aws:execute-api:${AWS::Region}:${AWS::AccountId}:${__ApiId__}/${__Stage__}/GET/",
            {
              "__ApiId__": {
                "Ref": "MyApi"
              },
              "__Stage__": "*"
            }
          ]
        }
      }
    },
    "MyApiprodStage": {
      "Type": "AWS::ApiGateway::Stage",
      "Properties": {
        "DeploymentId": {
          "Ref": "MyApiDeployment1e98ccf811"
        },
        "RestApiId": {
          "Ref": "MyApi"
        },
        "StageName": "prod"
      }
    },
    "MyFunctionRole": {
      "Type": "AWS::IAM::Role",
      "Properties": {
        "AssumeRolePolicyDocument": {
          "Version": "2012-10-17",
          "Statement": [
            {
              "Action": [
                "sts:AssumeRole"
              ],
              "Effect": "Allow",
              "Principal": {
                "Service": [
                  "lambda.amazonaws.com"
                ]
              }
            }
          ]
        },
        "ManagedPolicyArns": [
          "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
        ],
        "Tags": [
          {
            "Key": "lambda:createdBy",
            "Value": "SAM"
          }
        ]
      }
    },
    "MyApiDeployment1e98ccf811": {
      "Type": "AWS::ApiGateway::Deployment",
      "Properties": {
        "Description": "RestApi deployment id: 1e98ccf811ae5bf67840ade2be6857ddf75298a9",
        "RestApiId": {
          "Ref": "MyApi"
        },
        "StageName": "Stage"
      }
    },
    "MyApi": {
      "Type": "AWS::ApiGateway::RestApi",
      "Metadata": {
        "SamResourceId": "MyApi"
      },
      "Properties": {
        "Body": {
          "info": {
            "version": "1.0",
            "title": {
              "Ref": "AWS::StackName"
            }
          },
          "paths": {
            "/": {
              "get": {
                "x-amazon-apigateway-integration": {
                  "httpMethod": "POST",
                  "type": "aws_proxy",
                  "uri": {
                    "Fn::Sub": "arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${MyFunction.Arn}/invocations"
                  }
                },
                "responses": {}
              }
            }
          },
          "swagger": "2.0"
        }
      }
    }
  }
}

The transformed template shows that MyApi (the transformed AWS::Serverless::Api) includes the reference to the Lambda function:

          "paths": {
            "/": {
              "get": {
                "x-amazon-apigateway-integration": {
                  "httpMethod": "POST",
                  "type": "aws_proxy",
                  "uri": {
                    "Fn::Sub": "arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${MyFunction.Arn}/invocations"
                  }
                },
                "responses": {}
              }
            }
          },

The API definition of the AWS::Serverless::Api is mutated here:

serverless-application-model/samtranslator/model/eventsources/push.py

Line 946 in a82a19f

api["DefinitionBody"] = editor.swagger

And here:

serverless-application-model/samtranslator/model/stepfunctions/events.py

Line 458 in a82a19f

api["DefinitionBody"] = editor.swagger

But since the API resource is not in the same template as where the event is defined, the SAM transform is unable to edit the API definition. Which is the current technical limitation described in the issue.

Any plans to fix this? It will be 5 years soon

Navigation Menu

Repository navigation

Possibility to get AWS::Serverless::RestApi passed a template variable to Template files with Functions only #349

56 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Possibility to get AWS::Serverless::RestApi passed a template variable to Template files with Functions only #349

Description

Activity

honglu commented on Mar 29, 2018

ashwgupt commented on Apr 1, 2018

lestephane commented on Apr 22, 2018

saratitan commented on May 14, 2018

mjbuonaccorsi commented on Oct 11, 2018

dehli commented on Jan 9, 2019

keetonian commented on Jan 11, 2019

dehli commented on Jan 11, 2019

keetonian commented on Jan 11, 2019

56 remaining items

omardoma commented on Nov 6, 2022

allanchua101 commented on Nov 7, 2022

siddharthmudgal commented on Nov 9, 2022

awood45 commented on Dec 2, 2022

troycampano commented on Dec 7, 2022

hoffa commented on Dec 8, 2022

wludwiniak commented on Dec 8, 2022

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions