APIGW: Template format error with resources exceeds error though using nested stack #7391

ssrikanta · 2020-04-16T18:52:13Z

Template format error is thrown though nested stacks are used.
Using single stack was throwing template format error with resource exceeds 200, hence started with nested stack, but issue still the same.

Reproduction Steps

Use case have 3 lambda functions, with 70 api with different method and resource in a single RestAPI.
it has been split into 3 nested stack for each with lambda and its associated API.

class StackOne(aws_cloudformation.NestedStack):
    def __init__(self, scope: core.Construct, id: str,  **kwargs) -> None:
        super().__init__(scope, id, **kwargs)
        funcOne = aws_lambda.Function(self, func_suffix, handler='handler',
                                      function_name='name', role=aws_iam.Role.from_role_arn(self,'role', role),
                                      runtime=aws_lambda.Runtime.JAVA_8, code=aws_lambda.S3Code(bucket, 'path'))
        request_template = {"name": "value"}
        integration_responses = [{'statusCode': '200',
                                  'responseParameters': {'method.response.header.Access-Control-Allow-Origin': "'*'"}}]
        method_responses = [{'statusCode': '200',
                             'responseParameters': {'method.response.header.Access-Control-Allow-Origin': True}}]
        stage_options = aws_apigateway.StageOptions(stage_name='name',variables=dict(key='value'))
        self.rest_api = aws_apigateway.RestApi(self, "gateway", rest_api_name="gateway", deploy_options=stage_options)
		api_entity = rest_api.root.add_resource('api')
        api_integ = aws_apigateway.LambdaIntegration(funcOne, proxy=False, request_templates={'application/json':json.dumps(cap_activate_factor_tempate)},
                                                     integration_responses=integration_responses, allow_test_invoke=False,
                                                     passthrough_behavior=aws_apigateway.PassthroughBehavior.WHEN_NO_TEMPLATES)
        api_entity.add_method('POST', api_integ, method_responses=method_responses)


class StackTwo(aws_cloudformation.NestedStack):
    def __init__(self, scope: core.Construct, id: str,  rest_api:aws_apigateway.RestApi, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)
        funcTwo = aws_lambda.Function(self, func_suffix, handler='handler',
                                      function_name='name', role=aws_iam.Role.from_role_arn(self,'role', role),
                                      runtime=aws_lambda.Runtime.JAVA_8, code=aws_lambda.S3Code(bucket, 'path'))
        request_template = {"name": "value"}
        integration_responses = [{'statusCode': '200',
                                  'responseParameters': {'method.response.header.Access-Control-Allow-Origin': "'*'"}}]
        method_responses = [{'statusCode': '200',
                             'responseParameters': {'method.response.header.Access-Control-Allow-Origin': True}}]
        api_entity = rest_api.root.get_resource('api')
        v1_entity = api_entity.add_resource('v1')
        v1_integ = aws_apigateway.LambdaIntegration(funcTwo, proxy=False, request_templates={'application/json':json.dumps(cap_activate_factor_tempate)},
                                                     integration_responses=integration_responses, allow_test_invoke=False,
                                                     passthrough_behavior=aws_apigateway.PassthroughBehavior.WHEN_NO_TEMPLATES)
        v1_entity.add_method('POST', v1_integ, method_responses=method_responses)


class MainStack(core.Stack):
    def __init__(self, scope: core.Construct, id: str, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)

		stack_one = StackOne(self, 'StackOne')
		StackTwo(self, 'StackTwo', rest_api=stack_one.rest_api)


app = core.App()
MainStack(app, 'MainStack', env={'region':'us-east-1'})

Error Log

Template format error: Number of resources, 204, is greater than maximum allowed, 200

Environment

CLI Version : 1.31.0
Framework Version:
OS : Windows
Language : Python3.7

Other

This is 🐛 Bug Report

eladb · 2020-04-21T07:58:49Z

Can you please paste your deployment log with the error?

Also, I am curious what’s the size of the JSON templates (both nested and top-level) in your cdk.out directory? How many resources each one has?

A quick bash command that will return the number of resources in a template:

node -p “Object.keys(require('./cdk.out/xxxx.json').Resources).length)”

ssrikanta · 2020-04-23T04:52:49Z

Main stack has 2 resources, StackOne(First nested stack) has resources 204, ideally this should be 196 stacks and StackTwo (second nested stack) has 0 resouces, ideally this should have 8 resources.
Resources from StackTwo are available in StackOne itself.
Is something wrong the way i create stack?
Error messages as below.

 0/4 | 9:47:31 AM | CREATE_IN_PROGRESS   | AWS::CloudFormation::Stack | odm-gateway-api-stack User Initiated
 0/4 | 9:48:00 AM | CREATE_IN_PROGRESS   | AWS::CDK::Metadata         | CDKMetadata
 0/4 | 9:48:01 AM | CREATE_IN_PROGRESS   | AWS::CloudFormation::Stack | GatewayNestedStackOne.NestedStack/GatewayNestedStackOne.NestedStackResource (GatewayNestedStackOneNestedStackGatewayNestedStackOneNestedStackResourceED75366C)
 0/4 | 9:48:01 AM | CREATE_IN_PROGRESS   | AWS::CloudFormation::Stack | StackTwo.NestedStack/StackTwo.NestedStackResource (StackTwoNestedStackStackTwoNestedStackResource974D9BA8)
 1/4 | 9:48:01 AM | CREATE_FAILED        | AWS::CloudFormation::Stack | StackTwo.NestedStack/StackTwo.NestedStackResource (StackTwoNestedStackStackTwoNestedStackResource974D9BA8) Template format error: At least one Resources member must be defined.
        new NestedStack (C:\Users\501300~1\AppData\Local\Temp\jsii-kernel-7yi8r1\node_modules\@aws-cdk\aws-cloudformation\lib\nested-stack.js:34:25)
        \_ obj._wrapSandboxCode (D:\Python37\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:7836:49)
        \_ Kernel._wrapSandboxCode (D:\Python37\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:8296:20)
        \_ Kernel._create (D:\Python37\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:7836:26)
        \_ Kernel.create (D:\Python37\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:7583:21)
        \_ KernelHost.processRequest (D:\Python37\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:7371:28)
        \_ KernelHost.run (D:\Python37\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:7311:14)
        \_ Immediate.setImmediate [as _onImmediate] (D:\Python37\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:7314:37)
        \_ runCallback (timers.js:705:18)
        \_ tryOnImmediate (timers.js:676:5)
        \_ processImmediate (timers.js:658:5)
 2/4 | 9:48:02 AM | CREATE_FAILED        | AWS::CloudFormation::Stack | GatewayNestedStackOne.NestedStack/GatewayNestedStackOne.NestedStackResource (GatewayNestedStackOneNestedStackGatewayNestedStackOneNestedStackResourceED75366C) Template format error: Number of resources, 204, is greater than maximum allowed, 200
        new NestedStack (C:\Users\501300~1\AppData\Local\Temp\jsii-kernel-7yi8r1\node_modules\@aws-cdk\aws-cloudformation\lib\nested-stack.js:34:25)
        \_ obj._wrapSandboxCode (D:\Python37\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:7836:49)
        \_ Kernel._wrapSandboxCode (D:\Python37\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:8296:20)
        \_ Kernel._create (D:\Python37\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:7836:26)
        \_ Kernel.create (D:\Python37\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:7583:21)
        \_ KernelHost.processRequest (D:\Python37\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:7371:28)
        \_ KernelHost.run (D:\Python37\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:7311:14)
        \_ Immediate.setImmediate [as _onImmediate] (D:\Python37\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:7314:37)
        \_ runCallback (timers.js:705:18)
        \_ tryOnImmediate (timers.js:676:5)
        \_ processImmediate (timers.js:658:5)
 3/4 | 9:48:03 AM | CREATE_FAILED        | AWS::CDK::Metadata         | CDKMetadata Resource creation cancelled
 3/4 | 9:48:04 AM | ROLLBACK_IN_PROGRESS | AWS::CloudFormation::Stack | odm-gateway-api-stack The following resource(s) failed to create: [StackTwoNestedStackStackTwoNestedStackResource974D9BA8, GatewayNestedStackOneNestedStackGatewayNestedStackOneNestedStackResourceED75366C, CDKMetadata]. . Rollback requested by user.
 4/4 | 9:48:08 AM | DELETE_COMPLETE      | AWS::CloudFormation::Stack | GatewayNestedStackOne.NestedStack/GatewayNestedStackOne.NestedStackResource (GatewayNestedStackOneNestedStackGatewayNestedStackOneNestedStackResourceED75366C)
 5/4 | 9:48:08 AM | DELETE_COMPLETE      | AWS::CloudFormation::Stack | StackTwo.NestedStack/StackTwo.NestedStackResource (StackTwoNestedStackStackTwoNestedStackResource974D9BA8)
 5/4 | 9:48:08 AM | DELETE_IN_PROGRESS   | AWS::CDK::Metadata         | CDKMetadata
 6/4 | 9:48:10 AM | DELETE_COMPLETE      | AWS::CDK::Metadata         | CDKMetadata
 7/4 | 9:48:11 AM | ROLLBACK_COMPLETE    | AWS::CloudFormation::Stack | odm-gateway-api-stack

eladb · 2020-04-23T05:56:29Z

I believe the issue here is that the API resources of APIGW are always defined in the same stack as the REST API resource itself. So even if you call addResource from the context of the second stack, the APIGW resource itself will be defined in the first stack.

What I would recommend is to split things up a bit differently: put both Lambda functions in one nested stack (or two) and the API definitions in the other.

Let me know how it goes.

eladb · 2020-04-23T05:57:07Z

This is actually more of an APIGW guidance issue then a CLI bug. Reassigning/labeling.

ssrikanta · 2020-04-23T17:26:02Z

@eladb API gateway stack resource has 204, hence it can be alone in a single stack. but tried other ways as well, in all cases, API gateway is coming under a single stack. and exceeding the count 200

eladb · 2020-04-23T17:32:01Z

@eladb API gateway stack resource has 204, hence it can be alone in a single stack. but tried other ways as well, in all cases, API gateway is coming under a single stack. and exceeding the count 200

Is it 204 resources even after you move away everything else like the lambda functions?

ssrikanta · 2020-04-23T18:55:14Z

Each Lambda has around 6 resources
Api gateway template has around 204 resources.
Case1: All Lambda in nested stackOne,
partial api gateway in nested stackTwo and partial gateway in nested stackThree, stackOne has 12 resource, stackTwo has 204 resource and stackThree has no resource.
Case2: stackOne has one lambda and partial api gateway, stackTwo has one lambda and partial API gateway, on this case stackOne has around 210 resources and stackTwo has around 6 resources

The apigateway CDK construct library creates a large number of CloudFormation resources. To define a single Method that is backed with a Lambda function and authorizer with an Authorizer, ~8 CloudFormation resources are created. This quickly grows and a reasonable sized RestApi is bound to hit the 200 resource limit on their stack. Re-organizing the CDK app into multiple `Stack`s that share the same instance of `RestApi` will not resolve this problem, since the CDK still makes an executive decision to keep the Methods and Resources in the same stack as the `RestApi` construct. (see #7391) This change introduces a new import style API `fromRestApiAttributes()` that returns an instance of `IRestApi` that allows for new Resources to be defined across stacks. As a nice side effect, this change also adds the ability to define Resources on SpecRestApi in addition to what has already been defined in the OpenAPI spec. closes #1477

The apigateway CDK construct library creates a large number of CloudFormation resources. To define a single Method that is backed with a Lambda function and authorizer with an Authorizer, ~8 CloudFormation resources are created. This quickly grows and a reasonable sized RestApi is bound to hit the 200 resource limit on their stack. Re-organizing the CDK app into multiple `Stack`s that share the same instance of `RestApi` will not resolve this problem, since the CDK still makes an executive decision to keep the Methods and Resources in the same stack as the `RestApi` construct. (see #7391) This change introduces a new import style API `fromRestApiAttributes()` that returns an instance of `IRestApi` that allows for new Resources to be defined across stacks. As a nice side effect, this change also adds the ability to define Resources on SpecRestApi in addition to what has already been defined in the OpenAPI spec. closes #1477 closes #7391 fixes #8347 ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*

ssrikanta added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Apr 16, 2020

SomayaB assigned eladb and shivlaks and unassigned eladb Apr 20, 2020

SomayaB added the package/tools Related to AWS CDK Tools or CLI label Apr 20, 2020

eladb changed the title ~~Template format error with resources exceeds error though using nested stack~~ APIGW: Template format error with resources exceeds error though using nested stack Apr 23, 2020

eladb added @aws-cdk/aws-apigateway Related to Amazon API Gateway guidance Question that needs advice or information. and removed bug This issue is a bug. package/tools Related to AWS CDK Tools or CLI labels Apr 23, 2020

eladb assigned nija-at and unassigned shivlaks Apr 23, 2020

SomayaB removed the needs-triage This issue or PR still needs to be triaged. label May 19, 2020

nija-at mentioned this issue May 29, 2020

feat(apigateway): define Resources on imported RestApi #8270

Merged

mergify bot closed this as completed in #8270 Jun 12, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

APIGW: Template format error with resources exceeds error though using nested stack #7391

APIGW: Template format error with resources exceeds error though using nested stack #7391

ssrikanta commented Apr 16, 2020 •

edited by eladb

Loading

eladb commented Apr 21, 2020

ssrikanta commented Apr 23, 2020

eladb commented Apr 23, 2020

eladb commented Apr 23, 2020 •

edited

Loading

ssrikanta commented Apr 23, 2020

eladb commented Apr 23, 2020

ssrikanta commented Apr 23, 2020

APIGW: Template format error with resources exceeds error though using nested stack #7391

APIGW: Template format error with resources exceeds error though using nested stack #7391

Comments

ssrikanta commented Apr 16, 2020 • edited by eladb Loading

Reproduction Steps

Error Log

Environment

Other

eladb commented Apr 21, 2020

ssrikanta commented Apr 23, 2020

eladb commented Apr 23, 2020

eladb commented Apr 23, 2020 • edited Loading

ssrikanta commented Apr 23, 2020

eladb commented Apr 23, 2020

ssrikanta commented Apr 23, 2020

ssrikanta commented Apr 16, 2020 •

edited by eladb

Loading

eladb commented Apr 23, 2020 •

edited

Loading