Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

latest cdklocal/localstack fails to bootstrap #36

Closed
guoliang opened this issue Dec 11, 2020 · 27 comments
Closed

latest cdklocal/localstack fails to bootstrap #36

guoliang opened this issue Dec 11, 2020 · 27 comments

Comments

@guoliang
Copy link

guoliang commented Dec 11, 2020

I upgraded cdklocal and localstack

I'm running localstack on Windows 10 . Below are the versions of the tools

$ cdklocal --version
1.77.0 (build a941c53)
$ docker images | grep localstack
localstack/localstack                  latest                  25b151909437        58 minutes ago      638MB
$ localstack --version
0.12.2

I have started localstack by running

$ localstack start

Then when I run

$ cdklocal bootstrap

I get the following errors

 ⏳  Bootstrapping environment aws://000000000000/eu-central-1...
CDKToolkit: creating CloudFormation changeset...
 0/3 | 14:00:48 | REVIEW_IN_PROGRESS   | AWS::CloudFormation::Stack | CDKToolkit User Initiated
 0/3 | 14:00:48 | CREATE_IN_PROGRESS   | AWS::CloudFormation::Stack | CDKToolkit 
 0/3 | 14:00:48 | UPDATE_IN_PROGRESS   | AWS::CloudFormation::Stack | CDKToolkit
 0/3 | 14:00:48 | UPDATE_COMPLETE      | AWS::CloudFormation::Stack | CDKToolkit
 0/3 | 14:00:48 | CREATE_COMPLETE      | AWS::CloudFormation::Stack | CDKToolkit 
 0/3 | 14:00:48 | CREATE_COMPLETE      | AWS::CloudFormation::Stack | CDKToolkit
 0/3 | 14:00:48 | CREATE_FAILED        | AWS::CloudFormation::Stack | CDKToolkit
 ❌  Environment aws://000000000000/eu-central-1 failed bootstrapping: Error: The stack named CDKToolkit failed to deploy: CREATE_FAILED
    at Object.waitForStackDeploy (C:\Users\lwong1\.npm-packages\node_modules\aws-cdk\lib\api\util\cloudformation.ts:307:11)
    at processTicksAndRejections (node:internal/process/task_queues:93:5)
    at Object.deployStack (C:\Users\lwong1\.npm-packages\node_modules\aws-cdk\lib\api\deploy-stack.ts:283:26)
    at C:\Users\lwong1\.npm-packages\node_modules\aws-cdk\lib\cdk-toolkit.ts:358:24
    at async Promise.all (index 0)
    at CdkToolkit.bootstrap (C:\Users\lwong1\.npm-packages\node_modules\aws-cdk\lib\cdk-toolkit.ts:355:5)
    at initCommandLine (C:\Users\lwong1\.npm-packages\node_modules\aws-cdk\bin\cdk.ts:201:9)
The stack named CDKToolkit failed to deploy: CREATE_FAILED

Prior to upgrading localstack it was still working.
But perhaps this issue has the same cause as #5 ?

@guoliang
Copy link
Author

I re-ran with DEBUG=1 on, and I got the following log in localstack after I run cdklocal bootstrap

2020-12-11T13:42:27:INFO:localstack.utils.analytics.profiler: Execution of "start_api_services" took 13678.445100784302ms
2020-12-11T13:42:43:DEBUG:localstack.services.cloudformation.cloudformation_listener: Error response for CloudFormation action "DescribeStacks" (400) POST /: b'<ErrorResponse xmlns="http://cloudformation.amazonaws.com/doc/2010-05-15/">\n  <Error>\n    <Type>Sender</Type>\n    <Code>ValidationError</Code>\n    <Message>Stack with id LocalcdkTestStack does not exist</Message>\n  </Error>\n  <RequestId>cf4c737e-5ae2-11e4-a7c9-ad44eEXAMPLE</RequestId>\n</ErrorResponse>'
2020-12-11T13:43:27:DEBUG:localstack.services.cloudformation.cloudformation_listener: Error response for CloudFormation action "DescribeStacks" (400) POST /: b'<ErrorResponse xmlns="http://cloudformation.amazonaws.com/doc/2010-05-15/">\n  <Error>\n    <Type>Sender</Type>\n    <Code>ValidationError</Code>\n    <Message>Stack with id CDKToolkit does not exist</Message>\n  </Error>\n  <RequestId>cf4c737e-5ae2-11e4-a7c9-ad44eEXAMPLE</RequestId>\n</ErrorResponse>'
2020-12-11T13:43:27:DEBUG:localstack.services.cloudformation.cloudformation_listener: Error response for CloudFormation action "DescribeStacks" (400) POST /: b'<ErrorResponse xmlns="http://cloudformation.amazonaws.com/doc/2010-05-15/">\n  <Error>\n    <Type>Sender</Type>\n    <Code>ValidationError</Code>\n    <Message>Stack with id CDKToolkit does not exist</Message>\n  </Error>\n  <RequestId>cf4c737e-5ae2-11e4-a7c9-ad44eEXAMPLE</RequestId>\n</ErrorResponse>'
2020-12-11T13:43:28:DEBUG:localstack.services.cloudformation.cloudformation_starter: Currently processing (update=False) resource CDKToolkit/StagingBucket: None
2020-12-11T13:43:28:DEBUG:localstack.services.cloudformation.cloudformation_starter: Deploying CloudFormation resource (update=False, exists=False, updateable=False): {'Type': 'AWS::S3::Bucket', 'Properties': {'AccessControl': 'Private', 'BucketEncryption': {'ServerSideEncryptionConfiguration': [{'ServerSideEncryptionByDefault': {'SSEAlgorithm': 'aws:kms'}}]}, 'PublicAccessBlockConfiguration': {'BlockPublicAcls': True, 'BlockPublicPolicy': True, 'IgnorePublicAcls': True, 'RestrictPublicBuckets': True}, 'BucketName': 'cdktoolkit-stagingbucket-a8243204'}}
2020-12-11T13:43:28:DEBUG:localstack.utils.cloudformation.template_deployer: Running action "create" for resource type "S3::Bucket" id "StagingBucket"
2020-12-11T13:43:28:DEBUG:localstack.utils.cloudformation.template_deployer: Request for resource type "S3::Bucket" in region us-east-1: create_bucket {'Bucket': 'cdktoolkit-stagingbucket-a8243204', 'ACL': 'private'}
2020-12-11T13:43:28:DEBUG:localstack.services.cloudformation.cloudformation_starter: Currently processing (update=False) resource CDKToolkit/StagingBucketPolicy: None
2020-12-11T13:43:28:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: S3::Bucket PhysicalResourceId
2020-12-11T13:43:28:WARNING:localstack.services.cloudformation.cloudformation_starter: Unable to determine physical_resource_id for resource <class 'localstack.services.cloudformation.service_models.S3BucketPolicy'>
2020-12-11T13:43:28:DEBUG:localstack.services.cloudformation.cloudformation_starter: Deploying CloudFormation resource (update=False, exists=False, updateable=False): {'Type': 'AWS::S3::BucketPolicy', 'Properties': {'Bucket': 'cdktoolkit-stagingbucket-a8243204', 'PolicyDocument': {'Id': 'AccessControl', 'Version': '2012-10-17', 'Statement': [{'Sid': 'AllowSSLRequestsOnly', 'Action': 's3:*', 'Effect': 'Deny', 'Resource': ['arn:aws:s3:::cdktoolkit-stagingbucket-a8243204', 'arn:aws:s3:::cdktoolkit-stagingbucket-a8243204/*'], 'Condition': {'Bool': {'aws:SecureTransport': 'false'}}, 'Principal': '*'}]}}}
2020-12-11T13:43:28:DEBUG:localstack.utils.cloudformation.template_deployer: Running action "create" for resource type "S3::BucketPolicy" id "StagingBucketPolicy"
2020-12-11T13:43:28:DEBUG:localstack.utils.cloudformation.template_deployer: Request for resource type "S3::BucketPolicy" in region us-east-1: put_bucket_policy {'Bucket': 'cdktoolkit-stagingbucket-a8243204', 'Policy': '{"Id": "AccessControl", "Version": "2012-10-17", "Statement": [{"Sid": "AllowSSLRequestsOnly", "Action": "s3:*", "Effect": "Deny", "Resource": ["arn:aws:s3:::cdktoolkit-stagingbucket-a8243204", "arn:aws:s3:::cdktoolkit-stagingbucket-a8243204/*"], "Condition": {"Bool": {"aws:SecureTransport": "false"}}, "Principal": "*"}]}'}
2020-12-11T13:43:28:WARNING:localstack.services.cloudformation.cloudformation_starter: Unable to determine physical_resource_id for resource <class 'localstack.services.cloudformation.service_models.S3BucketPolicy'>
2020-12-11T13:43:28:DEBUG:localstack.services.cloudformation.cloudformation_starter: Running CloudFormation stack deployment loop iteration 1
2020-12-11T13:43:28:DEBUG:localstack.services.cloudformation.cloudformation_starter: Found 0 unresolved dependencies in loop iteration 1
2020-12-11T13:43:28:INFO:localstack.services.cloudformation.cloudformation_starter: Error running stack deployment loop: 'CloudFormationBackend' object has no attribute 'set_exports'
2020-12-11T13:43:28:WARNING:bootstrap.py: Thread run method <function apply_patches.<locals>.run_dependencies_deployment_loop.<locals>.run_loop at 0x7f269c033b80>(None) failed: Unable to resolve all CloudFormation resources after traversing dependency tree (maximum depth 40): ['StagingBucket', 'StagingBucketPolicy'] Traceback (most recent call last):
  File "/opt/code/localstack/localstack/utils/bootstrap.py", line 581, in run
    result = self.func(self.params)
  File "/opt/code/localstack/localstack/services/cloudformation/cloudformation_starter.py", line 1210, in run_loop
    raise Exception('Unable to resolve all CloudFormation resources after traversing ' +
Exception: Unable to resolve all CloudFormation resources after traversing dependency tree (maximum depth 40): ['StagingBucket', 'StagingBucketPolicy']

@guoliang
Copy link
Author

BTW, I noticed that when I start localstack in the log it has configured my region to be us-east-1

So if I run

cdklocal bootstrap aws://000000000000/us-east-1

 ⏳  Bootstrapping environment aws://000000000000/us-east-1...
CDKToolkit: creating CloudFormation changeset...
 0/3 | 14:46:34 | REVIEW_IN_PROGRESS   | AWS::CloudFormation::Stack | CDKToolkit User Initiated
 0/3 | 14:46:34 | CREATE_IN_PROGRESS   | AWS::CloudFormation::Stack | CDKToolkit 
 0/3 | 14:46:34 | UPDATE_IN_PROGRESS   | AWS::CloudFormation::Stack | CDKToolkit 
 0/3 | 14:46:34 | UPDATE_COMPLETE      | AWS::CloudFormation::Stack | CDKToolkit 
 0/3 | 14:46:35 | CREATE_COMPLETE      | AWS::CloudFormation::Stack | CDKToolkit 
 0/3 | 14:46:35 | CREATE_FAILED        | AWS::CloudFormation::Stack | CDKToolkit 
 0/3 | 14:46:35 | CREATE_COMPLETE      | AWS::CloudFormation::Stack | CDKToolkit 
 ✅  Environment aws://000000000000/us-east-1 bootstrapped.

Looks like it succeeded, even though it got one CREATE_FAILED

Here's the localstack logs for it too

2020-12-11T13:45:05:INFO:localstack.utils.analytics.profiler: Execution of "start_api_services" took 11115.358114242554ms
2020-12-11T13:46:34:DEBUG:localstack.services.cloudformation.cloudformation_listener: Error response for CloudFormation action "DescribeStacks" (400) POST /: b'<ErrorResponse xmlns="http://cloudformation.amazonaws.com/doc/2010-05-15/">\n  <Error>\n    <Type>Sender</Type>\n    <Code>ValidationError</Code>\n    <Message>Stack with id CDKToolkit does not exist</Message>\n  </Error>\n  <RequestId>cf4c737e-5ae2-11e4-a7c9-ad44eEXAMPLE</RequestId>\n</ErrorResponse>'
2020-12-11T13:46:34:DEBUG:localstack.services.cloudformation.cloudformation_listener: Error response for CloudFormation action "DescribeStacks" (400) POST /: b'<ErrorResponse xmlns="http://cloudformation.amazonaws.com/doc/2010-05-15/">\n  <Error>\n    <Type>Sender</Type>\n    <Code>ValidationError</Code>\n    <Message>Stack with id CDKToolkit does not exist</Message>\n  </Error>\n  <RequestId>cf4c737e-5ae2-11e4-a7c9-ad44eEXAMPLE</RequestId>\n</ErrorResponse>'
2020-12-11T13:46:34:DEBUG:localstack.services.cloudformation.cloudformation_starter: Currently processing (update=False) resource CDKToolkit/StagingBucket: None
2020-12-11T13:46:34:DEBUG:localstack.services.cloudformation.cloudformation_starter: Deploying CloudFormation resource (update=False, exists=False, updateable=False): {'Type': 'AWS::S3::Bucket', 'Properties': {'AccessControl': 'Private', 'BucketEncryption': {'ServerSideEncryptionConfiguration': [{'ServerSideEncryptionByDefault': {'SSEAlgorithm': 'aws:kms'}}]}, 'PublicAccessBlockConfiguration': {'BlockPublicAcls': True, 'BlockPublicPolicy': True, 'IgnorePublicAcls': True, 'RestrictPublicBuckets': True}, 'BucketName': 'cdktoolkit-stagingbucket-76f60428'}}
2020-12-11T13:46:34:DEBUG:localstack.utils.cloudformation.template_deployer: Running action "create" for resource type "S3::Bucket" id "StagingBucket"
2020-12-11T13:46:34:DEBUG:localstack.utils.cloudformation.template_deployer: Request for resource type "S3::Bucket" in region us-east-1: create_bucket {'Bucket': 'cdktoolkit-stagingbucket-76f60428', 'ACL': 'private'}
2020-12-11T13:46:34:DEBUG:localstack.services.cloudformation.cloudformation_starter: Currently processing (update=False) resource CDKToolkit/StagingBucketPolicy: None
2020-12-11T13:46:34:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: S3::Bucket PhysicalResourceId
2020-12-11T13:46:34:WARNING:localstack.services.cloudformation.cloudformation_starter: Unable to determine physical_resource_id for resource <class 'localstack.services.cloudformation.service_models.S3BucketPolicy'>
2020-12-11T13:46:34:DEBUG:localstack.services.cloudformation.cloudformation_starter: Deploying CloudFormation resource (update=False, exists=False, updateable=False): {'Type': 'AWS::S3::BucketPolicy', 'Properties': {'Bucket': 'cdktoolkit-stagingbucket-76f60428', 'PolicyDocument': {'Id': 'AccessControl', 'Version': '2012-10-17', 'Statement': [{'Sid': 'AllowSSLRequestsOnly', 'Action': 's3:*', 'Effect': 'Deny', 'Resource': ['arn:aws:s3:::cdktoolkit-stagingbucket-76f60428', 'arn:aws:s3:::cdktoolkit-stagingbucket-76f60428/*'], 'Condition': {'Bool': {'aws:SecureTransport': 'false'}}, 'Principal': '*'}]}}}
2020-12-11T13:46:34:DEBUG:localstack.utils.cloudformation.template_deployer: Running action "create" for resource type "S3::BucketPolicy" id "StagingBucketPolicy"
2020-12-11T13:46:34:DEBUG:localstack.utils.cloudformation.template_deployer: Request for resource type "S3::BucketPolicy" in region us-east-1: put_bucket_policy {'Bucket': 'cdktoolkit-stagingbucket-76f60428', 'Policy': '{"Id": "AccessControl", "Version": "2012-10-17", "Statement": [{"Sid": "AllowSSLRequestsOnly", "Action": "s3:*", "Effect": "Deny", "Resource": ["arn:aws:s3:::cdktoolkit-stagingbucket-76f60428", "arn:aws:s3:::cdktoolkit-stagingbucket-76f60428/*"], "Condition": {"Bool": {"aws:SecureTransport": "false"}}, "Principal": "*"}]}'}
2020-12-11T13:46:35:WARNING:localstack.services.cloudformation.cloudformation_starter: Unable to determine physical_resource_id for resource <class 'localstack.services.cloudformation.service_models.S3BucketPolicy'>
2020-12-11T13:46:35:DEBUG:localstack.services.cloudformation.cloudformation_starter: Running CloudFormation stack deployment loop iteration 1
2020-12-11T13:46:35:DEBUG:localstack.services.cloudformation.cloudformation_starter: Found 0 unresolved dependencies in loop iteration 1
2020-12-11T13:46:35:INFO:localstack.services.cloudformation.cloudformation_starter: Error running stack deployment loop: 'CloudFormationBackend' object has no attribute 'set_exports'
2020-12-11T13:46:35:WARNING:bootstrap.py: Thread run method <function apply_patches.<locals>.run_dependencies_deployment_loop.<locals>.run_loop at 0x7f52315d0f70>(None) failed: Unable to resolve all CloudFormation resources after traversing dependency tree (maximum depth 40): ['StagingBucket', 'StagingBucketPolicy'] Traceback (most recent call last):
  File "/opt/code/localstack/localstack/utils/bootstrap.py", line 581, in run
    result = self.func(self.params)
  File "/opt/code/localstack/localstack/services/cloudformation/cloudformation_starter.py", line 1210, in run_loop
    raise Exception('Unable to resolve all CloudFormation resources after traversing ' +
Exception: Unable to resolve all CloudFormation resources after traversing dependency tree (maximum depth 40): ['StagingBucket', 'StagingBucketPolicy']

@guoliang
Copy link
Author

When I go ahead and run

$ cdklocal deploy
...
...
Do you wish to deploy these changes (y/n)? y
LocalcdkTestStack: deploying...
[0%] start: Publishing 804d69e29081cd169ea6bb165c6275208b2a223191337de83f3c2eb0e899a6c8:current
[100%] success: Published 804d69e29081cd169ea6bb165c6275208b2a223191337de83f3c2eb0e899a6c8:current
LocalcdkTestStack: creating CloudFormation changeset...
  0/13 | 14:51:20 | REVIEW_IN_PROGRESS   | AWS::CloudFormation::Stack  | LocalcdkTestStack User Initiated
  0/13 | 14:51:20 | CREATE_IN_PROGRESS   | AWS::CloudFormation::Stack  | LocalcdkTestStack 
  0/13 | 14:51:20 | UPDATE_IN_PROGRESS   | AWS::CloudFormation::Stack  | LocalcdkTestStack 
  0/13 | 14:51:20 | UPDATE_COMPLETE      | AWS::CloudFormation::Stack  | LocalcdkTestStack 
  0/13 | 14:51:22 | CREATE_COMPLETE      | AWS::CloudFormation::Stack  | LocalcdkTestStack 
  0/13 | 14:51:25 | CREATE_COMPLETE      | AWS::CloudFormation::Stack  | LocalcdkTestStack 
  0/13 | 14:51:25 | CREATE_FAILED        | AWS::CloudFormation::Stack  | LocalcdkTestStack 

 ❌  LocalcdkTestStack failed: Error: The stack named LocalcdkTestStack failed to deploy: CREATE_FAILED
    at Object.waitForStackDeploy (C:\Users\lwong1\.npm-packages\node_modules\aws-cdk\lib\api\util\cloudformation.ts:307:11)
    at processTicksAndRejections (node:internal/process/task_queues:93:5)
    at Object.deployStack (C:\Users\lwong1\.npm-packages\node_modules\aws-cdk\lib\api\deploy-stack.ts:283:26)
    at CdkToolkit.deploy (C:\Users\lwong1\.npm-packages\node_modules\aws-cdk\lib\cdk-toolkit.ts:180:24)
    at initCommandLine (C:\Users\lwong1\.npm-packages\node_modules\aws-cdk\bin\cdk.ts:201:9)
The stack named LocalcdkTestStack failed to deploy: CREATE_FAILED

it failed and in the localstack log

2020-12-11T13:51:10:DEBUG:localstack.services.cloudformation.cloudformation_listener: Error response for CloudFormation action "DescribeStacks" (400) POST /: b'<ErrorResponse xmlns="http://cloudformation.amazonaws.com/doc/2010-05-15/">\n  <Error>\n    <Type>Sender</Type>\n    <Code>ValidationError</Code>\n    <Message>Stack with id LocalcdkTestStack does not exist</Message>\n  </Error>\n  <RequestId>cf4c737e-5ae2-11e4-a7c9-ad44eEXAMPLE</RequestId>\n</ErrorResponse>'
2020-12-11T13:51:19:DEBUG:localstack.services.cloudformation.cloudformation_listener: Error response for CloudFormation action "DescribeStacks" (400) POST /: b'<ErrorResponse xmlns="http://cloudformation.amazonaws.com/doc/2010-05-15/">\n  <Error>\n    <Type>Sender</Type>\n    <Code>ValidationError</Code>\n    <Message>Stack with id LocalcdkTestStack does not exist</Message>\n  </Error>\n  <RequestId>cf4c737e-5ae2-11e4-a7c9-ad44eEXAMPLE</RequestId>\n</ErrorResponse>'
2020-12-11T13:51:20:DEBUG:localstack.services.cloudformation.cloudformation_starter: Currently processing (update=False) resource LocalcdkTestStack/DynamoLambdaHandlerServiceRole4C867B01: None
2020-12-11T13:51:20:DEBUG:localstack.services.cloudformation.cloudformation_starter: Resource DynamoLambdaHandlerServiceRole4C867B01 need not be deployed (is_updateable=False): {'Type': 'AWS::IAM::Role', 'Properties': {'AssumeRolePolicyDocument': {'Statement': [{'Action': 'sts:AssumeRole', 'Effect': 'Allow', 'Principal': {'Service': 'lambda.amazonaws.com'}}], 'Version': '2012-10-17'}, 'ManagedPolicyArns': ['arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole'], 'RoleName': 'cf-LocalcdkTestStack-DynamoLambdaHandlerServiceRole4C867B01', 'Path': '/', 'RoleId': 'mvyvrptv43r7dvl5pyw8', 'Arn': 'arn:aws:iam::000000000000:role/cf-LocalcdkTestStack-DynamoLambdaHandlerServiceRole4C867B01', 'CreateDate': datetime.datetime(2020, 12, 11, 13, 51, 20, 213000, tzinfo=tzlocal()), 'MaxSessionDuration': 3600}, 'Metadata': {'aws:cdk:path': 'LocalcdkTestStack/DynamoLambdaHandler/ServiceRole/Resource'}} True
2020-12-11T13:51:20:DEBUG:localstack.services.cloudformation.cloudformation_starter: Currently processing (update=False) resource LocalcdkTestStack/DynamoLambdaHandlerFB6EB814: None
2020-12-11T13:51:20:INFO:localstack.services.awslambda.lambda_api: Function not found: arn:aws:lambda:us-east-1:000000000000:function:LocalcdkTestStack-lambda-007c23ca
2020-12-11T13:51:20:INFO:localstack.services.awslambda.lambda_api: Function not found: arn:aws:lambda:us-east-1:000000000000:function:LocalcdkTestStack-lambda-007c23ca
2020-12-11T13:51:20:DEBUG:localstack.services.cloudformation.cloudformation_starter: Deploying CloudFormation resource (update=False, exists=False, updateable=False): {'Type': 'AWS::Lambda::Function', 'Properties': {'Code': {'S3Bucket': 'cdktoolkit-stagingbucket-76f60428', 'S3Key': 'assets/804d69e29081cd169ea6bb165c6275208b2a223191337de83f3c2eb0e899a6c8.zip'}, 'Handler': 'function.handler', 'Role': 'arn:aws:iam::000000000000:role/cf-LocalcdkTestStack-DynamoLambdaHandlerServiceRole4C867B01', 'Runtime': 'nodejs12.x', 'FunctionName': 'LocalcdkTestStack-lambda-007c23ca'}, 'DependsOn': ['DynamoLambdaHandlerServiceRole4C867B01'], 'Metadata': {'aws:cdk:path': 'LocalcdkTestStack/DynamoLambdaHandler/Resource', 'aws:asset:path': 'asset.804d69e29081cd169ea6bb165c6275208b2a223191337de83f3c2eb0e899a6c8', 'aws:asset:property': 'Code'}}
2020-12-11T13:51:20:DEBUG:localstack.utils.cloudformation.template_deployer: Running action "create" for resource type "Lambda::Function" id "DynamoLambdaHandlerFB6EB814"
2020-12-11T13:51:20:DEBUG:localstack.utils.cloudformation.template_deployer: Request for resource type "Lambda::Function" in region us-east-1: create_function {'FunctionName': 'LocalcdkTestStack-lambda-007c23ca', 'Runtime': 'nodejs12.x', 'Role': 'arn:aws:iam::000000000000:role/cf-LocalcdkTestStack-DynamoLambdaHandlerServiceRole4C867B01', 'Handler': 'function.handler', 'Code': {'S3Bucket': 'cdktoolkit-stagingbucket-76f60428', 'S3Key': 'assets/804d69e29081cd169ea6bb165c6275208b2a223191337de83f3c2eb0e899a6c8.zip'}}
2020-12-11T13:51:20:DEBUG:localstack.services.cloudformation.cloudformation_starter: Currently processing (update=False) resource LocalcdkTestStack/helloapi4446A35B: None
2020-12-11T13:51:20:DEBUG:localstack.services.cloudformation.cloudformation_starter: Deploying CloudFormation resource (update=False, exists=False, updateable=False): {'Type': 'AWS::ApiGateway::RestApi', 'Properties': {'Name': 'hello-api'}, 'Metadata': {'aws:cdk:path': 'LocalcdkTestStack/hello-api/Resource'}}
2020-12-11T13:51:20:DEBUG:localstack.utils.cloudformation.template_deployer: Running action "create" for resource type "ApiGateway::RestApi" id "helloapi4446A35B"
2020-12-11T13:51:20:DEBUG:localstack.utils.cloudformation.template_deployer: Request for resource type "ApiGateway::RestApi" in region us-east-1: create_rest_api {'name': 'hello-api'}
2020-12-11T13:51:20:DEBUG:localstack.services.cloudformation.cloudformation_starter: Updating resource id: 83b62094 - ggolkkmkui, 83b62094 - {'Type': 'AWS::ApiGateway::RestApi', 'Properties': {'Name': 'hello-api'}, 'Metadata': {'aws:cdk:path': 'LocalcdkTestStack/hello-api/Resource'}}
2020-12-11T13:51:20:INFO:localstack.services.cloudformation.cloudformation_starter: Updating resource ID from 83b62094 to ggolkkmkui (us-east-1)
2020-12-11T13:51:20:DEBUG:localstack.services.cloudformation.cloudformation_starter: Currently processing (update=False) resource LocalcdkTestStack/helloapiCloudWatchRoleD13E913E: None
2020-12-11T13:51:20:DEBUG:localstack.services.cloudformation.cloudformation_starter: Resource helloapiCloudWatchRoleD13E913E need not be deployed (is_updateable=False): {'Type': 'AWS::IAM::Role', 'Properties': {'AssumeRolePolicyDocument': {'Statement': [{'Action': 'sts:AssumeRole', 'Effect': 'Allow', 'Principal': {'Service': 'apigateway.amazonaws.com'}}], 'Version': '2012-10-17'}, 'ManagedPolicyArns': ['arn:aws:iam::aws:policy/service-role/AmazonAPIGatewayPushToCloudWatchLogs'], 'RoleName': 'cf-LocalcdkTestStack-helloapiCloudWatchRoleD13E913E', 'Path': '/', 'RoleId': '9alukkcj8xbctvnyjjlz', 'Arn': 'arn:aws:iam::000000000000:role/cf-LocalcdkTestStack-helloapiCloudWatchRoleD13E913E', 'CreateDate': datetime.datetime(2020, 12, 11, 13, 51, 20, 801000, tzinfo=tzlocal()), 'MaxSessionDuration': 3600}, 'Metadata': {'aws:cdk:path': 'LocalcdkTestStack/hello-api/CloudWatchRole/Resource'}} True
2020-12-11T13:51:20:DEBUG:localstack.services.cloudformation.cloudformation_starter: Currently processing (update=False) resource LocalcdkTestStack/helloapiAccountD8C38BCE: None
2020-12-11T13:51:20:WARNING:moto: No Moto CloudFormation support for AWS::ApiGateway::Account
2020-12-11T13:51:20:DEBUG:localstack.services.cloudformation.cloudformation_starter: Currently processing (update=False) resource LocalcdkTestStack/helloapihelloGETE6A58337: None
2020-12-11T13:51:20:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: ApiGateway::RestApi PhysicalResourceId
2020-12-11T13:51:21:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: ApiGateway::RestApi RootResourceId
2020-12-11T13:51:21:DEBUG:localstack.utils.cloudformation.template_deployer: Dependency for resource helloapihelloGETE6A58337 not yet deployed: helloapihello4AA00177 {'Type': 'AWS::ApiGateway::Resource', 'Properties': {'ParentId': {'Fn::GetAtt': ['helloapi4446A35B', 'RootResourceId']}, 'PathPart': 'hello', 'RestApiId': {'Ref': 'helloapi4446A35B'}}, 'Metadata': {'aws:cdk:path': 'LocalcdkTestStack/hello-api/Default/hello/Resource'}}
2020-12-11T13:51:21:INFO:localstack.services.cloudformation.cloudformation_starter: Resource helloapihelloGETE6A58337 cannot be deployed, found unsatisfied dependencies. ['helloapihello4AA00177'] - {'Type': 'AWS::ApiGateway::Method', 'Properties': {'HttpMethod': 'GET', 'ResourceId': {'Ref': 'helloapihello4AA00177'}, 'RestApiId': {'Ref': 'helloapi4446A35B'}, 'AuthorizationType': 'NONE', 'Integration': {'IntegrationHttpMethod': 'POST', 'Type': 'AWS_PROXY', 'Uri': {'Fn::Join': ['', ['arn:', {'Ref': 'AWS::Partition'}, ':apigateway:', {'Ref': 'AWS::Region'}, ':lambda:path/2015-03-31/functions/', {'Fn::GetAtt': ['DynamoLambdaHandlerFB6EB814', 'Arn']}, '/invocations']]}}}, 'Metadata': {'aws:cdk:path': 'LocalcdkTestStack/hello-api/Default/hello/GET/Resource'}} (existing: {'DynamoLambdaHandlerServiceRole4C867B01', 'helloapiCloudWatchRoleD13E913E', 'AWS::StackId', 'AWS::Region', 'DynamoLambdaHandlerFB6EB814', 'helloapi4446A35B', 'AWS::StackName', 'AWS::Partition', 'AssetParameters804d69e29081cd169ea6bb165c6275208b2a223191337de83f3c2eb0e899a6c8S3Bucket80627059', 'AssetParameters804d69e29081cd169ea6bb165c6275208b2a223191337de83f3c2eb0e899a6c8S3VersionKey5EB7D7C8', 'AssetParameters804d69e29081cd169ea6bb165c6275208b2a223191337de83f3c2eb0e899a6c8ArtifactHashFB78D611', 'AWS::AccountId', 'AWS::URLSuffix'})
2020-12-11T13:51:21:DEBUG:localstack.services.cloudformation.cloudformation_starter: Currently processing (update=False) resource LocalcdkTestStack/helloapihello4AA00177: None
2020-12-11T13:51:21:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: ApiGateway::RestApi PhysicalResourceId
2020-12-11T13:51:21:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: ApiGateway::RestApi RootResourceId
2020-12-11T13:51:21:DEBUG:localstack.services.cloudformation.cloudformation_starter: Deploying CloudFormation resource (update=False, exists=False, updateable=False): {'Type': 'AWS::ApiGateway::Resource', 'Properties': {'ParentId': '9b7khelkyu', 'PathPart': 'hello', 'RestApiId': 'ggolkkmkui'}, 'Metadata': {'aws:cdk:path': 'LocalcdkTestStack/hello-api/Default/hello/Resource'}}
2020-12-11T13:51:21:DEBUG:localstack.utils.cloudformation.template_deployer: Running action "create" for resource type "ApiGateway::Resource" id "helloapihello4AA00177"
2020-12-11T13:51:21:DEBUG:localstack.utils.cloudformation.template_deployer: Request for resource type "ApiGateway::Resource" in region us-east-1: create_resource {'restApiId': 'ggolkkmkui', 'pathPart': 'hello', 'parentId': '9b7khelkyu'}
2020-12-11T13:51:21:DEBUG:localstack.services.cloudformation.cloudformation_starter: Updating resource id: dbf78292 - aqghrud4v9, <moto.apigateway.models.Resource object at 0x7f5230f3bc40> - {'Type': 'AWS::ApiGateway::Resource', 'Properties': {'ParentId': '9b7khelkyu', 'PathPart': 'hello', 'RestApiId': 'ggolkkmkui'}, 'Metadata': {'aws:cdk:path': 'LocalcdkTestStack/hello-api/Default/hello/Resource'}}
2020-12-11T13:51:21:INFO:localstack.services.cloudformation.cloudformation_starter: Updating resource ID from dbf78292 to aqghrud4v9 (us-east-1)
2020-12-11T13:51:21:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: ApiGateway::RestApi PhysicalResourceId
2020-12-11T13:51:21:DEBUG:localstack.services.cloudformation.cloudformation_starter: Currently processing (update=False) resource LocalcdkTestStack/helloapiDeploymentFA89AEEC83909b4e0ef130351933b7171b4a76ad: None
2020-12-11T13:51:21:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: ApiGateway::RestApi PhysicalResourceId
2020-12-11T13:51:21:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: ApiGateway::RestApi RootResourceId
2020-12-11T13:51:21:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: ApiGateway::RestApi PhysicalResourceId
2020-12-11T13:51:21:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: ApiGateway::RestApi PhysicalResourceId
2020-12-11T13:51:21:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: ApiGateway::RestApi RootResourceId
2020-12-11T13:51:22:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: ApiGateway::Resource PhysicalResourceId
2020-12-11T13:51:22:DEBUG:localstack.utils.cloudformation.template_deployer: Dependency for resource helloapiDeploymentFA89AEEC83909b4e0ef130351933b7171b4a76ad not yet deployed: helloapihelloGETE6A58337 {'Type': 'AWS::ApiGateway::Method', 'Properties': {'HttpMethod': 'GET', 'ResourceId': {'Ref': 'helloapihello4AA00177'}, 'RestApiId': {'Ref': 'helloapi4446A35B'}, 'AuthorizationType': 'NONE', 'Integration': {'IntegrationHttpMethod': 'POST', 'Type': 'AWS_PROXY', 'Uri': {'Fn::Join': ['', ['arn:', {'Ref': 'AWS::Partition'}, ':apigateway:', {'Ref': 'AWS::Region'}, ':lambda:path/2015-03-31/functions/', {'Fn::GetAtt': ['DynamoLambdaHandlerFB6EB814', 'Arn']}, '/invocations']]}}}, 'Metadata': {'aws:cdk:path': 'LocalcdkTestStack/hello-api/Default/hello/GET/Resource'}}
2020-12-11T13:51:22:INFO:localstack.services.cloudformation.cloudformation_starter: Resource helloapiDeploymentFA89AEEC83909b4e0ef130351933b7171b4a76ad cannot be deployed, found unsatisfied dependencies. ['helloapihelloGETE6A58337'] - {'Type': 'AWS::ApiGateway::Deployment', 'Properties': {'RestApiId': {'Ref': 'helloapi4446A35B'}, 'Description': 'Automatically created by the RestApi construct'}, 'DependsOn': ['helloapihelloGETE6A58337', 'helloapihello4AA00177'], 'Metadata': {'aws:cdk:path': 'LocalcdkTestStack/hello-api/Deployment/Resource'}} (existing: {'DynamoLambdaHandlerServiceRole4C867B01', 'helloapiCloudWatchRoleD13E913E', 'AWS::StackId', 'AWS::Region', 'DynamoLambdaHandlerFB6EB814', 'helloapi4446A35B', 'AWS::StackName', 'helloapihello4AA00177', 'AWS::Partition', 'AssetParameters804d69e29081cd169ea6bb165c6275208b2a223191337de83f3c2eb0e899a6c8S3Bucket80627059', 'AssetParameters804d69e29081cd169ea6bb165c6275208b2a223191337de83f3c2eb0e899a6c8S3VersionKey5EB7D7C8', 'AssetParameters804d69e29081cd169ea6bb165c6275208b2a223191337de83f3c2eb0e899a6c8ArtifactHashFB78D611', 'AWS::AccountId', 'AWS::URLSuffix'})
2020-12-11T13:51:22:DEBUG:localstack.services.cloudformation.cloudformation_starter: Currently processing (update=False) resource LocalcdkTestStack/helloapiDeploymentStageprod677E2C4F: None
2020-12-11T13:51:22:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: ApiGateway::RestApi PhysicalResourceId
2020-12-11T13:51:22:DEBUG:localstack.utils.cloudformation.template_deployer: Dependency for resource helloapiDeploymentStageprod677E2C4F not yet deployed: helloapiDeploymentFA89AEEC83909b4e0ef130351933b7171b4a76ad {'Type': 'AWS::ApiGateway::Deployment', 'Properties': {'RestApiId': {'Ref': 'helloapi4446A35B'}, 'Description': 'Automatically created by the RestApi construct'}, 'DependsOn': ['helloapihelloGETE6A58337', 'helloapihello4AA00177'], 'Metadata': {'aws:cdk:path': 'LocalcdkTestStack/hello-api/Deployment/Resource'}}
2020-12-11T13:51:22:INFO:localstack.services.cloudformation.cloudformation_starter: Resource helloapiDeploymentStageprod677E2C4F cannot be deployed, found unsatisfied dependencies. ['helloapiDeploymentFA89AEEC83909b4e0ef130351933b7171b4a76ad'] - {'Type': 'AWS::ApiGateway::Stage', 'Properties': {'RestApiId': {'Ref': 'helloapi4446A35B'}, 'DeploymentId': {'Ref': 'helloapiDeploymentFA89AEEC83909b4e0ef130351933b7171b4a76ad'}, 'StageName': 'prod'}, 'Metadata': {'aws:cdk:path': 'LocalcdkTestStack/hello-api/DeploymentStage.prod/Resource'}} (existing: {'DynamoLambdaHandlerServiceRole4C867B01', 'helloapiCloudWatchRoleD13E913E', 'AWS::StackId', 'AWS::Region', 'DynamoLambdaHandlerFB6EB814', 'helloapi4446A35B', 'AWS::StackName', 'helloapihello4AA00177', 'AWS::Partition', 'AssetParameters804d69e29081cd169ea6bb165c6275208b2a223191337de83f3c2eb0e899a6c8S3Bucket80627059', 'AssetParameters804d69e29081cd169ea6bb165c6275208b2a223191337de83f3c2eb0e899a6c8S3VersionKey5EB7D7C8', 'AssetParameters804d69e29081cd169ea6bb165c6275208b2a223191337de83f3c2eb0e899a6c8ArtifactHashFB78D611', 'AWS::AccountId', 'AWS::URLSuffix'})
2020-12-11T13:51:22:DEBUG:localstack.services.cloudformation.cloudformation_starter: Currently processing (update=False) resource LocalcdkTestStack/helloapihelloGETApiPermissionLocalcdkTestStackhelloapi045AD2A0GEThello55445FC5: None
2020-12-11T13:51:22:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: ApiGateway::RestApi PhysicalResourceId
2020-12-11T13:51:22:DEBUG:localstack.utils.cloudformation.template_deployer: Dependency for resource helloapihelloGETApiPermissionLocalcdkTestStackhelloapi045AD2A0GEThello55445FC5 not yet deployed: helloapiDeploymentStageprod677E2C4F {'Type': 'AWS::ApiGateway::Stage', 'Properties': {'RestApiId': {'Ref': 'helloapi4446A35B'}, 'DeploymentId': {'Ref': 'helloapiDeploymentFA89AEEC83909b4e0ef130351933b7171b4a76ad'}, 'StageName': 'prod'}, 'Metadata': {'aws:cdk:path': 'LocalcdkTestStack/hello-api/DeploymentStage.prod/Resource'}}
2020-12-11T13:51:22:INFO:localstack.services.cloudformation.cloudformation_starter: Resource helloapihelloGETApiPermissionLocalcdkTestStackhelloapi045AD2A0GEThello55445FC5 cannot be deployed, found unsatisfied dependencies. ['helloapiDeploymentStageprod677E2C4F'] - {'Type': 'AWS::Lambda::Permission', 'Properties': {'Action': 'lambda:InvokeFunction', 'FunctionName': {'Fn::GetAtt': ['DynamoLambdaHandlerFB6EB814', 'Arn']}, 'Principal': 'apigateway.amazonaws.com', 'SourceArn': {'Fn::Join': ['', ['arn:', {'Ref': 'AWS::Partition'}, ':execute-api:', {'Ref': 'AWS::Region'}, ':', {'Ref': 'AWS::AccountId'}, ':', {'Ref': 'helloapi4446A35B'}, '/', {'Ref': 'helloapiDeploymentStageprod677E2C4F'}, '/GET/hello']]}}, 'Metadata': {'aws:cdk:path': 'LocalcdkTestStack/hello-api/Default/hello/GET/ApiPermission.LocalcdkTestStackhelloapi045AD2A0.GET..hello'}} (existing: {'DynamoLambdaHandlerServiceRole4C867B01', 'helloapiCloudWatchRoleD13E913E', 'AWS::StackId', 'AWS::Region', 'DynamoLambdaHandlerFB6EB814', 'helloapi4446A35B', 'AWS::StackName', 'helloapihello4AA00177', 'AWS::Partition', 'AssetParameters804d69e29081cd169ea6bb165c6275208b2a223191337de83f3c2eb0e899a6c8S3Bucket80627059', 'AssetParameters804d69e29081cd169ea6bb165c6275208b2a223191337de83f3c2eb0e899a6c8S3VersionKey5EB7D7C8', 'AssetParameters804d69e29081cd169ea6bb165c6275208b2a223191337de83f3c2eb0e899a6c8ArtifactHashFB78D611', 'AWS::AccountId', 'AWS::URLSuffix'})
2020-12-11T13:51:22:DEBUG:localstack.services.cloudformation.cloudformation_starter: Currently processing (update=False) resource LocalcdkTestStack/helloapihelloGETApiPermissionTestLocalcdkTestStackhelloapi045AD2A0GEThello8D8E3B3D: None
2020-12-11T13:51:22:WARNING:moto: No Moto CloudFormation support for AWS::Lambda::Permission
2020-12-11T13:51:22:DEBUG:localstack.services.cloudformation.cloudformation_starter: Currently processing (update=False) resource LocalcdkTestStack/CDKMetadata: None
2020-12-11T13:51:22:WARNING:moto: No Moto CloudFormation support for AWS::CDK::Metadata
2020-12-11T13:51:22:DEBUG:localstack.services.cloudformation.cloudformation_starter: Running CloudFormation stack deployment loop iteration 1
2020-12-11T13:51:22:DEBUG:localstack.services.cloudformation.cloudformation_starter: Found 4 unresolved dependencies in loop iteration 1
2020-12-11T13:51:22:DEBUG:localstack.services.cloudformation.cloudformation_starter: Currently processing (update=False) resource LocalcdkTestStack/helloapihelloGETE6A58337: None
2020-12-11T13:51:22:DEBUG:localstack.services.cloudformation.cloudformation_starter: Currently processing (update=False) resource LocalcdkTestStack/helloapiDeploymentStageprod677E2C4F: None
2020-12-11T13:51:22:DEBUG:localstack.services.cloudformation.cloudformation_starter: Currently processing (update=False) resource LocalcdkTestStack/helloapiDeploymentStageprod677E2C4F: True
2020-12-11T13:51:22:DEBUG:localstack.services.cloudformation.cloudformation_starter: Currently processing (update=False) resource LocalcdkTestStack/helloapiDeploymentStageprod677E2C4F: True
2020-12-11T13:51:22:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: ApiGateway::RestApi PhysicalResourceId
2020-12-11T13:51:22:DEBUG:localstack.utils.cloudformation.template_deployer: Dependency for resource helloapiDeploymentStageprod677E2C4F not yet deployed: helloapiDeploymentFA89AEEC83909b4e0ef130351933b7171b4a76ad {'Type': 'AWS::ApiGateway::Deployment', 'Properties': {'RestApiId': {'Ref': 'helloapi4446A35B'}, 'Description': 'Automatically created by the RestApi construct'}, 'DependsOn': ['helloapihelloGETE6A58337', 'helloapihello4AA00177'], 'Metadata': {'aws:cdk:path': 'LocalcdkTestStack/hello-api/Deployment/Resource'}}
2020-12-11T13:51:22:INFO:localstack.services.cloudformation.cloudformation_starter: Resource helloapiDeploymentStageprod677E2C4F cannot be deployed, found unsatisfied dependencies. ['helloapiDeploymentFA89AEEC83909b4e0ef130351933b7171b4a76ad'] - {'Type': 'AWS::ApiGateway::Stage', 'Properties': {'RestApiId': {'Ref': 'helloapi4446A35B'}, 'DeploymentId': {'Ref': 'helloapiDeploymentFA89AEEC83909b4e0ef130351933b7171b4a76ad'}, 'StageName': 'prod'}, 'Metadata': {'aws:cdk:path': 'LocalcdkTestStack/hello-api/DeploymentStage.prod/Resource'}} (existing: {'DynamoLambdaHandlerServiceRole4C867B01', 'helloapiCloudWatchRoleD13E913E', 'AWS::StackId', 'AWS::Region', 'DynamoLambdaHandlerFB6EB814', 'helloapi4446A35B', 'AWS::StackName', 'helloapihello4AA00177', 'AWS::Partition', 'AssetParameters804d69e29081cd169ea6bb165c6275208b2a223191337de83f3c2eb0e899a6c8S3Bucket80627059', 'AssetParameters804d69e29081cd169ea6bb165c6275208b2a223191337de83f3c2eb0e899a6c8S3VersionKey5EB7D7C8', 'AssetParameters804d69e29081cd169ea6bb165c6275208b2a223191337de83f3c2eb0e899a6c8ArtifactHashFB78D611', 'AWS::AccountId', 'AWS::URLSuffix'})
2020-12-11T13:51:22:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: ApiGateway::RestApi PhysicalResourceId
2020-12-11T13:51:22:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: ApiGateway::RestApi RootResourceId
2020-12-11T13:51:22:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: ApiGateway::RestApi PhysicalResourceId
2020-12-11T13:51:23:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: ApiGateway::RestApi PhysicalResourceId
2020-12-11T13:51:23:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: ApiGateway::RestApi RootResourceId
2020-12-11T13:51:23:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: ApiGateway::Resource PhysicalResourceId
2020-12-11T13:51:23:WARNING:localstack.services.cloudformation.cloudformation_starter: Unable to determine physical_resource_id for resource <class 'moto.apigateway.models.Method'>
2020-12-11T13:51:23:DEBUG:localstack.services.cloudformation.cloudformation_starter: Deploying CloudFormation resource (update=False, exists=False, updateable=False): {'Type': 'AWS::ApiGateway::Method', 'Properties': {'HttpMethod': 'GET', 'ResourceId': 'aqghrud4v9', 'RestApiId': 'ggolkkmkui', 'AuthorizationType': 'NONE', 'Integration': {'IntegrationHttpMethod': 'POST', 'Type': 'AWS_PROXY', 'Uri': 'arn:aws:apigateway:us-east-1:lambda:path/2015-03-31/functions/arn:aws:lambda:us-east-1:000000000000:function:LocalcdkTestStack-lambda-007c23ca/invocations'}}, 'Metadata': {'aws:cdk:path': 'LocalcdkTestStack/hello-api/Default/hello/GET/Resource'}}
2020-12-11T13:51:23:DEBUG:localstack.utils.cloudformation.template_deployer: Running action "create" for resource type "ApiGateway::Method" id "helloapihelloGETE6A58337"
2020-12-11T13:51:23:DEBUG:localstack.utils.cloudformation.template_deployer: Request for resource type "ApiGateway::Method" in region us-east-1: put_method {'restApiId': 'ggolkkmkui', 'resourceId': 'aqghrud4v9', 'httpMethod': 'GET', 'authorizationType': 'NONE'}
2020-12-11T13:51:23:WARNING:localstack.services.cloudformation.cloudformation_starter: Unable to determine physical_resource_id for resource <class 'moto.apigateway.models.Method'>
2020-12-11T13:51:23:DEBUG:localstack.services.cloudformation.cloudformation_starter: Currently processing (update=False) resource LocalcdkTestStack/helloapiDeploymentFA89AEEC83909b4e0ef130351933b7171b4a76ad: None
2020-12-11T13:51:23:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: ApiGateway::RestApi PhysicalResourceId
2020-12-11T13:51:23:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: ApiGateway::RestApi RootResourceId
2020-12-11T13:51:23:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: ApiGateway::RestApi PhysicalResourceId
2020-12-11T13:51:23:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: ApiGateway::RestApi PhysicalResourceId
2020-12-11T13:51:23:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: ApiGateway::RestApi RootResourceId
2020-12-11T13:51:23:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: ApiGateway::Resource PhysicalResourceId
2020-12-11T13:51:23:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: ApiGateway::RestApi PhysicalResourceId
2020-12-11T13:51:24:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: ApiGateway::RestApi PhysicalResourceId
2020-12-11T13:51:24:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: ApiGateway::RestApi RootResourceId
2020-12-11T13:51:24:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: ApiGateway::RestApi PhysicalResourceId
2020-12-11T13:51:24:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: ApiGateway::RestApi PhysicalResourceId
2020-12-11T13:51:24:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: ApiGateway::RestApi RootResourceId
2020-12-11T13:51:24:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: ApiGateway::Resource PhysicalResourceId
2020-12-11T13:51:24:WARNING:localstack.services.cloudformation.cloudformation_starter: Unable to determine physical_resource_id for resource <class 'moto.apigateway.models.Deployment'>
2020-12-11T13:51:24:DEBUG:localstack.services.cloudformation.cloudformation_starter: Deploying CloudFormation resource (update=False, exists=False, updateable=False): {'Type': 'AWS::ApiGateway::Deployment', 'Properties': {'RestApiId': 'ggolkkmkui', 'Description': 'Automatically created by the RestApi construct'}, 'DependsOn': ['helloapihelloGETE6A58337', 'helloapihello4AA00177'], 'Metadata': {'aws:cdk:path': 'LocalcdkTestStack/hello-api/Deployment/Resource'}}
2020-12-11T13:51:24:DEBUG:localstack.utils.cloudformation.template_deployer: Running action "create" for resource type "ApiGateway::Deployment" id "helloapiDeploymentFA89AEEC83909b4e0ef130351933b7171b4a76ad"
2020-12-11T13:51:24:DEBUG:localstack.utils.cloudformation.template_deployer: Request for resource type "ApiGateway::Deployment" in region us-east-1: create_deployment {'restApiId': 'ggolkkmkui', 'description': 'Automatically created by the RestApi construct'}
2020-12-11T13:51:24:DEBUG:localstack.services.cloudformation.cloudformation_starter: Updating resource id: 54b34d71 - fl686d26z7, {'id': '54b34d71', 'stageName': None, 'description': 'Automatically created by the RestApi construct', 'createdDate': 1607694683} - {'Type': 'AWS::ApiGateway::Deployment', 'Properties': {'RestApiId': 'ggolkkmkui', 'Description': 'Automatically created by the RestApi construct'}, 'DependsOn': ['helloapihelloGETE6A58337', 'helloapihello4AA00177'], 'Metadata': {'aws:cdk:path': 'LocalcdkTestStack/hello-api/Deployment/Resource'}}
2020-12-11T13:51:24:INFO:localstack.services.cloudformation.cloudformation_starter: Updating resource ID from 54b34d71 to fl686d26z7 (us-east-1)
2020-12-11T13:51:24:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: ApiGateway::RestApi PhysicalResourceId
2020-12-11T13:51:24:WARNING:localstack.services.cloudformation.cloudformation_starter: Unable to determine physical_resource_id for resource <class 'moto.apigateway.models.Deployment'>
2020-12-11T13:51:24:DEBUG:localstack.services.cloudformation.cloudformation_starter: Currently processing (update=False) resource LocalcdkTestStack/helloapiDeploymentStageprod677E2C4F: True
2020-12-11T13:51:24:DEBUG:localstack.services.cloudformation.cloudformation_starter: Currently processing (update=False) resource LocalcdkTestStack/helloapiDeploymentStageprod677E2C4F: True
2020-12-11T13:51:24:DEBUG:localstack.services.cloudformation.cloudformation_starter: Currently processing (update=False) resource LocalcdkTestStack/helloapihelloGETApiPermissionLocalcdkTestStackhelloapi045AD2A0GEThello55445FC5: None
2020-12-11T13:51:24:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: ApiGateway::RestApi PhysicalResourceId
2020-12-11T13:51:24:DEBUG:localstack.utils.cloudformation.template_deployer: Dependency for resource helloapihelloGETApiPermissionLocalcdkTestStackhelloapi045AD2A0GEThello55445FC5 not yet deployed: helloapiDeploymentStageprod677E2C4F {'Type': 'AWS::ApiGateway::Stage', 'Properties': {'RestApiId': {'Ref': 'helloapi4446A35B'}, 'DeploymentId': {'Ref': 'helloapiDeploymentFA89AEEC83909b4e0ef130351933b7171b4a76ad'}, 'StageName': 'prod'}, 'Metadata': {'aws:cdk:path': 'LocalcdkTestStack/hello-api/DeploymentStage.prod/Resource'}}
2020-12-11T13:51:24:INFO:localstack.services.cloudformation.cloudformation_starter: Resource helloapihelloGETApiPermissionLocalcdkTestStackhelloapi045AD2A0GEThello55445FC5 cannot be deployed, found unsatisfied dependencies. ['helloapiDeploymentStageprod677E2C4F'] - {'Type': 'AWS::Lambda::Permission', 'Properties': {'Action': 'lambda:InvokeFunction', 'FunctionName': {'Fn::GetAtt': ['DynamoLambdaHandlerFB6EB814', 'Arn']}, 'Principal': 'apigateway.amazonaws.com', 'SourceArn': {'Fn::Join': ['', ['arn:', {'Ref': 'AWS::Partition'}, ':execute-api:', {'Ref': 'AWS::Region'}, ':', {'Ref': 'AWS::AccountId'}, ':', {'Ref': 'helloapi4446A35B'}, '/', {'Ref': 'helloapiDeploymentStageprod677E2C4F'}, '/GET/hello']]}}, 'Metadata': {'aws:cdk:path': 'LocalcdkTestStack/hello-api/Default/hello/GET/ApiPermission.LocalcdkTestStackhelloapi045AD2A0.GET..hello'}} (existing: {'DynamoLambdaHandlerServiceRole4C867B01', 'helloapiCloudWatchRoleD13E913E', 'AWS::StackId', 'AWS::Region', 'DynamoLambdaHandlerFB6EB814', 'helloapi4446A35B', 'helloapiDeploymentFA89AEEC83909b4e0ef130351933b7171b4a76ad', 'AWS::StackName', 'helloapihello4AA00177', 'AWS::Partition', 'AssetParameters804d69e29081cd169ea6bb165c6275208b2a223191337de83f3c2eb0e899a6c8S3Bucket80627059', 'helloapihelloGETE6A58337', 'AssetParameters804d69e29081cd169ea6bb165c6275208b2a223191337de83f3c2eb0e899a6c8S3VersionKey5EB7D7C8', 'AssetParameters804d69e29081cd169ea6bb165c6275208b2a223191337de83f3c2eb0e899a6c8ArtifactHashFB78D611', 'AWS::AccountId', 'AWS::URLSuffix'})
2020-12-11T13:51:24:DEBUG:localstack.services.cloudformation.cloudformation_starter: Running CloudFormation stack deployment loop iteration 2
2020-12-11T13:51:24:DEBUG:localstack.services.cloudformation.cloudformation_starter: Found 2 unresolved dependencies in loop iteration 2
2020-12-11T13:51:24:DEBUG:localstack.services.cloudformation.cloudformation_starter: Currently processing (update=False) resource LocalcdkTestStack/helloapiDeploymentStageprod677E2C4F: None
2020-12-11T13:51:24:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: ApiGateway::RestApi PhysicalResourceId
2020-12-11T13:51:24:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: ApiGateway::RestApi PhysicalResourceId
2020-12-11T13:51:24:DEBUG:localstack.services.cloudformation.cloudformation_starter: Deploying CloudFormation resource (update=False, exists=False, updateable=False): {'Type': 'AWS::ApiGateway::Stage', 'Properties': {'RestApiId': 'ggolkkmkui', 'DeploymentId': 'fl686d26z7', 'StageName': 'prod'}, 'Metadata': {'aws:cdk:path': 'LocalcdkTestStack/hello-api/DeploymentStage.prod/Resource'}}
2020-12-11T13:51:24:DEBUG:localstack.utils.cloudformation.template_deployer: Running action "create" for resource type "ApiGateway::Stage" id "helloapiDeploymentStageprod677E2C4F"
2020-12-11T13:51:24:DEBUG:localstack.utils.cloudformation.template_deployer: Request for resource type "ApiGateway::Stage" in region us-east-1: create_stage {'restApiId': 'ggolkkmkui', 'deploymentId': 'fl686d26z7', 'stageName': 'prod'}
2020-12-11T13:51:24:DEBUG:localstack.services.cloudformation.cloudformation_starter: Currently processing (update=False) resource LocalcdkTestStack/helloapihelloGETApiPermissionLocalcdkTestStackhelloapi045AD2A0GEThello55445FC5: None
2020-12-11T13:51:25:DEBUG:localstack.utils.cloudformation.template_deployer: Extract resource attribute: ApiGateway::RestApi PhysicalResourceId
2020-12-11T13:51:25:WARNING:moto: No Moto CloudFormation support for AWS::Lambda::Permission
2020-12-11T13:51:25:DEBUG:localstack.services.cloudformation.cloudformation_starter: Running CloudFormation stack deployment loop iteration 3
2020-12-11T13:51:25:DEBUG:localstack.services.cloudformation.cloudformation_starter: Found 0 unresolved dependencies in loop iteration 3
2020-12-11T13:51:25:INFO:localstack.services.cloudformation.cloudformation_starter: Error running stack deployment loop: 'CloudFormationBackend' object has no attribute 'set_exports'
2020-12-11T13:51:25:WARNING:bootstrap.py: Thread run method <function apply_patches.<locals>.run_dependencies_deployment_loop.<locals>.run_loop at 0x7f522f9d5b80>(None) failed: Unable to resolve all CloudFormation resources after traversing dependency tree (maximum depth 40): ['DynamoLambdaHandlerServiceRole4C867B01', 'DynamoLambdaHandlerFB6EB814', 'helloapi4446A35B', 'helloapiCloudWatchRoleD13E913E', 'helloapiAccountD8C38BCE', 'helloapiDeploymentFA89AEEC83909b4e0ef130351933b7171b4a76ad', 'helloapiDeploymentStageprod677E2C4F', 'helloapihello4AA00177', 'helloapihelloGETApiPermissionLocalcdkTestStackhelloapi045AD2A0GEThello55445FC5', 'helloapihelloGETApiPermissionTestLocalcdkTestStackhelloapi045AD2A0GEThello8D8E3B3D', 'helloapihelloGETE6A58337', 'CDKMetadata'] Traceback (most recent call last):
  File "/opt/code/localstack/localstack/utils/bootstrap.py", line 581, in run
    result = self.func(self.params)
  File "/opt/code/localstack/localstack/services/cloudformation/cloudformation_starter.py", line 1210, in run_loop
    raise Exception('Unable to resolve all CloudFormation resources after traversing ' +
Exception: Unable to resolve all CloudFormation resources after traversing dependency tree (maximum depth 40): ['DynamoLambdaHandlerServiceRole4C867B01', 'DynamoLambdaHandlerFB6EB814', 'helloapi4446A35B', 'helloapiCloudWatchRoleD13E913E', 'helloapiAccountD8C38BCE', 'helloapiDeploymentFA89AEEC83909b4e0ef130351933b7171b4a76ad', 'helloapiDeploymentStageprod677E2C4F', 'helloapihello4AA00177', 'helloapihelloGETApiPermissionLocalcdkTestStackhelloapi045AD2A0GEThello55445FC5', 'helloapihelloGETApiPermissionTestLocalcdkTestStackhelloapi045AD2A0GEThello8D8E3B3D', 'helloapihelloGETE6A58337', 'CDKMetadata']

@sbassin
Copy link

sbassin commented Dec 12, 2020

I'm having a very similar issue, though with different output. Also on the latest localstack, but I'm on 1.78.0 of cdklocal.

When I cdklocal bootstrap, I get:

Unable to resolve AWS account to use. It must be either configured when you define your CDK or through the environment

When I cdklocal bootstrap aws://000000000000/us-east-1 like @guoliang, I get:

 ⏳  Bootstrapping environment aws://000000000000/us-east-1...
 ❌  Environment aws://000000000000/us-east-1 failed bootstrapping: Error: Need to perform AWS calls for account 000000000000, but no credentials have been configured
    at SdkProvider.forEnvironment (/Users/sbassin/.nvm/versions/node/v12.20.0/lib/node_modules/aws-cdk-local/node_modules/aws-cdk/lib/api/aws-auth/sdk-provider.ts:149:46)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
    at Function.lookup (/Users/sbassin/.nvm/versions/node/v12.20.0/lib/node_modules/aws-cdk-local/node_modules/aws-cdk/lib/api/bootstrap/deploy-bootstrap.ts:35:17)
    at Bootstrapper.legacyBootstrap (/Users/sbassin/.nvm/versions/node/v12.20.0/lib/node_modules/aws-cdk-local/node_modules/aws-cdk/lib/api/bootstrap/bootstrap-environment.ts:61:21)
    at /Users/sbassin/.nvm/versions/node/v12.20.0/lib/node_modules/aws-cdk-local/node_modules/aws-cdk/lib/cdk-toolkit.ts:358:24
    at async Promise.all (index 0)
    at CdkToolkit.bootstrap (/Users/sbassin/.nvm/versions/node/v12.20.0/lib/node_modules/aws-cdk-local/node_modules/aws-cdk/lib/cdk-toolkit.ts:355:5)
    at initCommandLine (/Users/sbassin/.nvm/versions/node/v12.20.0/lib/node_modules/aws-cdk-local/node_modules/aws-cdk/bin/cdk.ts:201:9)
Need to perform AWS calls for account 000000000000, but no credentials have been configured

This was working recently with 1.76.0. The only change that stood out to me as a possible breaking one between the two versions of CDK was aws/aws-cdk#11966, but I can't find anything in that PR itself that looks like it would cause this problem. Though that aws-cdk/lib/api/aws-auth/sdk-provider.ts:149:46 is part of the change set.

Edit More Context:
FWIW, my local I have in my config file:

[profile localstack]
region=us-east-1
output=json

and in my credentials file:

[localstack]
aws_access_key_id=test
aws_secret_access_key=test

This is also failing in my CI environment where I use environment variables:

AWS_DEFAULT_REGION=us-east-1
AWS_SECRET_ACCESS_KEY=test
AWS_ACCESS_KEY_ID=test

@whummer
Copy link
Member

whummer commented Dec 13, 2020

Thanks for reporting @guoliang . Can you please share a full copy of your CDK program (e.g., as a zip file) to help us reproduce this issue? Thanks

@sbassin - looks like you're facing a different issue. Can you please share some details about your environment/configuration - is the STS service running in your LocalStack instance?

@sbassin
Copy link

sbassin commented Dec 13, 2020

Yup. I've got the following services running in LocalStack: cloudformation,s3,iam,sts,events,lambda,dynamodb. Anything else you'd want to know?

@whummer
Copy link
Member

whummer commented Dec 13, 2020

Thanks @sbassin . I think we were able to reproduce the issue you're facing - can you please give it another try with version aws-cdk-local@1.65.2? Please note that aws-cdk now needs to be installed manually, to decouple the version dependency between the two libraries. Thanks

@guoliang
Copy link
Author

guoliang commented Dec 14, 2020

@whummer I tried it out and it works now
So currently I have the following versions

localstack/localstack@latest
aws-cdk-local@1.65.2
aws-cdk@1.78.0

then when I run

$ cdklocal bootstrap
$ cdklocal deploy

both commands ran OK.

However when I run deploy again with a small change to my lambda function it failed.
image

@sbassin
Copy link

sbassin commented Dec 14, 2020

Hi, @whummer. I just tried with aws-cdk-local@1.65.2 and aws-cdk@1.78.0 and am not noticing a difference. To your point:

Please note that aws-cdk now needs to be installed manually

I've got aws-cdk@1.78.0 installed globally and in my CDK project's package.json. Or were you hoping for something different from that.

For what it's worth, I can use awslocal with LocalStack…

Thanks.

@sbassin
Copy link

sbassin commented Dec 14, 2020

I'm noticing that my cdklocal version is 1.78… so maybe there's still some entanglement?

cdklocal --version
1.78.0 (build 2c74f4c)

@guoliang
Copy link
Author

guoliang commented Dec 14, 2020

@sbassin
I think you mixing it up maybe?

when you run cdklocal --version the version you get is the version of aws-cdk

@whummer
Copy link
Member

whummer commented Dec 14, 2020

@sbassin Perhaps you may need to clean the existing state in your ~/.cdk/... or local cdk.out folder...?

@guoliang
Copy link
Author

@whummer any idea why it's not possible to update a cdk deploy?

@sbassin
Copy link

sbassin commented Dec 15, 2020

Ah, @guoliang. I was thinking I'd get the version of cdklocal as the README seems to indicate:

The cdklocal command line is published as an npm library:

$ npm install aws-cdk-local aws-cdk
...
$ cdklocal --version
1.65.2

I thought I might be onto something there about my cdklocal and aws-cdk being improperly configured…

@whummer, I deleted both ~/.cdk and the local cdk.out directory, but to no avail. I'd cleared cdk.out before, but not the other.

However, I then iterated through CDK versions 1.75 through 1.78, each time blowing away my CDK's node_modules as well as those two directories you suggested, and I was able to get past my problem.

Each time, cdklocal bootstrap fails with this error message.

cdklocal bootstrap                             
Unable to resolve AWS account to use. It must be either configured when you define your CDK or through the environment

However! I tried running npx cdklocal bootstrap, and that works – even on CDK 1.78! 🎉

npx cdklocal bootstrap
 ⏳  Bootstrapping environment aws://000000000000/us-east-1...
CDKToolkit: creating CloudFormation changeset...
[··························································] (0/3)

 ✅  Environment aws://000000000000/us-east-1 bootstrapped.

I'm not sure what made cdklocal bootstrap stop working – are there other artifacts left behind by the globally deployed cdk or cdklocal modules?

Thanks for your help, @whummer and @guoliang!

@rbrianhazzard
Copy link

Hi apologies if I'm just adding to the noise. @sbassin - I'm seeing the same issues you're seeing with bootstrap. Just to clarify, when you say:

However, I then iterated through CDK versions 1.75 through 1.78, each time blowing away my CDK's node_modules as well as those two directories you suggested, and I was able to get past my problem.

Do you mean your global node_modules as well? I am still hitting this problem through npx cdklocal bootstrap. I have cleared out my local node-modules, ~/.cdk and cdk.out, but I'm still seeing:

$ npx cdklocal bootstrap
Unable to resolve AWS account to use. It must be either configured when you define your CDK or through the environment
$ npm list -g --depth=0 | grep cdk
npm info it worked if it ends with ok
npm info using npm@6.14.6
npm info using node@v12.18.3
├── aws-cdk@1.78.0
├── aws-cdk-local@1.65.2

@sbassin
Copy link

sbassin commented Dec 15, 2020

Interesting… I did not touch my global node_modules. Just the one in my project. Looks like we've got different cdk packages installed globally:

$ npm list -g --depth=0 | grep cdk
├── aws-cdk-local@1.65.2
├── cdk@1.78.0

@denisdifazio
Copy link

I'm also having the same problem. Looking at the output from cdklocal bootstrap -v, I receive the error:

Determining if we're on an EC2 instance.
Does not look like an EC2 instance.
Toolkit stack: CDKToolkit
Setting "CDK_DEFAULT_REGION" environment variable to us-east-1
Looking up default account ID from STS
Unable to determine the default AWS account: 404: 
    at Request.extractError (/workspace/security-service/node_modules/aws-cdk/node_modules/aws-sdk/lib/protocol/query.js:50:29)
    at Request.callListeners (/workspace/security-service/node_modules/aws-cdk/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
    at Request.emit (/workspace/security-service/node_modules/aws-cdk/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
    at Request.emit (/workspace/security-service/node_modules/aws-cdk/node_modules/aws-sdk/lib/request.js:688:14)
    at Request.transition (/workspace/security-service/node_modules/aws-cdk/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/workspace/security-service/node_modules/aws-cdk/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /workspace/security-service/node_modules/aws-cdk/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (/workspace/security-service/node_modules/aws-cdk/node_modules/aws-sdk/lib/request.js:38:9)
    at Request.<anonymous> (/workspace/security-service/node_modules/aws-cdk/node_modules/aws-sdk/lib/request.js:690:12)
    at Request.callListeners (/workspace/security-service/node_modules/aws-cdk/node_modules/aws-sdk/lib/sequential_executor.js:116:18) {
  code: 404,
  time: 2020-12-16T13:15:39.111Z,
  requestId: undefined,
  statusCode: 404,
  retryable: false,
  retryDelay: 79.38828676157019
}

@Tehnix
Copy link

Tehnix commented Jan 1, 2021

Something seems to have changed in aws-cdk which results in the error here.

I've tested the following constellations:

aws-cdk-local aws-cdk localstack working
1.65.0 1.81.0 0.12.4 (latest) ⛔️
1.65.2 1.81.0 0.12.4 (latest) ⛔️
1.65.0 1.76.0 0.12.4 (latest)
1.65.2 1.76.0 0.12.4 (latest)

In all test runs I've nuked node_modules and docker to start from a clean slate.

@whummer
Copy link
Member

whummer commented Jan 2, 2021

Thanks for the update @Tehnix @denisdifazio @sbassin . We just did a quick test with the versions specified above (aws-cdk-local@1.65.2, aws-cdk@1.81.0), and it seems to be working with latest. Can you please do a docker pull localstack/localstack and give it another try with the latest Docker image?

What are the error logs you're seeing - is it the same message as @denisdifazio posted above: Unable to determine the default AWS account: 404: ..? Can you please share a minimal example (e.g., as a small project in a Github repo) to help reproduce this issue? Thanks for your help

@Tehnix
Copy link

Tehnix commented Jan 2, 2021

@whummer awesome, I just tested and with latest and aws-cdk version 1.81.0 (build 6ef67c7) the whole flow worked :)

Just wanted to also say thanks for all the effort y'all are doing! Especially responding on weekends like this 😅

@whummer whummer closed this as completed Jan 3, 2021
@riceyrice
Copy link

riceyrice commented Feb 3, 2021

I have had the same error (well colleagues have and I haven't) - what I had not realized was that something added the following to my ~/.cdk/cache/accounts_partitions.json:

  "test": {
    "accountId": "000000000000",
    "partition": "aws"
  },

Removing this entry caused the error to occur (aws-cdk-local: 1.65.3, aws-cdk: 1.86.0)

@riceyrice
Copy link

riceyrice commented Feb 3, 2021

Using localstack latest (0.12.6) when CDK attempts to lookup default account ID from STS, localstack outputs:

localstack_1  | localstack.utils.server.http2_server.HTTPErrorResponse: Unable to find listener for service "sts" - please make sure to include it in $SERVICES

Followed the advice, all working now 😊

@ben-eb
Copy link

ben-eb commented May 11, 2022

I'm also seeing CREATE_FAILED when trying to run cdklocal bootstrap on a previously bootstrapped environment. I am getting the following error:

CDKToolkit |  0/15 | 2:38:12 PM | CREATE_IN_PROGRESS   | AWS::CloudFormation::Stack | CDKToolkit 
CDKToolkit |  0/15 | 2:38:12 PM | UPDATE_IN_PROGRESS   | AWS::CloudFormation::Stack | FileAssetsBucketEncryptionKey 
CDKToolkit |  0/15 | 2:38:12 PM | UPDATE_IN_PROGRESS   | AWS::CloudFormation::Stack | FileAssetsBucketEncryptionKeyAlias 
CDKToolkit |  0/15 | 2:38:12 PM | UPDATE_IN_PROGRESS   | AWS::CloudFormation::Stack | StagingBucket 
CDKToolkit |  0/15 | 2:38:12 PM | UPDATE_IN_PROGRESS   | AWS::CloudFormation::Stack | StagingBucketPolicy 
CDKToolkit |  0/15 | 2:38:12 PM | UPDATE_IN_PROGRESS   | AWS::CloudFormation::Stack | ContainerAssetsRepository 
CDKToolkit |  0/15 | 2:38:12 PM | UPDATE_IN_PROGRESS   | AWS::CloudFormation::Stack | FilePublishingRole 
CDKToolkit |  0/15 | 2:38:12 PM | UPDATE_IN_PROGRESS   | AWS::CloudFormation::Stack | ImagePublishingRole 
CDKToolkit |  0/15 | 2:38:12 PM | UPDATE_IN_PROGRESS   | AWS::CloudFormation::Stack | LookupRole 
CDKToolkit |  0/15 | 2:38:12 PM | UPDATE_IN_PROGRESS   | AWS::CloudFormation::Stack | FilePublishingRoleDefaultPolicy 
CDKToolkit |  0/15 | 2:38:12 PM | UPDATE_IN_PROGRESS   | AWS::CloudFormation::Stack | ImagePublishingRoleDefaultPolicy 
CDKToolkit |  0/15 | 2:38:12 PM | UPDATE_IN_PROGRESS   | AWS::CloudFormation::Stack | DeploymentActionRole 
CDKToolkit |  0/15 | 2:38:12 PM | UPDATE_IN_PROGRESS   | AWS::CloudFormation::Stack | CloudFormationExecutionRole 
CDKToolkit |  0/15 | 2:38:12 PM | UPDATE_IN_PROGRESS   | AWS::CloudFormation::Stack | CdkBootstrapVersion 
CDKToolkit |  1/15 | 2:38:13 PM | CREATE_COMPLETE      | AWS::CloudFormation::Stack | FileAssetsBucketEncryptionKey 
CDKToolkit |  2/15 | 2:38:14 PM | CREATE_COMPLETE      | AWS::CloudFormation::Stack | FileAssetsBucketEncryptionKeyAlias 
CDKToolkit |  3/15 | 2:38:15 PM | UPDATE_COMPLETE      | AWS::CloudFormation::Stack | StagingBucket 
CDKToolkit |  4/15 | 2:38:15 PM | UPDATE_COMPLETE      | AWS::CloudFormation::Stack | StagingBucketPolicy 
CDKToolkit |  5/15 | 2:38:15 PM | UPDATE_COMPLETE      | AWS::CloudFormation::Stack | ContainerAssetsRepository 
CDKToolkit |  6/15 | 2:38:16 PM | UPDATE_COMPLETE      | AWS::CloudFormation::Stack | FilePublishingRole 
CDKToolkit |  7/15 | 2:38:16 PM | UPDATE_COMPLETE      | AWS::CloudFormation::Stack | ImagePublishingRole 
CDKToolkit |  8/15 | 2:38:16 PM | UPDATE_COMPLETE      | AWS::CloudFormation::Stack | LookupRole 
CDKToolkit |  9/15 | 2:38:19 PM | UPDATE_COMPLETE      | AWS::CloudFormation::Stack | DeploymentActionRole 
CDKToolkit | 10/15 | 2:38:19 PM | UPDATE_COMPLETE      | AWS::CloudFormation::Stack | CloudFormationExecutionRole 
CDKToolkit | 11/15 | 2:38:19 PM | UPDATE_COMPLETE      | AWS::CloudFormation::Stack | CdkBootstrapVersion 
CDKToolkit | 11/15 | 2:38:19 PM | CREATE_FAILED        | AWS::CloudFormation::Stack | CDKToolkit 

Failed resources:
 ❌  Environment aws://000000000000/eu-west-1 failed bootstrapping: Error: The stack named CDKToolkit failed to deploy: CREATE_FAILED (Deployment failed)

And the output from docker:

localstack  | 2022-05-11T13:38:19.756:DEBUG:localstack.utils.cloudformation.template_deployer: Handling "Add" for resource "ImagePublishingRoleDefaultPolicy" (2/2) type "AWS::IAM::Policy" in loop iteration 2 (should_deploy=True)
localstack  | 2022-05-11T13:38:19.770:DEBUG:localstack.utils.cloudformation.template_deployer: Dependency for resource {'Type': 'AWS::IAM::Policy', 'LogicalResourceId': 'ImagePublishingRoleDefaultPolicy', 'Properties': {'PolicyDocument': {'Statement': [{'Action': ['ecr:PutImage', 'ecr:InitiateLayerUpload', 'ecr:UploadLayerPart', 'ecr:CompleteLayerUpload', 'ecr:BatchCheckLayerAvailability', 'ecr:DescribeRepositories', 'ecr:DescribeImages', 'ecr:BatchGetImage', 'ecr:GetDownloadUrlForLayer'], 'Resource': 'arn:aws:ecr:eu-west-1:000000000000:repository/cdk--container-assets-000000000000-eu-west-1', 'Effect': 'Allow'}, {'Action': ['ecr:GetAuthorizationToken'], 'Resource': '*', 'Effect': 'Allow'}], 'Version': '2012-10-17'}, 'Roles': ['cdk--image-publishing-role-000000000000-eu-west-1'], 'PolicyName': 'cdk--image-publishing-role-default-policy-000000000000-eu-west-1'}, '_state_': {'role:cdk-hnb659fds-image-publishing-role-000000000000-eu-west-1': {'RoleName': 'cdk-hnb659fds-image-publishing-role-000000000000-eu-west-1', 'PolicyName': 'cdk-hnb659fds-image-publishing-role-default-policy-000000000000-eu-west-1', 'PolicyDocument': {'Statement': [{'Action': ['ecr:PutImage', 'ecr:InitiateLayerUpload', 'ecr:UploadLayerPart', 'ecr:CompleteLayerUpload', 'ecr:BatchCheckLayerAvailability', 'ecr:DescribeRepositories', 'ecr:DescribeImages', 'ecr:BatchGetImage', 'ecr:GetDownloadUrlForLayer'], 'Resource': 'arn:aws:ecr:eu-west-1:000000000000:repository/cdk-hnb659fds-container-assets-000000000000-eu-west-1', 'Effect': 'Allow'}, {'Action': ['ecr:GetAuthorizationToken'], 'Resource': '*', 'Effect': 'Allow'}], 'Version': '2012-10-17'}, 'ResponseMetadata': {'RequestId': '7e7cd8bc-99ef-11e1-a4c3-27EXAMPLE804', 'HTTPStatusCode': 200, 'HTTPHeaders': {'content-type': 'text/html; charset=utf-8', 'content-length': '942', 'access-control-allow-origin': '*', 'access-control-allow-methods': 'HEAD,GET,PUT,POST,DELETE,OPTIONS,PATCH', 'access-control-allow-headers': 'authorization,cache-control,content-length,content-md5,content-type,etag,location,x-amz-acl,x-amz-content-sha256,x-amz-date,x-amz-request-id,x-amz-security-token,x-amz-tagging,x-amz-target,x-amz-user-agent,x-amz-version-id,x-amzn-requestid,x-localstack-target,amz-sdk-invocation-id,amz-sdk-request', 'access-control-expose-headers': 'etag,x-amz-version-id', 'connection': 'close', 'date': 'Wed, 11 May 2022 13:36:17 GMT', 'server': 'hypercorn-h11'}, 'RetryAttempts': 0}}}, 'PhysicalResourceId': 'cdk-hnb659fds-image-publishing-role-default-policy-000000000000-eu-west-1'} not yet deployed: ImagePublishingRole {'Type': 'AWS::IAM::Role', 'Properties': {'AssumeRolePolicyDocument': {'Statement': [{'Action': 'sts:AssumeRole', 'Effect': 'Allow', 'Principal': {'AWS': {'Ref': 'AWS::AccountId'}}}, {'Fn::If': ['HasTrustedAccounts', {'Action': 'sts:AssumeRole', 'Effect': 'Allow', 'Principal': {'AWS': {'Ref': 'TrustedAccounts'}}}, {'Ref': 'AWS::NoValue'}]}]}, 'RoleName': {'Fn::Sub': 'cdk-${Qualifier}-image-publishing-role-${AWS::AccountId}-${AWS::Region}'}, 'Tags': [{'Key': 'aws-cdk:bootstrap-role', 'Value': 'image-publishing'}]}, 'LogicalResourceId': 'ImagePublishingRole'}
localstack  | 2022-05-11T13:38:19.771:DEBUG:localstack.utils.cloudformation.template_deployer: Error applying changes for CloudFormation stack "CDKToolkit": Resource deployment loop completed, pending resource changes: [{'Type': 'Resource', 'ResourceChange': {'Action': 'Add', 'LogicalResourceId': 'FilePublishingRoleDefaultPolicy', 'PhysicalResourceId': None, 'ResourceType': 'AWS::IAM::Policy', 'Replacement': 'False', 'ChangeSetId': None}}, {'Type': 'Resource', 'ResourceChange': {'Action': 'Add', 'LogicalResourceId': 'ImagePublishingRoleDefaultPolicy', 'PhysicalResourceId': None, 'ResourceType': 'AWS::IAM::Policy', 'Replacement': 'False', 'ChangeSetId': None}}] Traceback (most recent call last):
localstack  |   File "/opt/code/localstack/localstack/utils/cloudformation/template_deployer.py", line 1527, in _run
localstack  |     self.do_apply_changes_in_loop(changes, stack, stack_name)
localstack  |   File "/opt/code/localstack/localstack/utils/cloudformation/template_deployer.py", line 1613, in do_apply_changes_in_loop
localstack  |     raise Exception(
localstack  | Exception: Resource deployment loop completed, pending resource changes: [{'Type': 'Resource', 'ResourceChange': {'Action': 'Add', 'LogicalResourceId': 'FilePublishingRoleDefaultPolicy', 'PhysicalResourceId': None, 'ResourceType': 'AWS::IAM::Policy', 'Replacement': 'False', 'ChangeSetId': None}}, {'Type': 'Resource', 'ResourceChange': {'Action': 'Add', 'LogicalResourceId': 'ImagePublishingRoleDefaultPolicy', 'PhysicalResourceId': None, 'ResourceType': 'AWS::IAM::Policy', 'Replacement': 'False', 'ChangeSetId': None}}]
localstack  |

I am using cdklocal at 2.15.0 and aws-cdk at 1.152.0 currently. I've not tried upgrading aws-cdk to v2 yet, but will be doing so in the coming weeks.

I would expect that running cdklocal bootstrap would print this instead of failing:

 ⏳  Bootstrapping environment aws://000000000000/eu-west-1...
 ✅  Environment aws://000000000000/eu-west-1 bootstrapped (no changes).

As a workaround I can stop the container and start it again if I need to redeploy my stack, but it's not ideal! 🙂

@whummer
Copy link
Member

whummer commented May 11, 2022

Hi @ben-eb, thanks for the update. Can you please share a copy of your CDK program with us (ideally in a new Github issue) - we'll be happy to look into it and try to reproduce this issue. Thanks for your help!

Also - just to confirm, does this problem persist after pulling the latest Docker image?

@ben-eb
Copy link

ben-eb commented May 12, 2022

@whummer Yes, to clarify I am using 0.14.2 here. 👍

I'm afraid the project this is happening in is closed source. I'll see if I can have a look into making a reproduction with less defined in CDK to check if it's a problem with the stack resources. 🙂

@whummer
Copy link
Member

whummer commented May 14, 2022

Hi again @ben-eb , looks like we've been able to replicate this issue, and have now pushed a change which should (hopefully) get it resolved. Can you please pull the latest Docker image and give it another try? Thanks

@ben-eb
Copy link

ben-eb commented May 16, 2022

@whummer Yep, it's working with the latest release. Thanks very much! 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

8 participants