Error: Failed to create managed resources:

[dazzag24]

Hi,

Attempting to run this stack but I'm running into an error:

Configuring SAM deploy
======================

        Looking for config file [samconfig.toml] :  Not found

        Setting default arguments for 'sam deploy'
        =========================================
        Stack Name [sam-app]: dicom-metadata-extractor-serverless-datalake
        AWS Region [eu-west-2]:
        Parameter S3InputBucketName []: ml-dicom-metadata-extractor-input
        Parameter S3OutputBucketName []: ml-dicom-metadata-extractor-datalake
        Parameter VpcBatch []: vpc-dicom-metadata-extractor
        Parameter SubnetsBatch []: subnet-dicom-metadata-extractor-1,subnet-dicom-metadata-extractor-2
        Parameter ContainerMemory [1024]:
        Parameter ContainervCPU [0.5]:
        Parameter LambdaMemory [256]:
        Parameter LambdaDuration [600]:
        Parameter AssignPublicIp [ENABLED]:
        Parameter PartitionKey [study_date]:
        Parameter GlueTableName [dicom_metadata]:
        Parameter LogLevel [INFO]:
        Parameter VersionDescription [1]:
        #Shows you resources changes to be deployed and require a 'Y' to initiate deploy
        Confirm changes before deploy [y/N]: y
        #SAM needs permission to be able to create roles to connect to the resources in your template
        Allow SAM CLI IAM role creation [Y/n]: Y
        Save arguments to configuration file [Y/n]: Y
        SAM configuration file [samconfig.toml]:
        SAM configuration environment [default]:

        Looking for resources needed for deployment:
        Creating the required resources...
Error: Failed to create managed resources: Waiter StackCreateComplete failed: Waiter encountered a terminal failure state: For expression "Stacks[].StackStatus" we matched expected path: "ROLLBACK_COMPLETE" at least once

which isn't very useful. Meanwhile over in the CloudFormation console I have the following errors:

Can you please confirm:

1. The stack creates the input and output S3 buckets for you
2. The VPC and subnets must already exist? This step isn't clear from the docs, but my reading of the serverless.yml file indicates that these are not created.

My bucket names appear to be fine. So how do I get more detailed info on the reason for the failure?

Thanks

[hemandee]

1. The stack creates the input and output S3 buckets for you
   • Yes, the input and output buckets are created by the stack
2. The VPC and subnets must already exist? This step isn't clear from the docs, but my reading of the serverless.yml file indicates that these are not created.
   • The subnets are not created by the stack.

For parameter entry:
Parameter SubnetsBatch []: subnet-dicom-metadata-extractor-1,subnet-dicom-metadata-extractor-2,
needs to be subnet ids like subnet-111a1122

The error appears to be related to SAM CLI Bucket creation, in the process a SAM CLI bucket will be created first in separate Cloudformation stack (aws-sam-cli-managed-default) then it proceeds to create resources in serverless.yaml [1]. The guided version of sam deploy will check if this stack (aws-sam-cli-managed-default) is good before proceeding. If the stack is in not in good standing this will need to fixed. If none of the resources are being used you can delete the stack (aws-sam-cli-managed-default) and sam deploy --guided will try and recreate it again.

To get more details on failure, if you have CloudTrail enabled you can query the outcome of CreateBucket API failed [2].

The sam deploy command has a --debug flag [3]

[1] https://aws.amazon.com/blogs/compute/a-simpler-deployment-experience-with-aws-sam-cli/ Q How many buckets are created?

[2] https://docs.aws.amazon.com/awscloudtrail/latest/userguide/view-cloudtrail-events-console.html#filtering-cloudtrail-events

[3] https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-cli-command-reference-sam-deploy.html

[dazzag24]

Thanks for the hints. Using correct existing vpc and subnets, and the --debug flag I now observe this:

Note the use of aws-vault as I have a policy of not allowing AWS keys in plain text on disk.

> aws-vault exec XXXXXXXXXXX--no-session -- sam deploy --debug --guided --capabilities CAPABILITY_NAMED_IAM --image-repository $REPO_URL
2022-03-03 16:16:55,573 | Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
2022-03-03 16:16:55,573 | Using config file: samconfig.toml, config environment: default
2022-03-03 16:16:55,574 | Expand command line arguments to:
2022-03-03 16:16:55,574 | --guided --template_file=/home/XXXXXXXXXXX/code/XXXXXXXXXXX/dicom-metadata-extractor-serverless-datalake/.aws-sam/build/template.yaml --capabilities=('CAPABILITY_NAMED_IAM',) --image_repository=XXXXXXXXXXX.dkr.ecr.eu-west-2.amazonaws.com/dicom-parser --stack_name=sam-app --fail_on_empty_changeset

Configuring SAM deploy
======================

        Looking for config file [samconfig.toml] :  Not found

        Setting default arguments for 'sam deploy'
        =========================================
        Stack Name [sam-app]:  dicom-metadata-extractor-serverless-datalake
        AWS Region [eu-west-2]:
        Parameter S3InputBucketName []: xxxxx-dicom-metadata-extractor-input
        Parameter S3OutputBucketName []: xxxxxxx-dicom-metadata-extractor-datalake
        Parameter VpcBatch []: vpc-XXXXXXXXXX
        Parameter SubnetsBatch []: subnet-XXXXXXXXXXX,subnet-XXXXXXXXXXX
        Parameter ContainerMemory [1024]:
        Parameter ContainervCPU [0.5]:
        Parameter LambdaMemory [256]:
        Parameter LambdaDuration [600]:
        Parameter AssignPublicIp [ENABLED]:
        Parameter PartitionKey [study_date]:
        Parameter GlueTableName [dicom_metadata]:
        Parameter LogLevel [INFO]:
        Parameter VersionDescription [1]:
2022-03-03 16:21:43,933 | Collected default values for parameters: {'ContainerMemory': 1024, 'ContainervCPU': 0.5, 'LambdaMemory': 256, 'LambdaDuration': 600, 'AssignPublicIp': 'ENABLED', 'PartitionKey': 'study_date', 'GlueTableName': 'dicom_metadata', 'LogLevel': 'INFO', 'VersionDescription': 1}
2022-03-03 16:21:43,961 | Unable to resolve property SecurityGroupIds: [OrderedDict([('Fn::GetAtt', ['SecurityGroup', 'GroupId'])])]. Leaving as is.
2022-03-03 16:21:43,961 | Unable to resolve property Image: OrderedDict([('Fn::GetAtt', ['CustomGetImageResource', 'ImageUri'])]). Leaving as is.
2022-03-03 16:21:43,962 | 25 stacks found in the template
        #Shows you resources changes to be deployed and require a 'Y' to initiate deploy
        Confirm changes before deploy [y/N]: y
        #SAM needs permission to be able to create roles to connect to the resources in your template
        Allow SAM CLI IAM role creation [Y/n]:
2022-03-03 16:22:12,829 | Collected default values for parameters: {'ContainerMemory': 1024, 'ContainervCPU': 0.5, 'LambdaMemory': 256, 'LambdaDuration': 600, 'AssignPublicIp': 'ENABLED', 'PartitionKey': 'study_date', 'GlueTableName': 'dicom_metadata', 'LogLevel': 'INFO', 'VersionDescription': 1}
2022-03-03 16:22:12,877 | Unable to resolve property SecurityGroupIds: [OrderedDict([('Fn::GetAtt', ['SecurityGroup', 'GroupId'])])]. Leaving as is.
2022-03-03 16:22:12,877 | Unable to resolve property Image: OrderedDict([('Fn::GetAtt', ['CustomGetImageResource', 'ImageUri'])]). Leaving as is.
2022-03-03 16:22:12,878 | 25 resources found in the stack
2022-03-03 16:22:12,878 | Collected default values for parameters: {'ContainerMemory': 1024, 'ContainervCPU': 0.5, 'LambdaMemory': 256, 'LambdaDuration': 600, 'AssignPublicIp': 'ENABLED', 'PartitionKey': 'study_date', 'GlueTableName': 'dicom_metadata', 'LogLevel': 'INFO', 'VersionDescription': 1}
2022-03-03 16:22:12,909 | Unable to resolve property SecurityGroupIds: [OrderedDict([('Fn::GetAtt', ['SecurityGroup', 'GroupId'])])]. Leaving as is.
2022-03-03 16:22:12,909 | Unable to resolve property Image: OrderedDict([('Fn::GetAtt', ['CustomGetImageResource', 'ImageUri'])]). Leaving as is.
2022-03-03 16:22:12,910 | Found Serverless function with name='DicomParser' and ImageUri='dicomparser:latest'
2022-03-03 16:22:12,910 | --base-dir is not presented, adjusting uri ../ relative to /home/xxxxx/code/xxxxx/dicom-metadata-extractor-serverless-datalake/.aws-sam/build/template.yaml
2022-03-03 16:22:12,911 | --base-dir is not presented, adjusting uri . relative to /home/xxxxx/code/xxxxx/dicom-metadata-extractor-serverless-datalake/.aws-sam/build/template.yaml
2022-03-03 16:22:12,911 | Found Serverless function with name='CustomGetImageFunction' and InlineCode
2022-03-03 16:22:12,911 | Collected default values for parameters: {'ContainerMemory': 1024, 'ContainervCPU': 0.5, 'LambdaMemory': 256, 'LambdaDuration': 600, 'AssignPublicIp': 'ENABLED', 'PartitionKey': 'study_date', 'GlueTableName': 'dicom_metadata', 'LogLevel': 'INFO', 'VersionDescription': 1}
2022-03-03 16:22:12,940 | Unable to resolve property SecurityGroupIds: [OrderedDict([('Fn::GetAtt', ['SecurityGroup', 'GroupId'])])]. Leaving as is.
2022-03-03 16:22:12,940 | Unable to resolve property Image: OrderedDict([('Fn::GetAtt', ['CustomGetImageResource', 'ImageUri'])]). Leaving as is.
2022-03-03 16:22:12,942 | 25 resources found in the stack
2022-03-03 16:22:12,942 | Collected default values for parameters: {'ContainerMemory': 1024, 'ContainervCPU': 0.5, 'LambdaMemory': 256, 'LambdaDuration': 600, 'AssignPublicIp': 'ENABLED', 'PartitionKey': 'study_date', 'GlueTableName': 'dicom_metadata', 'LogLevel': 'INFO', 'VersionDescription': 1}
2022-03-03 16:22:12,970 | Unable to resolve property SecurityGroupIds: [OrderedDict([('Fn::GetAtt', ['SecurityGroup', 'GroupId'])])]. Leaving as is.
2022-03-03 16:22:12,970 | Unable to resolve property Image: OrderedDict([('Fn::GetAtt', ['CustomGetImageResource', 'ImageUri'])]). Leaving as is.
2022-03-03 16:22:12,972 | Found Serverless function with name='DicomParser' and ImageUri='dicomparser:latest'
2022-03-03 16:22:12,972 | --base-dir is not presented, adjusting uri ../ relative to /home/xxxxx/code/xxxxx/dicom-metadata-extractor-serverless-datalake/.aws-sam/build/template.yaml
2022-03-03 16:22:12,973 | --base-dir is not presented, adjusting uri . relative to /home/xxxxx/code/xxxxx/dicom-metadata-extractor-serverless-datalake/.aws-sam/build/template.yaml
2022-03-03 16:22:12,973 | Found Serverless function with name='CustomGetImageFunction' and InlineCode
2022-03-03 16:22:12,973 | No function or layer definition found with code sign config, skipping
        Save arguments to configuration file [Y/n]:
        SAM configuration file [samconfig.toml]:
        SAM configuration environment [default]:

        Looking for resources needed for deployment:
2022-03-03 16:22:37,119 | Managed S3 stack [aws-sam-cli-managed-default] not found. Creating a new one.
        Creating the required resources...
2022-03-03 16:23:07,971 | Failed to create managed resources
Traceback (most recent call last):
  File "/home/xxxxx/.pyenv/versions/3.8.8/lib/python3.8/site-packages/samcli/lib/utils/managed_cloudformation_stack.py", line 107, in _create_or_get_stack
    stack = _create_stack(
  File "/home/xxxxx/.pyenv/versions/3.8.8/lib/python3.8/site-packages/samcli/lib/utils/managed_cloudformation_stack.py", line 180, in _create_stack
    stack_waiter.wait(StackName=stack_id, WaiterConfig={"Delay": 15, "MaxAttempts": 60})
  File "/home/xxxxx/.pyenv/versions/3.8.8/lib/python3.8/site-packages/botocore/waiter.py", line 53, in wait
    Waiter.wait(self, **kwargs)
  File "/home/xxxxx/.pyenv/versions/3.8.8/lib/python3.8/site-packages/botocore/waiter.py", line 350, in wait
    raise WaiterError(
botocore.exceptions.WaiterError: Waiter StackCreateComplete failed: Waiter encountered a terminal failure state: For expression "Stacks[].StackStatus" we matched expected path: "ROLLBACK_COMPLETE" at least once
2022-03-03 16:23:07,980 | Sending Telemetry: {'metrics': [{'commandRun': {'requestId': 'fdde9e9b-eb75-4102-8253-2457803b526a', 'installationId': '0d7e8f01-b186-44ae-8188-455b1d7f2a8e', 'sessionId': '2fbc6fbc-268d-4999-9882-557655d9f720', 'executionEnvironment': 'CLI', 'ci': False, 'pyversion': '3.8.8', 'samcliVersion': '1.30.0', 'awsProfileProvided': False, 'debugFlagProvided': True, 'region': '', 'commandName': 'sam deploy', 'duration': 372405, 'exitReason': 'ManagedStackError', 'exitCode': 1}}]}
2022-03-03 16:23:08,751 | HTTPSConnectionPool(host='aws-serverless-tools-telemetry.us-west-2.amazonaws.com', port=443): Read timed out. (read timeout=0.1)
Error: Failed to create managed resources: Waiter StackCreateComplete failed: Waiter encountered a terminal failure state: For expression "Stacks[].StackStatus" we matched expected path: "ROLLBACK_COMPLETE" at least once

The cloud trail logs look as if the bucket was created.

so why would sam or cloud formation not be able to determine that the bucket was created?

[dazzag24]

I wonder if I am hitting this issue: aws/aws-sam-cli#3243 (comment)

[dazzag24]

I upgraded from SAM CLI v1.30.0 to 1.40.1 and the issue persists

[RajanTpss]

Hello folks, hope you are doing great.

Please help me in this problem. Facing the below problem while running on sam deploy.

Current Sam version : 1.53.0

Solution tried by me :

I deleted the stack using "sam delete --stack-name aws-sam-cli-managed-default" and confirmed it was removed in the UI and in the cli output. After that I tried again and got the same error.

I have also applied the below solution but it didn't worked for me :
aws/aws-sam-cli#3243 (comment)

Please help me if anyone can help.

Thank You

[hemandee]

I have updated the code to reflect newer SAM CLI workflow and SAM template, does that help get pass that error?

[RajanTpss]

I have updated the code to reflect newer SAM CLI workflow and SAM template, does that help get pass that error?

Nope, Still Facing same error.

Now, SAM CLI, version 1.41.0

Uninstalled the Previous Version (SAM CLI) and Installed again. Followed the same pattern as defined above in my comment. But the still facing the same error :( .

Your little help would be greatly appreciated.

Thanks