Deploy command freezes when tag containing space is provided in samconfig file

[jbutz]

Description:

There appears to be a situation where deployment either freezes and does not proceed, or takes an exceptionally long time to proceed, based on the tag values provided. For me it appears to be based on the presence of a space in a tag value being passed through the samconfig.toml file. The same issue appeared to occur when I tried to use a YAML samconfig file as well.

Steps to reproduce:

1. Initialize a new SAM App, I used Node v20 Hello World Example
2. Deploy the app without any changes
3. Add the following under [default.deploy.parameters] in the samconfig.toml file

   tags = [
       "ProjectName=sample-aws-sam-debugging-app",
       "SystemCustodian=john.doe@example.com",
       "SystemOwner=alice.public@example.com",
       "PrimaryItContact=john.doe@example.com",
       "CostCenter=1000000",
       "CostCenterApprover=jane.doe@example.com",
       'Level1BusinessArea="Digital Core"',
       "DataClassification=125796",
       "Hipaa=No",
   ]

4. Deploy the application again, it appears to freeze. ctrl + c to abort the deploy
5. Remove the space in Digital Core
6. Deploy the application again, the deploy will proceed at the expected rate

Observed result:

> sam deploy --region us-east-2 --debug
2024-02-06 12:52:10,544 | Config file location: /Users/jbutz/workspace/aws-sam-debug/sam-app/samconfig.toml                                                                                                                   
2024-02-06 12:52:10,546 | Loading configuration values from [default.['deploy'].parameters] (env.command_name.section) in config file at '/Users/jbutz/workspace/aws-sam-debug/sam-app/samconfig.toml'...                     
2024-02-06 12:52:10,547 | Configuration values successfully loaded.                                                                                                                                                           
2024-02-06 12:52:10,547 | Configuration values are: {'stack_name': 'sam-app', 'capabilities': 'CAPABILITY_IAM', 'confirm_changeset': True, 'resolve_s3': True, 'tags': ['ProjectName=sample-aws-sam-debugging-app',           
'SystemCustodian=john.doe@example.com', 'SystemOwner=alice.public@example.com', 'PrimaryItContact=john.doe@example.com', 'CostCenter=1000000', 'CostCenterApprover=jane.doe@example.com', 'Level1BusinessArea="Digital Core"',
'DataClassification=125796', 'Hipaa=No']}                                                                                                                                                                                     
2024-02-06 12:52:10,551 | Using SAM Template at /Users/jbutz/workspace/aws-sam-debug/sam-app/template.yaml

The deploy command appears to freeze here. Removing the space, I receive:

> sam deploy --region us-east-2 --debug
2024-02-06 12:53:23,234 | Config file location: /Users/jbutz/workspace/aws-sam-debug/sam-app/samconfig.toml                                                                                                                   
2024-02-06 12:53:23,236 | Loading configuration values from [default.['deploy'].parameters] (env.command_name.section) in config file at '/Users/jbutz/workspace/aws-sam-debug/sam-app/samconfig.toml'...                     
2024-02-06 12:53:23,237 | Configuration values successfully loaded.                                                                                                                                                           
2024-02-06 12:53:23,237 | Configuration values are: {'stack_name': 'sam-app', 'capabilities': 'CAPABILITY_IAM', 'confirm_changeset': True, 'resolve_s3': True, 'tags': ['ProjectName=sample-aws-sam-debugging-app',           
'SystemCustodian=john.doe@example.com', 'SystemOwner=alice.public@example.com', 'PrimaryItContact=john.doe@example.com', 'CostCenter=1000000', 'CostCenterApprover=jane.doe@example.com', 'Level1BusinessArea="DigitalCore"', 
'DataClassification=125796', 'Hipaa=No']}                                                                                                                                                                                     
2024-02-06 12:53:23,241 | Using SAM Template at /Users/jbutz/workspace/aws-sam-debug/sam-app/template.yaml                                                                                                                    
2024-02-06 12:53:23,270 | Using config file: samconfig.toml, config environment: default                                                                                                                                      
2024-02-06 12:53:23,271 | Expand command line arguments to:                                                                                                                                                                   
2024-02-06 12:53:23,271 | --template_file=/Users/jbutz/workspace/aws-sam-debug/sam-app/template.yaml --fail_on_empty_changeset --confirm_changeset --on_failure=ROLLBACK --stack_name=sam-app --resolve_s3                    
--tags={'ProjectName': 'sample-aws-sam-debugging-app', 'SystemCustodian': 'john.doe@example.com', 'SystemOwner': 'alice.public@example.com', 'PrimaryItContact': 'john.doe@example.com', 'CostCenter': '1000000',             
'CostCenterApprover': 'jane.doe@example.com', 'Level1BusinessArea': 'DigitalCore', 'DataClassification': '125796', 'Hipaa': 'No'} --capabilities=['CAPABILITY_IAM']                                                           

                Managed S3 bucket: aws-sam-cli-managed-default-samclisourcebucket-CHANGED
                A different default S3 bucket can be set in samconfig.toml
                Or by specifying --s3-bucket explicitly.
2024-02-06 12:53:23,675 | No Parameters detected in the template                                                                                                                                                              
2024-02-06 12:53:23,694 | There is no customer defined id or cdk path defined for resource HelloWorldFunction, so we will use the resource logical id as the resource id                                                      
2024-02-06 12:53:23,694 | There is no customer defined id or cdk path defined for resource ServerlessRestApi, so we will use the resource logical id as the resource id                                                       
2024-02-06 12:53:23,695 | 0 stacks found in the template                                                                                                                                                                      
2024-02-06 12:53:23,696 | No Parameters detected in the template                                                                                                                                                              
2024-02-06 12:53:23,706 | There is no customer defined id or cdk path defined for resource HelloWorldFunction, so we will use the resource logical id as the resource id                                                      
2024-02-06 12:53:23,707 | There is no customer defined id or cdk path defined for resource ServerlessRestApi, so we will use the resource logical id as the resource id                                                       
2024-02-06 12:53:23,841 | There is no customer defined id or cdk path defined for resource HelloWorldFunction, so we will use the resource logical id as the resource id                                                      
2024-02-06 12:53:23,842 | Sam customer defined id is more priority than other IDs. Customer defined id for resource HelloWorldFunction is HelloWorldFunction                                                                  
2024-02-06 12:53:24,017 | File with same data already exists at 5e7cb79c83de175ad43551d700ba93e9, skipping upload                                                                                                             

        Deploying with following values
        ===============================
        Stack name                   : sam-app
        Region                       : us-east-2
        Confirm changeset            : True
        Disable rollback             : False
        Deployment s3 bucket         : aws-sam-cli-managed-default-samclisourcebucket-CHANGED
        Capabilities                 : ["CAPABILITY_IAM"]
        Parameter overrides          : {}
        Signing Profiles             : {}

Initiating deployment
=====================

2024-02-06 12:53:24,042 | No Parameters detected in the template                                                                                                                                                              
2024-02-06 12:53:24,059 | Sam customer defined id is more priority than other IDs. Customer defined id for resource HelloWorldFunction is HelloWorldFunction                                                                  
2024-02-06 12:53:24,059 | There is no customer defined id or cdk path defined for resource ServerlessRestApi, so we will use the resource logical id as the resource id                                                       
2024-02-06 12:53:24,060 | 0 stacks found in the template                                                                                                                                                                      
2024-02-06 12:53:24,061 | No Parameters detected in the template                                                                                                                                                              
2024-02-06 12:53:24,074 | Sam customer defined id is more priority than other IDs. Customer defined id for resource HelloWorldFunction is HelloWorldFunction                                                                  
2024-02-06 12:53:24,075 | There is no customer defined id or cdk path defined for resource ServerlessRestApi, so we will use the resource logical id as the resource id                                                       
2024-02-06 12:53:24,076 | 2 resources found in the stack                                                                                                                                                                      
2024-02-06 12:53:24,539 | File with same data already exists at 9cfe222419be2175cb21f0973acef299.template, skipping upload                                                                                                    


Waiting for changeset to be created..

and the deploy command continues from there

Expected result:

I expect the deploy command to be able to function with tags that contain spaced being provided through the samconfig file.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

I have experienced this both on my local Mac and on a GitHub Codespace.

Local Machine

{
  "version": "1.108.0",
  "system": {
    "python": "3.8.13",
    "os": "macOS-14.3-arm64-arm-64bit"
  },
  "additional_dependencies": {
    "docker_engine": "24.0.7",
    "aws_cdk": "Not available",
    "terraform": "Not available"
  },
  "available_beta_feature_env_vars": [
    "SAM_CLI_BETA_FEATURES",
    "SAM_CLI_BETA_BUILD_PERFORMANCE",
    "SAM_CLI_BETA_TERRAFORM_SUPPORT",
    "SAM_CLI_BETA_RUST_CARGO_LAMBDA"
  ]
}

GitHub Codespace

{
  "version": "1.108.0",
  "system": {
    "python": "3.11.3",
    "os": "Linux-6.2.0-1018-azure-x86_64-with-glibc2.31"
  },
  "additional_dependencies": {
    "docker_engine": "24.0.7-1",
    "aws_cdk": "Not available",
    "terraform": "Not available"
  },
  "available_beta_feature_env_vars": [
    "SAM_CLI_BETA_FEATURES",
    "SAM_CLI_BETA_BUILD_PERFORMANCE",
    "SAM_CLI_BETA_TERRAFORM_SUPPORT",
    "SAM_CLI_BETA_RUST_CARGO_LAMBDA"
  ]
}

[hawflau]

Hey @jbutz thanks for reporting the issue. I tried the steps you provided. I found out that the process did not hang, but just took a significant amount of time:

2024-02-07 14:45:55,684 | Loading configuration values from [default.['deploy'].parameters] (env.command_name.section) in config file at
'/Users/lauwing/playground/202402/sam-app/samconfig.toml'...
2024-02-07 14:45:55,686 | Configuration values successfully loaded.
2024-02-07 14:45:55,687 | Configuration values are: {'stack_name': 'sam-app', 'capabilities': 'CAPABILITY_IAM', 'confirm_changeset': True,
'resolve_s3': True, 'tags': ['ProjectName=sample-aws-sam-debugging-app', 'SystemCustodian=john.doe@example.com',
'SystemOwner=alice.public@example.com', 'PrimaryItContact=john.doe@example.com', 'CostCenter=1000000',
'CostCenterApprover=jane.doe@example.com', 'Level1BusinessArea="Digital Core"', 'DataClassification=125796', 'Hipaa=No']}
2024-02-07 14:45:55,695 | Using SAM Template at /Users/lauwing/playground/202402/sam-app/.aws-sam/build/template.yaml
2024-02-07 14:47:07,850 | Using config file: samconfig.toml, config environment: default

From my example, it took slight more than a minute for the last line to show up. f I reduced the number of tags, the wait time significantly improved.

It seems like a performance issue. I have yet to locate where and what cause this perform. I'll mark this a bug and continue to look into it.

Meanwhile, can you also check if the command gets faster after removing any one tag?

[hawflau]

The performance issue came from regex parsing on this line, whereas the pattern used is '(\\"(?:\\\\[A-Za-z0-9\\"_:\\.\\/\\+-\\@=]|[^\\"\\\\]+)*\\"|\'(?:\\\\[A-Za-z0-9\\"_:\\.\\/\\+-\\@=]|[^\'\\\\]+)*\'|(?:\\\\[A-Za-z0-9\\"_:\\.\\/\\+-\\@=]|[^ \\"\\\\]+)+)=(\\"(?:\\\\[A-Za-z0-9\\"_:\\.\\/\\+-\\@=]|[^\\"\\\\]+)*\\"|\'(?:\\\\[A-Za-z0-9\\"_:\\.\\/\\+-\\@=]|[^\'\\\\]+)*\'|(?:\\\\[A-Za-z0-9\\"_:\\.\\/\\+-\\@=]|[^ \\"\\\\]+)+)'

When the space in the tag value is removed, it avoided the regex parsing and thus was much faster.

[github-actions]

Patch is released in v1.117.0. Closing