Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug: Templates that depend on number-to-string coalescing do not compile in all languages #615

Open
comcalvi opened this issue Mar 19, 2024 · 1 comment

Comments

@comcalvi
Copy link
Contributor

comcalvi commented Mar 19, 2024

Summary

In TS, templates that pass numbers where strings are expected causes a type error. This is arguably the right behavior, since the types don't match, but CFN allows passing 1 where '1' is expected.

The emitted Java code includes string literals that aren't properly escaped, or possibly just missing terminating / opening quotes. Not sure which.

CSharp does not like multi-line strings.

Go and Python are failing with type errors that are not obvious.

Unclear if the number-to-string coalescing is the root cause of this for all languages, or if something else is happening.

Reproduction Steps

  1. Copy the JSON template at the end of this report
  2. Create a new end-to-end test that uses that template
  3. unset CREATE_CFN_STACK
  4. cargo test --test end-to-end efs -- --nocapture
failures:
    efs::csharp
    efs::golang
    efs::java
    efs::python
    efs::typescript

TypeScript

Fails with

Type 'number' is not assignable to type 'string'.ts(2322)

This is caused by this code:

  public constructor(scope: cdk.App, id: string, props: EfsStackProps = {}) {
    super(scope, id, props);

    // Applying default props
    props = {
      ...props,
      // ... 
      asgMaxSize: props.asgMaxSize ?? 2, //   Type 'number' is not assignable to type 'string'.ts(2322)
    };

This is happening because in the JSON, we have:

  "AsgMaxSize": {
      "Type": "Number",
      "Description": "Maximum size and initial desired capacity of Auto Scaling Group",
      "Default": "2"
    },

This is passed here:

    "AutoScalingGroup": {
      "Type": "AWS::AutoScaling::AutoScalingGroup",
      "Properties": {
        "MinSize": "1",
        "MaxSize": { "Ref": "AsgMaxSize" }, // MaxSize is a string property

It's debatable whether this is really a bug. cdk-from-cfn is doing the right thing, this only works because CFN allows you to pass a string where a number is expected.

Java

Fails with unenclosed string literals:

java setup
cdk synth
[ERROR] COMPILATION ERROR : 
[ERROR] /Users/comcalvi/Documents/cdk-from-cfn/tests/end-to-end/efs-java-working-dir/src/main/java/com/myorg/Stack.java:[229,25] unclosed string literal
[ERROR] /Users/comcalvi/Documents/cdk-from-cfn/tests/end-to-end/efs-java-working-dir/src/main/java/com/myorg/Stack.java:[230,25] unclosed string literal
[ERROR] /Users/comcalvi/Documents/cdk-from-cfn/tests/end-to-end/efs-java-working-dir/src/main/java/com/myorg/Stack.java:[231,25] unclosed string literal
[ERROR] /Users/comcalvi/Documents/cdk-from-cfn/tests/end-to-end/efs-java-working-dir/src/main/java/com/myorg/Stack.java:[232,25] unclosed string literal
[ERROR] /Users/comcalvi/Documents/cdk-from-cfn/tests/end-to-end/efs-java-working-dir/src/main/java/com/myorg/Stack.java:[235,44] ';' expected
[ERROR] /Users/comcalvi/Documents/cdk-from-cfn/tests/end-to-end/efs-java-working-dir/src/main/java/com/myorg/Stack.java:[239,41] ';' expected
[ERROR] /Users/comcalvi/Documents/cdk-from-cfn/tests/end-to-end/efs-java-working-dir/src/main/java/com/myorg/Stack.java:[240,25] unclosed string literal
[ERROR] /Users/comcalvi/Documents/cdk-from-cfn/tests/end-to-end/efs-java-working-dir/src/main/java/com/myorg/Stack.java:[241,25] unclosed string literal
[ERROR] /Users/comcalvi/Documents/cdk-from-cfn/tests/end-to-end/efs-java-working-dir/src/main/java/com/myorg/Stack.java:[242,25] unclosed string literal
[ERROR] /Users/comcalvi/Documents/cdk-from-cfn/tests/end-to-end/efs-java-working-dir/src/main/java/com/myorg/Stack.java:[243,25] unclosed string literal
[ERROR] /Users/comcalvi/Documents/cdk-from-cfn/tests/end-to-end/efs-java-working-dir/src/main/java/com/myorg/Stack.java:[246,44] ';' expected
[ERROR] /Users/comcalvi/Documents/cdk-from-cfn/tests/end-to-end/efs-java-working-dir/src/main/java/com/myorg/Stack.java:[249,41] ';' expected

The errors are visible with syntax highlighting starting near line 254:

    launchConfiguration.addMetadata("AWS::CloudFormation::Init", Map.of("configSets", Map.of("MountConfig", Arrays.asList(
                "setup",
                "mount")),
        "setup", Map.of("packages", Map.of("yum", Map.of("nfs-utils", Arrays.asList(
        )),
        "files", Map.of("/home/ec2-user/post_nfsstat", Map.of("content", String.join("", // these opening parens are in red
                "#!/bin/bash
                ",
                "
                ",
                "INPUT="$(cat)"
                ",
                "CW_JSON_OPEN='{ "Namespace": "EFS", "MetricData": [ ' // highlighting is broken at this point
                ",

Python

Fails with:

pip install
cdk synth

===== cdk synth stdout ===== END =======

===== cdk synth stderr ===== START =====
Traceback (most recent call last):
  File "/Users/comcalvi/Documents/cdk-from-cfn/tests/end-to-end/efs-python-working-dir/app.py", line 10, in <module>
    EfsStack(app, 'Stack')
  File "/Users/comcalvi/Documents/cdk-from-cfn/tests/end-to-end/efs-python-working-dir/.venv/lib/python3.12/site-packages/jsii/_runtime.py", line 118, in __call__
    inst = super(JSIIMeta, cast(JSIIMeta, cls)).__call__(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/comcalvi/Documents/cdk-from-cfn/tests/end-to-end/efs-python-working-dir/stack.py", line 409, in __init__
    image_id = awsRegionArch2Ami[self.region][awsInstanceType2Arch[props['instanceType']]['Arch']],
               ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
KeyError: '${Token[AWS.Region.13]}'

No idea what causes this. It's trying to index a dictionary with an unresolved Token.

CSharp

Doesn't like newlines appearing in string literals:

===== cdk synth stdout ===== START =====
/Users/comcalvi/Documents/cdk-from-cfn/tests/end-to-end/efs-csharp-working-dir/Stack.cs(328,21): error CS1010: Newline in constant [/Users/comcalvi/Documents/cdk-from-cfn/tests/end-to-end/efs-csharp-working-dir/CSharp.csproj]
/// ...

Caused by:

  var launchConfiguration = new CfnLaunchConfiguration(this, "LaunchConfiguration", new CfnLaunchConfigurationProps
            {
                AssociatePublicIpAddress = true,
                ImageId = awsRegionArch2Ami[Region][awsInstanceType2Arch[props.InstanceType]["Arch"]],
                InstanceType = props.InstanceType,
                SecurityGroups = new []
                {
                    instanceSecurityGroup.Ref,
                },
                IamInstanceProfile = cloudWatchPutMetricsInstanceProfile.Ref,
                UserData = Fn.Base64(string.Join("", new []
                {
                    "#!/bin/bash -xe // these are newlines in the literal
                    ",
                    "yum install -y aws-cfn-bootstrap
                    ",
                    "/opt/aws/bin/cfn-init -v ",
                    "         --stack ",

Go

Fails with

===== cdk synth stderr ===== START =====
# command-line-arguments
./stack.go:308:20: cannot use &[]interface{}{…} (value of type *[]interface{}) as *[]*awsefs.CfnFileSystem_ElasticFileSystemTagProperty value in struct literal
./stack.go:309:6: undefined: ElasticFileSystemTagProperty
./stack.go:410:6: undefined: IngressProperty
./stack.go:416:6: undefined: IngressProperty

It's mad about this code:


	fileSystem := efs.NewCfnFileSystem(
		stack,
		jsii.String("FileSystem"),
		&efs.CfnFileSystemProps{
			PerformanceMode: jsii.String("generalPurpose"),
			FileSystemTags: &[]interface{}{
				&ElasticFileSystemTagProperty{
					Key: jsii.String("Name"),
					Value: props.VolumeName,
				},
			},
		},
	)

Full JSON template

This is a modified form of this sample template. It is identical, except I've removed the SSH KeyPair, because it's a resource that this template expects to exist and that makes this harder to reproduce.

{
  "AWSTemplateFormatVersion": "2010-09-09",
  "Description": "This template creates an Amazon EFS file system and mount target and associates it with Amazon EC2 instances in an Auto Scaling group. **WARNING** This template creates Amazon EC2 instances and related resources. You will be billed for the AWS resources used if you create a stack from this template.",
  "Parameters": {
    "InstanceType" : {
      "Description" : "WebServer EC2 instance type",
      "Type" : "String",
      "Default" : "t2.small",
      "AllowedValues" : [ 
        "t1.micro", 
        "t2.nano", 
        "t2.micro", 
        "t2.small", 
        "t2.medium", 
        "t2.large", 
        "m1.small", 
        "m1.medium", 
        "m1.large", 
        "m1.xlarge", 
        "m2.xlarge", 
        "m2.2xlarge", 
        "m2.4xlarge", 
        "m3.medium", 
        "m3.large", 
        "m3.xlarge", 
        "m3.2xlarge", 
        "m4.large", 
        "m4.xlarge", 
        "m4.2xlarge", 
        "m4.4xlarge", 
        "m4.10xlarge", 
        "c1.medium", 
        "c1.xlarge", 
        "c3.large", 
        "c3.xlarge", 
        "c3.2xlarge", 
        "c3.4xlarge", 
        "c3.8xlarge", 
        "c4.large", 
        "c4.xlarge", 
        "c4.2xlarge", 
        "c4.4xlarge", 
        "c4.8xlarge", 
        "g2.2xlarge", 
        "g2.8xlarge", 
        "r3.large", 
        "r3.xlarge", 
        "r3.2xlarge", 
        "r3.4xlarge", 
        "r3.8xlarge", 
        "i2.xlarge", 
        "i2.2xlarge", 
        "i2.4xlarge", 
        "i2.8xlarge", 
        "d2.xlarge", 
        "d2.2xlarge", 
        "d2.4xlarge", 
        "d2.8xlarge", 
        "hi1.4xlarge", 
        "hs1.8xlarge", 
        "cr1.8xlarge", 
        "cc2.8xlarge", 
        "cg1.4xlarge"
      ],
      "ConstraintDescription" : "must be a valid EC2 instance type."
    },
    "AsgMaxSize": {
      "Type": "Number",
      "Description": "Maximum size and initial desired capacity of Auto Scaling Group",
      "Default": "2"
    },
    "SSHLocation" : {
      "Description" : "The IP address range that can be used to connect to the EC2 instances by using SSH",
      "Type": "String",
      "MinLength": "9",
      "MaxLength": "18",
      "Default": "0.0.0.0/0",
      "AllowedPattern": "(\\d{1,3})\\.(\\d{1,3})\\.(\\d{1,3})\\.(\\d{1,3})/(\\d{1,2})",
      "ConstraintDescription": "must be a valid IP CIDR range of the form x.x.x.x/x."
    },
    "VolumeName" : {
      "Description" : "The name to be used for the EFS volume",
      "Type": "String",
      "MinLength": "1",
      "Default": "myEFSvolume"
    },
    "MountPoint" : {
      "Description" : "The Linux mount point for the EFS volume",
      "Type": "String",
      "MinLength": "1",
      "Default": "myEFSvolume"
    }
  },
  "Mappings" : {
    "AWSInstanceType2Arch" : {
      "t1.micro"    : { "Arch" : "HVM64"  },
      "t2.nano"     : { "Arch" : "HVM64"  },
      "t2.micro"    : { "Arch" : "HVM64"  },
      "t2.small"    : { "Arch" : "HVM64"  },
      "t2.medium"   : { "Arch" : "HVM64"  },
      "t2.large"    : { "Arch" : "HVM64"  },
      "m1.small"    : { "Arch" : "HVM64"  },
      "m1.medium"   : { "Arch" : "HVM64"  },
      "m1.large"    : { "Arch" : "HVM64"  },
      "m1.xlarge"   : { "Arch" : "HVM64"  },
      "m2.xlarge"   : { "Arch" : "HVM64"  },
      "m2.2xlarge"  : { "Arch" : "HVM64"  },
      "m2.4xlarge"  : { "Arch" : "HVM64"  },
      "m3.medium"   : { "Arch" : "HVM64"  },
      "m3.large"    : { "Arch" : "HVM64"  },
      "m3.xlarge"   : { "Arch" : "HVM64"  },
      "m3.2xlarge"  : { "Arch" : "HVM64"  },
      "m4.large"    : { "Arch" : "HVM64"  },
      "m4.xlarge"   : { "Arch" : "HVM64"  },
      "m4.2xlarge"  : { "Arch" : "HVM64"  },
      "m4.4xlarge"  : { "Arch" : "HVM64"  },
      "m4.10xlarge" : { "Arch" : "HVM64"  },
      "c1.medium"   : { "Arch" : "HVM64"  },
      "c1.xlarge"   : { "Arch" : "HVM64"  },
      "c3.large"    : { "Arch" : "HVM64"  },
      "c3.xlarge"   : { "Arch" : "HVM64"  },
      "c3.2xlarge"  : { "Arch" : "HVM64"  },
      "c3.4xlarge"  : { "Arch" : "HVM64"  },
      "c3.8xlarge"  : { "Arch" : "HVM64"  },
      "c4.large"    : { "Arch" : "HVM64"  },
      "c4.xlarge"   : { "Arch" : "HVM64"  },
      "c4.2xlarge"  : { "Arch" : "HVM64"  },
      "c4.4xlarge"  : { "Arch" : "HVM64"  },
      "c4.8xlarge"  : { "Arch" : "HVM64"  },
      "g2.2xlarge"  : { "Arch" : "HVMG2"  },
      "g2.8xlarge"  : { "Arch" : "HVMG2"  },
      "r3.large"    : { "Arch" : "HVM64"  },
      "r3.xlarge"   : { "Arch" : "HVM64"  },
      "r3.2xlarge"  : { "Arch" : "HVM64"  },
      "r3.4xlarge"  : { "Arch" : "HVM64"  },
      "r3.8xlarge"  : { "Arch" : "HVM64"  },
      "i2.xlarge"   : { "Arch" : "HVM64"  },
      "i2.2xlarge"  : { "Arch" : "HVM64"  },
      "i2.4xlarge"  : { "Arch" : "HVM64"  },
      "i2.8xlarge"  : { "Arch" : "HVM64"  },
      "d2.xlarge"   : { "Arch" : "HVM64"  },
      "d2.2xlarge"  : { "Arch" : "HVM64"  },
      "d2.4xlarge"  : { "Arch" : "HVM64"  },
      "d2.8xlarge"  : { "Arch" : "HVM64"  },
      "hi1.4xlarge" : { "Arch" : "HVM64"  },
      "hs1.8xlarge" : { "Arch" : "HVM64"  },
      "cr1.8xlarge" : { "Arch" : "HVM64"  },
      "cc2.8xlarge" : { "Arch" : "HVM64"  }
    },
    "AWSRegionArch2AMI" : {
      "us-east-1"        : {"HVM64" : "ami-0ff8a91507f77f867", "HVMG2" : "ami-0a584ac55a7631c0c"},
      "us-west-2"        : {"HVM64" : "ami-a0cfeed8", "HVMG2" : "ami-0e09505bc235aa82d"},
      "us-west-1"        : {"HVM64" : "ami-0bdb828fd58c52235", "HVMG2" : "ami-066ee5fd4a9ef77f1"},
      "eu-west-1"        : {"HVM64" : "ami-047bb4163c506cd98", "HVMG2" : "ami-0a7c483d527806435"},
      "eu-west-2"        : {"HVM64" : "ami-f976839e", "HVMG2" : "NOT_SUPPORTED"},
      "eu-west-3"        : {"HVM64" : "ami-0ebc281c20e89ba4b", "HVMG2" : "NOT_SUPPORTED"},
      "eu-central-1"     : {"HVM64" : "ami-0233214e13e500f77", "HVMG2" : "ami-06223d46a6d0661c7"},
      "ap-northeast-1"   : {"HVM64" : "ami-06cd52961ce9f0d85", "HVMG2" : "ami-053cdd503598e4a9d"},
      "ap-northeast-2"   : {"HVM64" : "ami-0a10b2721688ce9d2", "HVMG2" : "NOT_SUPPORTED"},
      "ap-northeast-3"   : {"HVM64" : "ami-0d98120a9fb693f07", "HVMG2" : "NOT_SUPPORTED"},
      "ap-southeast-1"   : {"HVM64" : "ami-08569b978cc4dfa10", "HVMG2" : "ami-0be9df32ae9f92309"},
      "ap-southeast-2"   : {"HVM64" : "ami-09b42976632b27e9b", "HVMG2" : "ami-0a9ce9fecc3d1daf8"},
      "ap-south-1"       : {"HVM64" : "ami-0912f71e06545ad88", "HVMG2" : "ami-097b15e89dbdcfcf4"},
      "us-east-2"        : {"HVM64" : "ami-0b59bfac6be064b78", "HVMG2" : "NOT_SUPPORTED"},
      "ca-central-1"     : {"HVM64" : "ami-0b18956f", "HVMG2" : "NOT_SUPPORTED"},
      "sa-east-1"        : {"HVM64" : "ami-07b14488da8ea02a0", "HVMG2" : "NOT_SUPPORTED"},
      "cn-north-1"       : {"HVM64" : "ami-0a4eaf6c4454eda75", "HVMG2" : "NOT_SUPPORTED"},
      "cn-northwest-1"   : {"HVM64" : "ami-6b6a7d09", "HVMG2" : "NOT_SUPPORTED"}
    }
  },
  "Resources": {
    "FileSystemNoProps": {
      "Type": "AWS::EFS::FileSystem"
    },
    "CloudWatchPutMetricsRole" : {
      "Type"  : "AWS::IAM::Role",
      "Properties" : {
          "AssumeRolePolicyDocument" : {
              "Statement" : [ {
                  "Effect" : "Allow",
                  "Principal" : {
                      "Service" : [ "ec2.amazonaws.com" ]
                  },
                  "Action" : [ "sts:AssumeRole" ]
              } ]
          },
          "Path" : "/"
      }
    },
    "CloudWatchPutMetricsRolePolicy" : {
        "Type" : "AWS::IAM::Policy",
        "Properties" : {
            "PolicyName" : "CloudWatch_PutMetricData",
            "PolicyDocument" : {
              "Version": "2012-10-17",
              "Statement": [
                {
                  "Sid": "CloudWatchPutMetricData",
                  "Effect": "Allow",
                  "Action": ["cloudwatch:PutMetricData"],
                  "Resource": ["*"]
                }
              ]
            },
            "Roles" : [ { "Ref" : "CloudWatchPutMetricsRole" } ]
        }
    },
    "CloudWatchPutMetricsInstanceProfile" : {
      "Type" : "AWS::IAM::InstanceProfile",
      "Properties" : {
        "Path" : "/",
        "Roles" : [ { "Ref" : "CloudWatchPutMetricsRole" } ]
      }
    },
    "VPC": {
      "Type": "AWS::EC2::VPC",
      "Properties": {
        "EnableDnsSupport" : "true",
        "EnableDnsHostnames" : "true",
        "CidrBlock": "10.0.0.0/16",
        "Tags": [ {"Key": "Application", "Value": { "Ref": "AWS::StackId"} } ]
      }
    },
    "InternetGateway" : {
      "Type" : "AWS::EC2::InternetGateway",
      "Properties" : {
        "Tags" : [
          { "Key" : "Application", "Value" : { "Ref" : "AWS::StackName" } },
          { "Key" : "Network", "Value" : "Public" }
        ]
      }
    },
    "GatewayToInternet" : {
      "Type" : "AWS::EC2::VPCGatewayAttachment",
      "Properties" : {
        "VpcId" : { "Ref" : "VPC" },
        "InternetGatewayId" : { "Ref" : "InternetGateway" }
      }
    },
    "RouteTable":{
      "Type":"AWS::EC2::RouteTable",
      "Properties":{
        "VpcId": {"Ref":"VPC"}
      }
    },
    "SubnetRouteTableAssoc": {
      "Type" : "AWS::EC2::SubnetRouteTableAssociation",
      "Properties" : {
        "RouteTableId" : {"Ref":"RouteTable"},
        "SubnetId" : {"Ref":"Subnet"}
      }
    },
    "InternetGatewayRoute": {
        "Type":"AWS::EC2::Route",
        "Properties":{
            "DestinationCidrBlock":"0.0.0.0/0",
            "RouteTableId":{"Ref":"RouteTable"},
            "GatewayId":{"Ref":"InternetGateway"}
        }
    },
    "Subnet": {
      "Type": "AWS::EC2::Subnet",
      "Properties": {
        "VpcId": { "Ref": "VPC" },
        "CidrBlock": "10.0.0.0/24",
        "Tags": [ { "Key": "Application", "Value": { "Ref": "AWS::StackId" } } ]
      }
    },    
    "InstanceSecurityGroup": {
      "Type": "AWS::EC2::SecurityGroup",
      "Properties": {
        "VpcId": { "Ref": "VPC" },
        "GroupDescription": "Enable SSH access via port 22",
        "SecurityGroupIngress": [
          { "IpProtocol": "tcp", "FromPort": 22, "ToPort": 22, "CidrIp": { "Ref": "SSHLocation" } },
          { "IpProtocol": "tcp", "FromPort": 80, "ToPort": 80, "CidrIp": "0.0.0.0/0" }
         ]
      }
    },
    "MountTargetSecurityGroup": {
      "Type": "AWS::EC2::SecurityGroup",
      "Properties": {
        "VpcId": { "Ref": "VPC" },
        "GroupDescription": "Security group for mount target",
        "SecurityGroupIngress": [
          {
            "IpProtocol": "tcp",
            "FromPort": 2049,
            "ToPort": 2049,
            "CidrIp": "0.0.0.0/0"
          }
        ]
      }
    },
    "FileSystem": {
      "Type": "AWS::EFS::FileSystem",
      "Properties": {
        "PerformanceMode": "generalPurpose",
        "FileSystemTags": [
          {
            "Key": "Name",
            "Value": { "Ref" : "VolumeName" }
          }
        ]
      }
    },
    "MountTarget": {
      "Type": "AWS::EFS::MountTarget",
      "Properties": {
        "FileSystemId": { "Ref": "FileSystem" },
        "SubnetId": { "Ref": "Subnet" },
        "SecurityGroups": [ { "Ref": "MountTargetSecurityGroup" } ]        
      }
    },
    "LaunchConfiguration": {
      "Type": "AWS::AutoScaling::LaunchConfiguration",
      "Metadata" : {
        "AWS::CloudFormation::Init" : {
          "configSets" : {
            "MountConfig" : [ "setup", "mount" ]
          },
          "setup" : {
            "packages" : {
              "yum" : {
                "nfs-utils" : []
              }
            },
            "files" : {
              "/home/ec2-user/post_nfsstat" : {
                "content" : { "Fn::Join" : [ "", [
                      "#!/bin/bash\n",
                      "\n",
                      "INPUT=\"$(cat)\"\n",
                      "CW_JSON_OPEN='{ \"Namespace\": \"EFS\", \"MetricData\": [ '\n",
                      "CW_JSON_CLOSE=' ] }'\n",
                      "CW_JSON_METRIC=''\n",
                      "METRIC_COUNTER=0\n",
                      "\n",
                      "for COL in 1 2 3 4 5 6; do\n",
                      "\n",
                      " COUNTER=0\n",
                      " METRIC_FIELD=$COL\n",
                      " DATA_FIELD=$(($COL+($COL-1)))\n",
                      "\n",
                      " while read line; do\n",
                      "   if [[ COUNTER -gt 0 ]]; then\n",
                      "\n",
                      "     LINE=`echo $line | tr -s ' ' `\n",
                      "     AWS_COMMAND=\"aws cloudwatch put-metric-data --region ", { "Ref": "AWS::Region" }, "\"\n",
                      "     MOD=$(( $COUNTER % 2))\n",
                      "\n",
                      "     if [ $MOD -eq 1 ]; then\n",
                      "       METRIC_NAME=`echo $LINE | cut -d ' ' -f $METRIC_FIELD`\n",
                      "     else\n",
                      "       METRIC_VALUE=`echo $LINE | cut -d ' ' -f $DATA_FIELD`\n",
                      "     fi\n",
                      "\n",
                      "     if [[ -n \"$METRIC_NAME\" && -n \"$METRIC_VALUE\" ]]; then\n",
                      "       INSTANCE_ID=$(curl -s http://169.254.169.254/latest/meta-data/instance-id)\n",
                      "       CW_JSON_METRIC=\"$CW_JSON_METRIC { \\\"MetricName\\\": \\\"$METRIC_NAME\\\", \\\"Dimensions\\\": [{\\\"Name\\\": \\\"InstanceId\\\", \\\"Value\\\": \\\"$INSTANCE_ID\\\"} ], \\\"Value\\\": $METRIC_VALUE },\"\n",
                      "       unset METRIC_NAME\n",
                      "       unset METRIC_VALUE\n",
                      "\n",
                      "       METRIC_COUNTER=$((METRIC_COUNTER+1))\n",
                      "       if [ $METRIC_COUNTER -eq 20 ]; then\n",
                      "         # 20 is max metric collection size, so we have to submit here\n",
                      "         aws cloudwatch put-metric-data --region ", { "Ref": "AWS::Region" }, " --cli-input-json \"`echo $CW_JSON_OPEN ${CW_JSON_METRIC%?} $CW_JSON_CLOSE`\"\n",
                      "\n",
                      "         # reset\n",
                      "         METRIC_COUNTER=0\n",
                      "         CW_JSON_METRIC=''\n",
                      "       fi\n",
                      "     fi  \n",
                      "\n",
                      "\n",
                      "\n",
                      "     COUNTER=$((COUNTER+1))\n",
                      "   fi\n",
                      "\n",
                      "   if [[ \"$line\" == \"Client nfs v4:\" ]]; then\n",
                      "     # the next line is the good stuff \n",
                      "     COUNTER=$((COUNTER+1))\n",
                      "   fi\n",
                      " done <<< \"$INPUT\"\n",
                      "done\n",
                      "\n",
                      "# submit whatever is left\n",
                      "aws cloudwatch put-metric-data --region ", { "Ref": "AWS::Region" }, " --cli-input-json \"`echo $CW_JSON_OPEN ${CW_JSON_METRIC%?} $CW_JSON_CLOSE`\""
                    ] ] },
                "mode": "000755",
                "owner": "ec2-user",
                "group": "ec2-user"
              },
              "/home/ec2-user/crontab" : {
                "content" : { "Fn::Join" : [ "", [
                  "* * * * * /usr/sbin/nfsstat | /home/ec2-user/post_nfsstat\n"
                ] ] },
                "owner": "ec2-user",
                "group": "ec2-user"
              }
            },
            "commands" : {
              "01_createdir" : {
                "command" : {"Fn::Join" : [ "", [ "mkdir /", { "Ref" : "MountPoint" }]]}
              }
            }
          },
          "mount" : {
            "commands" : {
              "01_mount" : {
                "command" : { "Fn::Sub": "sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2 ${FileSystem}.efs.${AWS::Region}.amazonaws.com:/ /${MountPoint}"}
              },
              "02_permissions" : {
                "command" : {"Fn::Join" : [ "", [ "chown ec2-user:ec2-user /", { "Ref" : "MountPoint" }]]}
              }
            }
          }
        }
      },
      "Properties": {
        "AssociatePublicIpAddress" : true,
        "ImageId": {
          "Fn::FindInMap": [ "AWSRegionArch2AMI", { "Ref": "AWS::Region" }, {
            "Fn::FindInMap": [ "AWSInstanceType2Arch", { "Ref": "InstanceType" }, "Arch" ]
          } ]
        },
        "InstanceType": { "Ref": "InstanceType" },
        "SecurityGroups": [ { "Ref": "InstanceSecurityGroup" } ],
        "IamInstanceProfile" : { "Ref" : "CloudWatchPutMetricsInstanceProfile" },
        "UserData"       : { "Fn::Base64" : { "Fn::Join" : ["", [
             "#!/bin/bash -xe\n",
             "yum install -y aws-cfn-bootstrap\n",

             "/opt/aws/bin/cfn-init -v ",
             "         --stack ", { "Ref" : "AWS::StackName" },
             "         --resource LaunchConfiguration ",
             "         --configsets MountConfig ",
             "         --region ", { "Ref" : "AWS::Region" }, "\n",

             "crontab /home/ec2-user/crontab\n",

             "/opt/aws/bin/cfn-signal -e $? ",
             "         --stack ", { "Ref" : "AWS::StackName" },
             "         --resource AutoScalingGroup ",
             "         --region ", { "Ref" : "AWS::Region" }, "\n"
        ]]}}
      }
    },
    "AutoScalingGroup": {
      "Type": "AWS::AutoScaling::AutoScalingGroup",
      "DependsOn": ["MountTarget", "GatewayToInternet"],
      "CreationPolicy" : {
        "ResourceSignal" : {
          "Timeout" : "PT15M",
          "Count"   : { "Ref": "AsgMaxSize" }
        }
      },
      "Properties": {
        "VPCZoneIdentifier": [ { "Ref": "Subnet" } ],
        "LaunchConfigurationName": { "Ref": "LaunchConfiguration" },
        "MinSize": "1",
        "MaxSize": { "Ref": "AsgMaxSize" },
        "DesiredCapacity": { "Ref": "AsgMaxSize" },
        "Tags": [ {
          "Key": "Name",
          "Value": "EFS FileSystem Mounted Instance",
          "PropagateAtLaunch": "true"
        } ]
      }
    }
  },
  "Outputs" : {
    "MountTargetID" : {
      "Description" : "Mount target ID",
      "Value" :  { "Ref" : "MountTarget" }
    },
    "FileSystemID" : {
      "Description" : "File system ID",
      "Value" :  { "Ref" : "FileSystem" }
    }
  }
}
@TheRealAmazonKendra
Copy link
Collaborator

I forgot to write a ticket for this but I actually have a fix in progress.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants