Skip to content

Latest commit

 

History

History
369 lines (286 loc) · 20.3 KB

README_OLD.md

File metadata and controls

369 lines (286 loc) · 20.3 KB
Raw

Service Catalog Terraform Reference Architecture

Solution Overview

Note

AWS Service Catalog recently introduced support for Terraform open source so we recommend users to use that instead. This sample will be deprecated in the future. More details on Service Catalog and Terraform open source can be found in the documentation.

AWS Service Catalog and Terraform Terminology

  • hub or fulfillment account: The account where the Terraform server or engine will live. The terraform instance will receive a template path and parameters from the custom lambda which will then be executed by the Terraform binary.
  • spoke account: The spoke accounts that will send Terraform file paths and parameters to the Terraform engine in the Hub account for execution.

This solution requires that a Terraform server be deployed and available. It is possible to use an existing Terraform server though instructions are not included here.
To demonstrate the process we have supplied a standardized CloudFormation templates which creates a new Amazon Elastic Compute Cloud (Amazon EC2) instance and installs Terraform.

You can use one Terraform server to target resources across multiple AWS accounts. For the Terraform server to operate, you must create AWS Identity and Access Management (IAM) roles in each of the spoke accounts and create an AWS Lambda launch function in each desired region of each spoke account. For example, a deployment for 3 accounts that each use us-east-1 and us-east-2 regions will result in 1 Terraform server account, 3 spoke account roles, and 6 Lambda launch functions. The hub account and region containing the Terraform server may also be a spoke account, in which case you must also deploy the IAM role and Lambda launch functions in the hub account and region.

The key to the solution is a CloudFormation template and custom resource lambda that act as a proxy to Terraform. The CloudFormation template uses a custom resource, implemented via an AWS Lambda function to relay template paths and parameters to the Terraform binary on the EC2 instance. The CloudFormation template can then be customized to capture specific parameters and used as a resource in an AWS Service Catalog product.

Solution Overview

The Terraform files are placed in an Amazon Simple Storage Service (Amazon S3) bucket that can be accessed from AWS Service Catalog in each of the spoke accounts. See the following figure.

Solution Overview

Installation

Copy the Service Catalog Terraform components to a S3 bucket to be launched from CloudFormation.
It is assumed that the user has Administrative privileges in IAM for S3, CloudFormation, EC2, VPC, Lambda, and Service Catalog.

  • Clone the repository to your local computer
  git clone https://github.com/aws-samples/aws-service-catalog-terraform-reference-architecture.git

The aws-service-catalog-terraform-reference-architecture folder is created and should look like this

  cd  aws-service-catalog-terraform-reference-architecture
  ls -l

    CODE_OF_CONDUCT.md
    CONTRIBUTING.md
    documentation
    LICENSE
    NOTICE
    README.md
    ServiceCatalogSamples
    TerraformCustomResourceHandler
    TerraformScripts

Replace your AWS account ID with the place holder in the lines below [YOUR-ACCOUNT-ID] NO DASHES

AWS CLI Install Instructions

  1. Create a S3 bucket to host the installation content
    aws s3 mb s3://scterraform-[YOUR-ACCOUNT-ID]

  2. Copy the following files to the bucket maintaining the file structure:

     aws s3 sync ./TerraformCustomResourceHandler/bin/ s3://scterraform-[YOUR-ACCOUNT-ID]/TerraformCustomResourceHandler/bin/  
 aws s3 sync ./ServiceCatalogSamples/ s3://scterraform-[YOUR-ACCOUNT-ID]/ServiceCatalogSamples/  
 aws s3 sync ./TerraformScripts/ s3://scterraform-[YOUR-ACCOUNT-ID]/TerraformScripts/

Login to S3 console and make the folder TerraformScripts public so that the templates can be accessed from child accounts.

You now have all the files needed for launch in the S3 Bucket which has the same file structure as this github repo:

S3 Bukcet: scterraform-[YOUR-ACCOUNT-ID]  
├── TerraformScripts/  
│   ├── cloudformation-templates/  
│   │   ├── terraform-architecture-single-account.yaml  
│   │   ├── terraform-fulfillment-server.yaml  
│   │   ├── terraform-launch-lambda.yaml  
│   │   └── terraform-spoke-principals.yaml  
│   └── bin/  
│       └── sc_terraform_wrapper-1.2-py3-none-any.whl  
└── TerraformCustomResourceHandler/  
    └── bin/  
        └── aws-servicecatalog-terraform-wrapper.jar

Note We have only listed the files you need for this demonstration, more files are included in the actual git repository

Installing the Service Catalog Terraform reference architecture into a single account hub account

  1. Navigate to the S3 console.In the S3 Console, choose the TerraformScripts/cloudformation-templates/terraform-architecture-single-account.yaml file
  2. Copy the Object URL
  3. Navigate to the CloudFormation console.
  4. Verify the region.
  5. Choose Create Stack.
  6. Under Choose a template, select Specify an Amazon S3 template URL.
  7. Paste the URL you copied from step 1 above
  8. Choose Next.
  9. For Stack name, type TerraformArchitecture-SingleAccount.
  10. Leave all the parameters as defaults.
  11. Choose Next
  12. On the Configure stack options page chose Next
  13. On the Review page
  • choose the check box for I acknowledge that AWS CloudFormation might create IAM resources with custom names.
  • choose the check box for I acknowledge that AWS CloudFormation might require the following capability: CAPABILITY_AUTO_EXPAND.
  1. choose Create Stack

When the Status changes to CREATE_COMPLETE once the stack is created. The stack output will look similar to the image below. alt text

Create AWS Service Catalog portfolio and product based on Terraform

  1. Navigate to the S3 S3 console.
  2. Choose the **terraformarchitecture-single-terraformconfigstore-xxxx ** bucket
  3. Choose Upload
  4. Use the file explorer opened earlier to open the ServiceCatalogSamples folder
  5. Select and drag over the following files
  • sc-sample-lamp.json
  • sc-sample-lamp.tf
  • sc-sample-port-product-setup.json
  • sc-sample-S3.json
  • sc-sample-S3.tf
  1. Choose Upload.
  2. In the S3 Console, choose the sc-sample-port-product-setup.json file
  3. Click on it and copy the Object URL
  4. Navigate to the CloudFormation console https://console.aws.amazon.com/cloudformation/
  5. Verify the region.
  6. Choose Create Stack.
  7. Under Choose a template, select Specify an Amazon S3 template URL.
  8. Paste the URL you copied in the step above.
  9. For Stack name, type SCTFportfoliosetup.
  10. For all the other parameters use the defaults.
  11. Choose Next.
  12. Choose Create Stack to create the Service Catalog Sample Terraform Portfolio.
  13. When the stack is complete, choose the Outputs tab
  14. Choose the NewPortFolio to complete the setup and launch the sample products.

alt text alt text alt text

Service Catalog Console alt text

Congratulations, You have completed setting up the Service Catalog Terraform Reference Architecture components in a single account.

Set up the spoke accounts

For each spoke account, you’ll need to enable execution from the hub account by creating an executing role with a trust relationship back to the hub account. Then, create a Lambda launch function in each desired region within the spoke account. Finally, add each spoke account to the SNS topic that queues requests for the spoke account. The supplied CloudFormation stacks will provision these roles and Lambda functions.

Create the spoke account Terraform roles

Note: Make sure the TerraformArchitecture-SingleAccount CloudFormation stack has a status of CREATE_COMPLETE before proceeding.

  1. Sign in to the AWS Console using the spoke account.
  2. Navigate to the CloudFormation console https://console.aws.amazon.com/cloudformation/
  3. Verify the region.
  4. Choose Create Stack.
  5. Under Choose a template, select Specify an Amazon S3 template URL.
  6. Type the following URL https://s3.amazonaws.com/scterraform-[YOUR-ACCOUNT-ID]/TerraformScripts/cloudformation-templates/terraform-spoke-principals.yaml
  7. Choose Next.
  8. For Stack name, type TerraformLaunchRole.
  9. For Fulfillment Account ID, type the hub account ID.
  10. For FulfillmentRegion enter the region.
  11. Update the remaining parameters (optional).
  12. Choose Next.
  13. Choose Create stackto create the Terraform instance.

The Status changes to CREATE_COMPLETE once the stack is created.

Create the Terraform Lambda launch function in the regions you plan to use

Note: Make sure the TerraformLaunchRole CloudFormation stack has a status of CREATE_COMPLETE before proceeding.

  1. Sign in to the AWS Console using the spoke account.
  2. Navigate to the CloudFormation console. https://console.aws.amazon.com/cloudformation/
  3. Verify the region.
  4. Choose Create Stack.
  5. Under Choose a template, select Specify an Amazon S3 template URL.
  6. Type the following URL:https://s3.amazonaws.com/scterraform-[YOUR-ACCOUNT-ID]/TerraformScripts/cloudformation-templates/terraform-launch-lambda.yaml
  7. Choose Next
  8. For Stack name, type TerraformLaunchLambda.
  9. For Fulfillment Account ID** type the hub account ID.
  10. For FulfillmentRegion enter the region
  11. Update the remaining parameters (optional).
  12. Choose Next.
  13. Choose Next.
  14. Choose Create to create the Terraform instance.

The Status changes to CREATE_COMPLETE once the stack is created.

Deploy Service Catalog portfolio and products.

For the test we will deploy the same SC portfolio and products in spoke account. To access the CFN templates from the spoke account, add required access policy on bucket. For this example, just make the files from the bucket terraformarchitecture-single-terraformconfigstore-xxxx public. Go to S3 console of Hub account. Click on s3 bucket terraformarchitecture-single-terraformconfigstore-xxxx. Select all the files and click on Action and the Make public using ACL so that spoke accounts can able to access them. :warning: In your production environment you will setup secure cross account S3 access policy and must not make the templates public.

  1. Navigate to the CloudFormation console https://console.aws.amazon.com/cloudformation/
  2. Verify the region.
  3. Choose Create Stack.
  4. Under Choose a template, select Specify an Amazon S3 template URL.
  5. Copy the URL of json template sc-sample-port-product-setup.json from Hub account s3 bucket terraformarchitecture-single-terraformconfigstore-xxxx. Paste the URL you copied.
  6. For Stack name, type SCTFportfoliosetup.
  7. For all the other parameters use the defaults. But for TemplateConfigBucket enter the S3 bucket full name which is similar to terraformarchitecture-single-terraformconfigstore-xxxxx.s3.<s3-bucket-region>.amazonaws.com
  8. Choose Next.
  9. Choose Create Stack to create the Service Catalog Sample Terraform Portfolio.

Change SNS topic access policy

In order to publish the message on SNS topic terraform-commands-topic locate in Hub account from spoke account, you need to change the access policy of the SNS.

  1. Go to Hub account and SNS console.
  2. Search for SNS topic terraform-commands-topic and click on it.
  3. Choose Edit and the Access policy. Add the ARN of newly created IAM role in Spoke account. i.e. TerraformLaunchLambdaRole
  4. Click Save changes.

Now from the Spoke account's Service Catalog console you can launch the deployed products (by providing user/role access to deployed portfolio).

The above LambdaLaunch function, portfolio deployment steps needs to be repeated for each region.

Using the AWS Service Catalog Terraform Reference Architecture with GitHub

This section is intended for users with a basic working knowledge of GitHub. For more information on GitHub, see https://developer.github.com/v3/guides/. To use the Terraform Reference Architecture with GitHub, follow these steps.

  1. Create a MachineUser with access to the repositories you would like to reference: https://developer.github.com/v3/guides/managing-deploy-keys/#machine-users

  2. Create a secret in your hub account that contains your MachineUser's private key. For example:

    aws secretsmanager create-secret --secret-string "`cat id_rsa`" --name TerraformMachineUserIdentity

    For more information, see https://docs.aws.amazon.com/secretsmanager/latest/userguide/manage_create-basic-secret.html

  3. Generate a hashed known_hosts file containing the public keys of your repository's host:
    a. Retrieve Github's public key using the following commands:

    touch known_hosts
chmod 600 known_hosts
ssh-keyscan -t rsa github.com >> known_hosts

    b. Verify the public key against Github's public key fingerprints using the following command:
    cat known_hosts | ssh-keygen -lf -
    c. Compare the output to the public key fingerprints Github has posted: https://help.github.com/articles/github-s-ssh-key-fingerprints/
    d. Generate a hashed known_hosts file using the public key:
    ssh-keygen -H -f known_hosts

  4. Create a secret in your hub account that contains the hashed known_hosts file. For example:
    aws secretsmanager create-secret --secret-string "`cat known_hosts`" --name TerraformKnownHosts

  5. Update the SshIdentitySecret and SshKnownHostsSecret parameters of your TerraformWrapperServer stack. For example:

    aws cloudformation update-stack --template-url https://s3.amazonaws.com/scterraform-[YOUR-ACCOUNT-ID]/TerraformScripts/cloudformation-templates//terraform-fulfillment-server.yaml --parameters ParameterKey=SshKnownHostsSecret,ParameterValue=TerraformKnownHosts ParameterKey=SshIdentitySecret,ParameterValue=TerraformMachineUserIdentity --capabilities CAPABILITY_NAMED_IAM --stack-name TerraformWrapperServer

Executing the Terraform file plan/dry run command

The Terraform plan command creates an execution plan or determines actions that are necessary to achieve the desired state, specified in the configuration files, without making any changes to real resources or to the state. The output of the plan is available in the output section of the AWS Service Catalog end user console. Include the DryRunId property and set the value to 1. The Terraform wrapper server will execute the plan as a preview only, instead of creating or managing AWS resources.

"Resources": {
        "MyTerraformStack": {
            "Type": "Custom::TerraformStack",
            "Properties": {
                "ServiceToken": {
                    "Fn::Sub": "arn:aws:lambda:${AWS::Region}:${AWS::AccountId}:function:TerraformLaunchHandler"
                },
                "TerraformArtifactUrl": {
                    "Fn::Sub": https://s3.amazonaws.com/terraform-config-${AWS::AccountId}/sc-sample-lamp.tf 
                },
                "LaunchRoleArn": {
                    "Fn::Sub": "arn:aws:iam::${AWS::AccountId}:role/TerraformResourceCreationRole" 
                },
                "DryRunId": {“Ref”:“DryRunId”},  *

                "TerraformVariables": {
                    "aws_region": {
                        "Fn::Sub": "${AWS::Region}"
                    },
                    "aws_ami": {
                        "Ref": "ImageID"
    • Add this property for Plan or Dry run.

FAQ

Can a Terraform wrapper template reference multiple Terraform configuration files?

Yes. If you have multiple Terraform configuration files, you can package them in a ZIP file, upload the ZIP file to Amazon S3, and then reference the ZIP file in your wrapper template. The Terraform Wrapper Server detects ZIP files and extracts their contents before attempting to execute any commands. What are the limitations of this solution?

  • Stack creation will fail for Terraform configurations that take longer than two hours to create.
  • AWS is the only supported provider.
  • CloudFormation plan APIs will not list the resources generated by a TerraformStack.

Can an IAM role be used to launch Terraform products instead of an IAM user?

Yes, as long as a role has permissions to provision AWS Service Catalog products and invoke the spoke account’s Lambda launch function, it can be used to launch Terraform products. For example, the CloudFormation template defines a role that can be used to launch Terraform products in a spoke account.

  Resources:
    TerraformUserLaunchRole:
      Type: AWS::IAM::Role
      Properties:
        RoleName: "TerraformLaunchLambdaRole"
        AssumeRolePolicyDocument:
          Version: 2012-10-17
          Statement:
            - Effect: Allow
              Principal:
                AWS:
                  - arn:aws:iam::111111111111:user/terraform-account-x-enduser
              Action:
                - sts:AssumeRole
        ManagedPolicyArns:
          - arn:aws:iam::aws:policy/AWSServiceCatalogEndUserFullAccess
        Policies:
          - PolicyName: TerraformLaunchLambdaPolicy
       PolicyDocument:
              Statement:
                - Effect: Allow
                  Action: lambda:InvokeFunction
                  Resource: !Sub arn:aws:lambda:*:${AWS::AccountId}:function:TerraformLaunchHandler
                - Effect: Allow
                  Action: s3:GetObject
                  Resource: '*'

Security Best Practices

Update your Terraform binary version when you identify vulnerabilities

If you become aware of a security vulnerability affecting the Terraform binary version installed on your Terraform Wrapper Server, switch to a version that is not affected by the vulnerability. To do this, update your Terraform Wrapper Server stack by setting the stack's TerraformVersion parameter to the version you want to switch to. Keep in mind that you may need to update your Terraform configurations if a backwards incompatible change has been introduced in the Terraform binary version to which you are switching.

Restrict access to the Launch Lambda in the spoke account

Because the Lambda launch function has access to the hub account’s SNS topic, you should restrict access to the Lambda launch function in the spoke account. Do not grant non-administrator users in the spoke account permissions to lambda:UpdateFunctionCode , lambda:InvokeFunction or lambda:UpdateFunctionConfiguration unless you exclude the Lambda launch function using a resource element.

Do not remove the ExternalId from the spoke account’s ResourceCreationRole

The spoke account’s IAM Resource Creation Role uses an ExternalId to limit access to the spoke account. You should retain this ExternalId to help prevent inadvertent modification to resources in the spoke account by users that have access to the hub account’s Terraform Server Role.

Limit access to the hub account

The hub account’s Terraform Wrapper Server will have access to Resource Creation Roles in multiple spoke accounts. To limit the exposure of these roles, you should restrict access to the hub account as much as possible and use the hub account for Terraform functionality only.

Questions and Comments

aws-sa-servicecatalog@amazon.com

Further Reading