diff --git a/_docs-sources/foundations/accounts/index.md b/_docs-sources/foundations/accounts/index.md deleted file mode 100644 index f8f236241e..0000000000 --- a/_docs-sources/foundations/accounts/index.md +++ /dev/null @@ -1 +0,0 @@ -# Account Foundations Placeholder diff --git a/_docs-sources/foundations/accounts/manage-accounts.md b/_docs-sources/foundations/accounts/manage-accounts.md deleted file mode 100644 index 26cce6ebcf..0000000000 --- a/_docs-sources/foundations/accounts/manage-accounts.md +++ /dev/null @@ -1,59 +0,0 @@ -import Admonition from "@theme/Admonition" - -# Manage your accounts - -Gruntwork's Control Tower integration provides an IaC-based (infrastructure as code) approach to many of your account management needs. Operations within those accounts can be grouped according to whether they may be performed in IaC, the AWS Console, or either. _When operations may be performed in either location, we strongly recommend using IaC._ - -## Prerequisites - -- An AWS account with AWS Control Tower set up -- Access to an IAM User or Role with administrative permissions to AWS Control Tower - -## How to manage your accounts - -Below you'll find a table with common account operations and the Gruntwork recommendation for if the operation should be done using IaC or in the AWS Console. When both options are available, using IaC is strongly recommended. You can explore more documentation for each operation by clicking on the operation name in the table. - -| Management Operation | Terraform (IaC) | AWS Console (ClickOps) | -| ------------------------------------------------------------------------------------------------------------------------------------------- | ---------------- | ---------------------- | -| [Create a new Organization Unit](https://docs.aws.amazon.com/controltower/latest/userguide/create-new-ou.html) (OU) | ❌ | ✅ | -| [Request a new account](./add-account.md) | ✅ | ❌ | -| [Create a new account](./add-account.md) | ✅ | ❌ | -| [Removing an account](./manage-accounts.md#removing-an-account) | ✅ | ❌ | -| [Renaming an account](https://docs.aws.amazon.com/controltower/latest/userguide/change-account-name.html) | ❌ | ✅ | -| [Update root account e-mail address](https://docs.aws.amazon.com/accounts/latest/reference/manage-acct-update-root-user.html) | ❌ | ✅ | -| [Modify account controls](https://docs.aws.amazon.com/controltower/latest/userguide/enable-controls-on-ou.html) | ❌ | ✅ | -| [Moving an account to a new Organizational Unit](./manage-accounts.md#un-managing-an-account) | ✅ (recommended) | ✅ (discouraged) | -| [Update account admin user in Account Access IAM Identity Center](./manage-accounts.md#un-managing-an-account) | ✅ (recommended) | ✅ (discouraged) | -| Granting additional users access to accounts in AWS IAM Identity Center | ✅ (recommended) | ✅ (discouraged) | - - -## Removing an account - -1. Create a branch and delete the directory in your `infrastructure-live` repository that corresponds to the account you would like to remove. - - -

This will delete all resources in the account. Make sure you are deleting the correct directory.

- - -Push your changes and create a PR. Pipelines will detect that an account is deleted. Verify that all expected resources will be removed in the `plan` output. - -Once you have confirmed everything looks as expected, merge the PR. - -1. Create a branch and delete the account request file in `_new_account_requests` - -This will de-provision the product in AWS Service Catalog, but will not delete the account. - -Push your changes and create a PR. Pipelines will detect that the account should be removed, which can be verified in the `plan` output. - -Once you have confirmed everything looks as expected, merge the PR. After `apply` runs, the account status should be `Suspended`. - -## Updating the account request - -You may update some attributes attributes of an AWS Account by modifying the the account request file in `_new_account_requests`. See below for steps to update each attribute. - -Start by creating a new branch that will contain your changes. - -1. Update the name of the OU by modifying the `organizational_unit_name` key -1. Updating the account admin user by modifying the `sso_user_first_name`, `sso_user_last_name`, and `sso_user_email` keys to the new users first name, last name, and email. - -After you have made your modifications, push your branch and create a pull request. Gruntwork Pipelines will detect the account changes and run a `plan` operation. Review the output of the `plan` to confirm the output is as expected. Once confirmed, merge the PR to apply the changes. diff --git a/_docs-sources/foundations/ci-cd/index.md b/_docs-sources/foundations/ci-cd/index.md deleted file mode 100644 index 1e3dc889f7..0000000000 --- a/_docs-sources/foundations/ci-cd/index.md +++ /dev/null @@ -1,68 +0,0 @@ -# Pipelines for Account Factory - -[Gruntwork Pipelines](../../pipelines/overview/) is a framework for deploying Infrastructure as Code changes and managing requests for and creation of AWS accounts, with first class support for [Terragrunt](https://terragrunt.gruntwork.io/). Pipelines enables you to use your preferred CI tool to detect changes to Infrastructure as Code Infrastructure Units (IUs — individual modules containing `terragrunt.hcl` files) and securely run `terragrunt plan` and `terragrunt apply` jobs on each IU. - -Pipelines runs as a step in your CI tool, determining the types of changes that were made (e.g., adding, changing, or deleting a module) and the action to take based on whether your change was made in a Pull Request (PR) or in a commit to your main branch (e.g., the PR has been merged). As an example, creating a pull request with changes that add a new IU containing an AWS RDS instance in your `dev` account would detect the new module and run a `terragrunt plan` in the directory containing the IU. - -:::note Supported CI systems - -GitHub Actions is currently the only CI system supported by Gruntwork Pipelines. Support for additional CI systems will be added in the future — let us know your preferred tool. In the meantime, our legacy version of Gruntwork Pipelines remains available and can be configured for the CI system of your choice. - -::: - -## Components of Pipelines - -Pipelines is a single binary that runs as a step in your CI tool. To perform the actions that Pipelines detects, each account will need an AWS IAM Role that allows GitHub Actions to assume it using OIDC. This role is automatically provisioned in new accounts that are provisioned when [adding an account](../accounts/add-account.md) using the Account Factory. - -Pipelines assumes that each top level directory in your `infrastructure-live` repository maps to a single AWS account, excluding the `_envcommon` directory. Each account mapped directory must have an entry in `accounts.yml` with a key matching the directory name and containing key/value pairs for the account ID and the e-mail for the root user of the account. - -For example, the following entry in `accounts.yml` would be mapped to a directory called `my-cool-account` in your `infrastructure-live` repository. - -```yml title=accounts.yml -"my-cool-account": - "email": "my-root-account-email@example.com" - "id": "123456789012" -``` - -```bash title="Infrastructure Live" -. -├── accounts.yml -├── _envcommon -│ └── services -│ └── my-app.hcl -├── my-cool-account -│ └── us-east-1 -│ └── dev -│ └── database -│ └── terragrunt.hcl -``` - -## Account Specific Pipelines - -:::note - -This section contains some advanced topics pertaining to Pipelines, how it differentiates between types of changes, and how it handles planning and applying changes. - -From the perspective of the end user, you will still only need one step in your CI job. Pipelines knows how to do the rest. - -::: - -Pipelines can handle two main types of events — additions, changes, or deletions of Terragrunt files, and creating new AWS accounts. With the exception of account creation, adding, changing, or deleting Terragrunt files (aka Infrastructure Units) is handled by the pipeline for each specific account. Provisioning new accounts and applying baselines in them is handled exclusively by the management account. - -### Management Account - -Provisioning AWS accounts must be handled by your Management AWS Account (e.g., your AWS Control Tower management account). When account requests are approved and the account is created, Pipelines will create a PR that contains an account baseline. Since Pipelines doesn’t have access to the account yet, the baseline will be applied using the Management Account Pipeline. - -When applying the baseline and provisioning the Pipelines role in the new child account, Pipelines will first assume the management account Pipelines role, then will assume an automatically provisioned role in the child account (this process is known as [role chaining](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_terms-and-concepts.html)). Once Pipelines has temporary credentials in the new account, it will run a plan or apply to provision the requested resources. After the initial baseline and Pipelines role has been applied in the child account, all subsequent events for that account will be handled by the Pipelines role directly in the child account. - -:::note - -The AWS IAM Role for the management account must have permissions to provision new AWS Accounts (via AWS Control Tower) and assume a role in all child accounts. - -::: - -### Child Accounts - -Each child account (e.g., `dev`, `stage`, `prod,` etc.) contains an AWS IAM role that Pipelines can assume from GitHub Actions using OIDC. This role is automatically provisioned as a part of the [account baseline process](../accounts/add-account#4-review-and-merge-the-account-baseline-pr). Once this role is in place in the child account, users may submit pull requests with new Infrastructure Units in the child account. - -When a Pull Request is created or synchronized, or a push to main occurs, Pipelines will detect the changes, map them to the new account, assume the role in the child account, then run a `terragrunt plan` or `terragrunt apply` job (plan for PRs, apply for pushes to main). diff --git a/_docs-sources/foundations/iac/folder_structure.md b/_docs-sources/foundations/iac-foundations/folder-structure.md similarity index 100% rename from _docs-sources/foundations/iac/folder_structure.md rename to _docs-sources/foundations/iac-foundations/folder-structure.md diff --git a/_docs-sources/foundations/iac-foundations/index.md b/_docs-sources/foundations/iac-foundations/index.md new file mode 100644 index 0000000000..ce78089838 --- /dev/null +++ b/_docs-sources/foundations/iac-foundations/index.md @@ -0,0 +1,27 @@ +# About IaC Foundations + +The IaC Foundations component is focused on: + +- Teaching you the considerations to think about when coming up with your foundational Terraform/OpenTofu and Terragrunt patterns +- Giving you a fully configured set of git repositories with an initial folder structure + +## What's included + +- **Strategy.** We recommend core patterns your Terragrunt and Terraform/OpenTofu git repo needs to incorporate to scale effectively. +- **IaC Modules.** No IaC modules are needed for this component. +- **Tooling.** We recommend Terragrunt to effectively use Terraform/OpenTofu at scale. +- **Setup.** We grant you access to a sophisticated git repo template that includes customization options and generates your repo code. +- **Updates.** We publish ongoing updates to IaC foundational patterns and will write [patches](/patcher) if applicable to adopt those changes. + +## IaC foundations considerations + +When creating your initial Terraform/OpenTofu and Terragrunt infrastructure, there are a number of patterns you need to consider how to solve, including: + +- How to create the backend (e.g. S3 bucket) for storing Terraform state +- How to structure your folder hierarchy +- How to handle tagging and labels +- Whether to use branches per environment +- How to handle global variables +- How to handle module default values + +This component includes either pre-baked implementations that address these considerations, or written guidance on how to incorporate them yourself. diff --git a/_docs-sources/foundations/iac-foundations/initial-setup.md b/_docs-sources/foundations/iac-foundations/initial-setup.md new file mode 100644 index 0000000000..0b04845452 --- /dev/null +++ b/_docs-sources/foundations/iac-foundations/initial-setup.md @@ -0,0 +1,27 @@ +# Initial setup + +To set up IaC Foundations, we use three pre-configured git repository templates that include best practices and also allow for customization. + +For each repository below, navigate to the template repository and select **Use this template** -> **Create a new Repository**. This will initiate repository creation. You should select your org as the owner, add a description if you like, make sure you are creating a **private** repo, and click **Create repository**. + +The repository template will be created, and you can follow the instructions in the `README` to bootstrap your IaC Foundations. Gruntwork is available to assist with questions around other patterns as they arise. + +### Infrastructure Live Template + +_[https://github.com/gruntwork-io/infrastructure-live-template](https://github.com/gruntwork-io/infrastructure-live-template)_ + +This template creates an infrastructure-live repository with scaffolding for a best practices Terragrunt configuration, including patterns for module defaults, global variables, and account baselines. It also configures Gruntwork Pipelines, which is easy to remove if you don't want it. + +### Infrastructure Modules Template + +_[https://github.com/gruntwork-io/infrastructure-modules-template](https://github.com/gruntwork-io/infrastructure-modules-template)_ + +This template creates an empty infrastructure-modules repository that will be used to store Terraform/OpenTofu modules that your organization has authored and intends to use within your organization. + +### Infrastructure Pipelines Template + +_[https://github.com/gruntwork-io/infrastructure-pipelines-template](https://github.com/gruntwork-io/infrastructure-pipelines-template)_ + +This template is only necessary if you plan on implementing [Gruntwork Pipelines](../pipelines). + + diff --git a/_docs-sources/foundations/iac/module_defaults/defining.md b/_docs-sources/foundations/iac-foundations/module_defaults/defining.md similarity index 100% rename from _docs-sources/foundations/iac/module_defaults/defining.md rename to _docs-sources/foundations/iac-foundations/module_defaults/defining.md diff --git a/_docs-sources/foundations/iac/module_defaults/index.md b/_docs-sources/foundations/iac-foundations/module_defaults/index.md similarity index 100% rename from _docs-sources/foundations/iac/module_defaults/index.md rename to _docs-sources/foundations/iac-foundations/module_defaults/index.md diff --git a/_docs-sources/foundations/iac/module_defaults/usage.md b/_docs-sources/foundations/iac-foundations/module_defaults/usage.md similarity index 100% rename from _docs-sources/foundations/iac/module_defaults/usage.md rename to _docs-sources/foundations/iac-foundations/module_defaults/usage.md diff --git a/_docs-sources/foundations/iac/index.md b/_docs-sources/foundations/iac/index.md deleted file mode 100644 index 3fddb0cf8b..0000000000 --- a/_docs-sources/foundations/iac/index.md +++ /dev/null @@ -1,37 +0,0 @@ -# About IaC Foundations - -The IaC Foundations component is focused on: - -- Teaching you the considerations to think about when coming up with your foundational Terraform and Terragrunt patterns -- Giving you a fully configured set of git repositories with an initial folder structure - -## IaC foundations considerations - -When creating your initial Terraform and Terragrunt and infrastructure, there are a number of patterns you need to consider how to solve. These include: - -- How to create the backend (e.g. S3 bucket) for storing Terraform state -- How to structure your folder hierarchy -- How to handle tagging and labels -- Whether to use branches per environment -- How to handle global variables -- How to handle module default values - -## Initial setup - -We provide 3 template repos to subscribers that implement best practices regarding the considerations above. - -For each of the repositories below, navigate to the template repository and select **Use this template** -> **Create a new Repository**. -This will initiate repository creation. You should select your org as the owner, add a description if you like, make sure you are creating a **private** repo, and click **Create repository**. - -The repository template will be created, and you can follow the instructions in the README to bootstrap your IaC Foundations. - -### [Infrastructure Live Template](https://github.com/gruntwork-io/infrastructure-live-template) -This [template](https://github.com/gruntwork-io/infrastructure-live-template) creates an infrastructure live repository with scaffolding for common module defaults, account baselines, and Gruntwork Pipelines - -### [Infrastructure Modules Template](https://github.com/gruntwork-io/infrastructure-modules-template) -This [template](https://github.com/gruntwork-io/infrastructure-modules-template) creates an empty infrastructure modules repository for populating shared modules within your organization. - -### [Infrastructure Pipelines Template](https://github.com/gruntwork-io/infrastructure-pipelines-template) - -This [template](https://github.com/gruntwork-io/infrastructure-pipelines-template) is only necessary if you plan on implementing [Gruntwork Pipelines](../ci-cd). - diff --git a/docs/foundations/accounts/add-account.md b/_docs-sources/foundations/landing-zone/add-aws-account.md similarity index 78% rename from docs/foundations/accounts/add-account.md rename to _docs-sources/foundations/landing-zone/add-aws-account.md index 03d86aa5b1..5c8db1ec60 100644 --- a/docs/foundations/accounts/add-account.md +++ b/_docs-sources/foundations/landing-zone/add-aws-account.md @@ -1,6 +1,6 @@ -# Add a new account +# Add a new AWS account -This document provides instructions for provisioning a new account in a Control Tower managed organization using Gruntwork. The described workflow gives you the flexibility to require approval for all new account requests in accordance with the permissions configured in your repository. +This document provides instructions for provisioning a new AWS account using Gruntwork Landing Zone. The described workflow gives you the flexibility to require approval for all new AWS account requests in accordance with the permissions configured in your git repository. ## Prerequisites @@ -8,9 +8,9 @@ Before proceeding, ensure you have an `infrastructure-live` repository which con - The [`control-tower-multi-account-factory` module](https://GitHub.com/gruntwork-io/terraform-aws-control-tower/tree/main/modules/landingzone/control-tower-multi-account-factory) configured in the root account -- An installation of [Gruntwork Pipelines](https://LINK-TO-VALID-DOC) +- An installation of [Gruntwork Pipelines](/pipelines/overview) -## 1. Create an account request file +## 1. Create an AWS account request file To initiate the process, create an `account-.yml` file in the `_new-account-requests` folder located in the root of your `infrastructure-live` repository. This file will be used to create a pull request and add the new account to your organization. The file should have the following format: @@ -40,7 +40,7 @@ requested_by: :::note -If you have configured GitHub Actions in your `infrastructure-live` repository with an [Account Factory workflow](https://LINK-TO-VALID-DOC), you can invoke that workflow via the GitHub UI or programmatically. This workflow will automatically create the account request file and open a pull request on your behalf. After doing so, jump to [step 3](#3-review-and-merge-the-account-request-pr). +If you have configured GitHub Actions in your `infrastructure-live` repository with a Gruntwork Landing Zone Account Factory workflow, you can invoke that workflow via the GitHub UI or programmatically. This workflow will automatically create the account request file and open a pull request on your behalf. After doing so, jump to [step 3](#3-review-and-merge-the-account-request-pr). ::: @@ -48,20 +48,8 @@ If you have configured GitHub Actions in your `infrastructure-live` repository w Next, create a pull request containing the new account request file. This action will trigger the Gruntwork pipeline to `terragrunt plan` the new account and update the pull request with the plan output. -:::caution - -Currently the output is still only available in the `infrastructure-pipelines` repository’ GitHub Actions logs. This will be updated shortly to be available in the pull request and this notice will be removed. - -::: - ## 3. Review and merge the account request PR -:::caution - -Currently the output is still only available in the `infrastructure-pipelines` repository’ GitHub Actions logs. This will be updated shortly to be available in the pull request and this notice will be removed. - -::: - Review the `plan` output in the pull request. Once satisfied, merge the pull request to trigger creation of the new account. Gruntwork Pipelines will run `terragrunt apply` to create the new account in Control Tower. :::tip @@ -88,11 +76,3 @@ After the pipeline for the new account request completes, the new account will e - A new OIDC role required to install the baseline and enable Gruntwork Pipelines to run within the new account When you merge this pull request, Gruntwork Pipelines will automatically deploy the Gruntwork account baselines into the new account and provision a role that Pipelines can assume to deploy resources into this account. Once this process completes, you may access your account and leverage Gruntwork Pipelines to make any further changes to the infrastructure in your new account. - - - diff --git a/docs/foundations/accounts/enable-control-tower.md b/_docs-sources/foundations/landing-zone/enable-control-tower.md similarity index 88% rename from docs/foundations/accounts/enable-control-tower.md rename to _docs-sources/foundations/landing-zone/enable-control-tower.md index b8f1419beb..ccef640d5b 100644 --- a/docs/foundations/accounts/enable-control-tower.md +++ b/_docs-sources/foundations/landing-zone/enable-control-tower.md @@ -1,19 +1,11 @@ # Enable Control Tower -Enabling Control Tower is the first step in getting started with Gruntwork Account Foundations. It must be completed +Enabling Control Tower is the first step in getting started with Gruntwork Landing Zone. It must be completed before infrastructure as code is generated for your infrastructure-live repository. -:::info - -[AWS Control Tower](https://docs.aws.amazon.com/controltower/latest/userguide/what-is-control-tower.html) is an AWS service -that orchestrates accounts, handling creation, deletion, security policies, and more. It serves as the base layer of -Gruntwork Account Foundations. - -::: - ## Prerequisites -In order to enable control tower you will need the resources described in [Prerequisites.](/foundations/accounts/prerequisites) +In order to enable AWS Control Tower you will need the resources described in [Prerequisites](./prerequisites). ## Enable AWS Control Tower @@ -107,7 +99,7 @@ Control Tower Creation will take around an hour to complete ## Initial Configuration Now that Control Tower is enabled in your root account, there are a few configuration changes that need to be made to -prepare the landing zone for Gruntwork Account Foundations. +prepare for Gruntwork Landing Zone. 1. Navigate to the [AWS Control Tower Organization Dashboard](https://console.aws.amazon.com/controltower/home/organization) 1. [Turn off the default VPC created for new accounts](https://docs.aws.amazon.com/controltower/latest/userguide/configure-without-vpc.html#create-without-vpc). Gruntwork VPCs will be created for each account using terraform. @@ -132,12 +124,4 @@ Control Tower is now configured! Next you should consider: - [Configuring IAM Identity Center](https://docs.aws.amazon.com/singlesignon/latest/userguide/get-started-choose-identity-source.html) for Access Control. - [Configuring any controls or SCPs](https://docs.aws.amazon.com/controltower/latest/userguide/controls.html) your organization requires. -- [Set up your Gruntwork IaC Foundations](/foundations/iac) - - - +- [Setting up your Gruntwork IaC Foundations](../iac-foundations) diff --git a/_docs-sources/foundations/landing-zone/index.md b/_docs-sources/foundations/landing-zone/index.md new file mode 100644 index 0000000000..f066ef9e38 --- /dev/null +++ b/_docs-sources/foundations/landing-zone/index.md @@ -0,0 +1,42 @@ +# About Landing Zone + +The Landing Zone component is focused on: + +- Creating an initial best-practice, AWS multi-account setup +- Putting in place a system for vending new AWS accounts (commonly known as an "account factory") +- Installing secure AWS account baselines compliant with the [CIS AWS Foundations Benchmark](https://www.cisecurity.org/benchmark/amazon_web_services). + +Gruntwork Landing Zone is intended to be deployed as your new paradigm for creating and managing AWS accounts. Once set up, you can import existing AWS accounts. + +## What's included + +- **Strategy.** Gruntwork has designed an approach that extends AWS Control Tower to support the full set of needs around AWS account management with Terraform/OpenTofu and GitHub. +- **IaC Modules.** Gruntwork Landing Zone includes modules to configure Control Tower, configure AWS IAM Identity Center, and install AWS account baselines that meet the CIS AWS Foundations Benchmark. +- **Tooling.** Gruntwork Landing Zone makes use of Gruntwork Pipelines and GitHub Action to support the account request and review process. +- **Setup.** Installing Gruntwork Landing Zone is fully documented. +- **Updates.** Gruntwork publishes regular updates to all Landing Zone modules, including support for the latest versions of the CIS AWS Foundations Benchmark, and ongoing updates to our Landing Zone workflows. + +## Extending AWS Control Tower + +Gruntwork Landing Zone builds on AWS Control Tower to add important new functionality. + +### What is AWS Control Tower? + +AWS Control Tower helps you set up and govern AWS accounts, and gives you a centralized way to manage compliance and configuration across your AWS accounts. But Control Tower is fundamentally a Platform-as-a-Service (PaaS) paradigm where you use "ClickOps" to create and configure AWS accounts. That stands in contrast to the vision behind IaC tools like Terraform, which are built on the philosophy that all Cloud resources are described in code. It can also be challenging to customize Control Tower and maintain it over time. + +### Additional functionality + +Gruntwork Landing Zone directly addresses these limitations by extending Control Tower with the following functionality: + +1. Configure and customize new AWS accounts using Terraform/OpenTofu +1. Have all new AWS accounts meet the CIS AWS Foundations Benchmark out of the box +1. Request new AWS accounts using a simple YAML file, GitHub Actions, or any system that can trigger a GitHub Action (e.g. ServiceNow) +1. Review and customize all new AWS account requests using GitHub Pull Request functionality +1. Define a customized account baseline unique to your organization +1. Enable AWS account baselines to stay up to date automatically using [Patcher](./patcher) + +## Greenfield vs. brownfield + +Gruntwork Landing Zone is optimized for situations where you are creating a "new world" infrastructure. Once your new world is stable, you can then import existing AWS accounts into Gruntwork Landing Zone, and migrate existing resources as needed. + +While it possible to migrate an existing Control Tower deployment into Gruntwork Landing Zone, we have not yet defined a formal process for this and we do not currently recommend this. That being said, other Gruntwork DevOps components can still be installed alongside your existing non-Gruntwork Landing Zone solution. diff --git a/_docs-sources/foundations/landing-zone/manage-accounts.md b/_docs-sources/foundations/landing-zone/manage-accounts.md new file mode 100644 index 0000000000..5fbcdafd45 --- /dev/null +++ b/_docs-sources/foundations/landing-zone/manage-accounts.md @@ -0,0 +1,67 @@ +# Managing your AWS accounts + +Over time you will need to run various operations on your AWS accounts such as requesting new accounts, creating new accounts, renaming accounts, etc. In Gruntwork Landing Zone, some AWS account management operations should only be done using IaC, some can only be done using ClickOps, and some can be done using either. + +In this page, we review which mode (IaC or ClickOps) to use for each AWS account operation. _When operations may be performed using either mode, we strongly recommend using IaC._ + +## Prerequisites + +This page applies to users who are: +- Actively running Gruntwork Landing Zone +- Have access to an AWS user or IAM Role with administrative permissions to AWS Control Tower + +## When to use IaC vs. ClickOps + +Below you'll find a table with common AWS account operations and the Gruntwork recommendation for if the operation should be done using IaC or in the AWS Console. When both options are available, using IaC is strongly recommended. You can explore more documentation for each operation by clicking on the operation name in the table. + +- ✅ means that the operation should or can only be done using the given mode. +- ❌ means that the operation should not or cannot be done using the given mode. + +| Management Operation | Terraform (IaC) | AWS Console (ClickOps) | +|-------------------------------------------------------------------------------------------------------------------------------| ---------------- | ---------------------- | +| [Create a new Organization Unit](https://docs.aws.amazon.com/controltower/latest/userguide/create-new-ou.html) (OU) | ❌ | ✅ | +| Request a new account | ✅ | ❌ | +| Create a new account | ✅ | ❌ | +| [Remove an account](#remove-an-aws-account) | ✅ | ❌ | +| [Rename an account](https://docs.aws.amazon.com/controltower/latest/userguide/change-account-name.html) | ❌ | ✅ | +| [Update root account e-mail address](https://docs.aws.amazon.com/accounts/latest/reference/manage-acct-update-root-user.html) | ❌ | ✅ | +| [Modify account controls](https://docs.aws.amazon.com/controltower/latest/userguide/enable-controls-on-ou.html) | ❌ | ✅ | +| Moving an account to a new Organizational Unit | ✅ (recommended) | ✅ (discouraged) | +| Update account admin user in Account Access IAM Identity Center | ✅ (recommended) | ✅ (discouraged) | +| Granting additional users access to accounts in AWS IAM Identity Center | ✅ (recommended) | ✅ (discouraged) | + + +## AWS account management operations + +### Remove an AWS account + +This operation removes an AWS account from AWS Control Tower but does _not_ delete the AWS account. + +1. Create a branch and delete the directory in your `infrastructure-live` repository that corresponds to the account you would like to remove. + + :::warning + This will delete all resources in the account. Make sure you are deleting the correct directory. + ::: + + Push your changes and create a PR. Pipelines will detect that an account is deleted. Verify that all expected resources will be removed in the `plan` output. + + Once you have confirmed everything looks as expected, merge the PR. + +1. Create a branch and delete the account request file in `_new_account_requests`. + + This will de-provision the product in AWS Service Catalog, but will not delete the account. + + Push your changes and create a PR. Pipelines will detect that the account should be removed, which can be verified in the `plan` output. + + Once you have confirmed everything looks as expected, merge the PR. After `apply` runs, the account status should be `Suspended`. + +### Update the new AWS account request + +You may update some attributes of an AWS Account by modifying the account request file in `_new_account_requests`. See below for steps to update each attribute. + +Start by creating a new branch that will contain your changes. + +1. Update the name of the OU by modifying the `organizational_unit_name` key +1. Updating the account admin user by modifying the `sso_user_first_name`, `sso_user_last_name`, and `sso_user_email` keys to the new users first name, last name, and email. + +After you have made your modifications, push your branch and create a pull request. Gruntwork Pipelines will detect the account changes and run a `plan` operation. Review the output of the `plan` to confirm the output is as expected. Once confirmed, merge the PR to apply the changes. diff --git a/docs/foundations/accounts/prerequisites.md b/_docs-sources/foundations/landing-zone/prerequisites.md similarity index 85% rename from docs/foundations/accounts/prerequisites.md rename to _docs-sources/foundations/landing-zone/prerequisites.md index 5c7c8cf221..fde9878dfd 100644 --- a/docs/foundations/accounts/prerequisites.md +++ b/_docs-sources/foundations/landing-zone/prerequisites.md @@ -1,6 +1,14 @@ # Prerequisites -In order to enable control tower you will need the following: +The first step to using Gruntwork Landing Zone is to use AWS Control Tower to create a new multi-account setup, which requires that you meet the prerequisites below. + +## Gruntwork Pipelines + +In order to get the full functionality of Gruntwork Landing Zone, you must use [Gruntwork Pipelines](/pipelines/overview/). + +## Enabling AWS Control Tower + +To use AWS Control Tower you must first enable it, which requires that you have the following: 1. A new AWS Account and a user with administrator permissions. (We recommend using an IAM user with admin permissions rather than the root user) @@ -15,7 +23,7 @@ In order to enable control tower you will need the following: Your home Region is the AWS Region in which you'll run most of your workloads or store most of your data. It **cannot** be changed after you've set up your AWS Control Tower landing zone. For more information about how to choose a home Region, see [Administrative tips for landing zone setup](https://docs.aws.amazon.com/controltower/latest/userguide/tips-for-admin-setup.html). ::: -## KMS Key Creation +### KMS Key Creation :::info For more help setting up KMS see the AWS docs: [Guidance for KMS keys](https://docs.aws.amazon.com/en_us/controltower/latest/userguide//kms-guidance.html) ::: @@ -72,11 +80,6 @@ For more help setting up KMS see the AWS docs: [Guidance for KMS keys](https://d 1. Click `Save Changes` +## Next steps - - +Now that you've met the prerequisites, it's time to enable Control Tower. diff --git a/_docs-sources/foundations/overview/index.md b/_docs-sources/foundations/overview/index.md index 65a7442f23..06f5d6f2ff 100644 --- a/_docs-sources/foundations/overview/index.md +++ b/_docs-sources/foundations/overview/index.md @@ -1,32 +1,29 @@ # DevOps Components -Gruntwork DevOps Foundations is made up of a collection of _DevOps components._ A DevOps component is an essential element of modern infrastructure for which Gruntwork has pre-built IaC modules and pre-written documentation to reflect our recommended best practices and support fast setup. +**Gruntwork DevOps Foundations is a collection of _DevOps components_ that you use as building blocks to assemble your own best-practice infrastructure.** -## Available components - -The Gruntwork DevOps components include: +In a modern cloud infrastructure there are many component parts, ranging from the infrastructure pipeline to secrets management to FinOps to how you deploy apps. Setting up and managing each component requires that you understand the core infrastructure need, develop a strategy for addressing it, implement a solution, and then maintain it forever. Doing this on your own is expensive and error-prone, so Gruntwork DevOps components are designed to "pre-solve" all of these issues by including: -1. **AWS Account Foundations.** Set up a best-practice multi-account setup, easily create new AWS accounts, and set secure account baselines. -1. **IaC Foundations.** Set up the foundational Terraform and Terragrunt coding patterns that your team can scale with -1. **CI/CD Foundations.** Roll out an infrastructure change in a way that meets the needs of your organization. -1. **Network Topology.** Protect your internal cloud resources from external access. -1. **Running apps.** Run your apps on EKS or ECS in a best-practices way. +- Pre-written recommendations on strategy +- A collection of pre-written IaC modules with extensive documentation +- A tool that directly solves the underlying need +- A streamlined approach to implementing the component in your environment +- A commitment by Gruntwork to update the component to match the latest best practices -## Working with components +When you set up a new DevOps component, you also have access to guidance from Gruntwork subject matter experts to make sure the component is applied correctly in your environment. -Some components can be installed independently, and some components work best bundled together. In addition, some components depend on others being in place. In general, we have identified two "layers" of components: - -### Layer 1 +## Available components -The first layer is the most foundational because all other infrastructure is built on top of it. It includes: +There are 5 DevOps components available today. -- AWS Account Foundations -- IaC foundations -- CI/CD Foundations -- Network Topology +1. **[Landing Zone.](../landing-zone)** Set up a best-practice AWS multi-account setup, easily create new AWS accounts, and set secure account baselines. +1. **[IaC Foundations.](../iac-foundations)** Set up the foundational Terraform and Terragrunt coding patterns that enable your team to scale. +1. **[Pipelines.](../pipelines)** Roll out an infrastructure change in a way that meets the needs of your organization. +1. **[Network Topology.](../network-topology)** Protect your internal cloud resources from external access. +1. **[Running Apps.](../running-apps)** Run your apps on EKS, ECS, or Lambda in a best-practices way. -### Layer 2 +All DevOps components are focused on AWS and Terraform/OpenTofu. We may add support for additional technologies in the future. -Once the layer-1 components are in place, the following components can be added: +## Building your own components -- Running apps +The Gruntwork DevOps components implement a meaningful portion of a modern cloud infrastructure, but not 100% of it. We expect you to build on top of the Gruntwork DevOps components by adding your own solutions to build out your full infrastructure. diff --git a/_docs-sources/foundations/overview/setup-order.md b/_docs-sources/foundations/overview/setup-order.md new file mode 100644 index 0000000000..1e1fea8c9a --- /dev/null +++ b/_docs-sources/foundations/overview/setup-order.md @@ -0,0 +1,46 @@ +# Setup order + +Some components can be installed on their own, while other components are inter-dependent and can only be installed as a bundle. + +## Installable independently + +The following components can each be installed on their own, without a dependency on any other component: + +- IaC Foundations +- Pipelines +- Network Topology +- Running Apps + +## Installable as bundles + +The following components can only be installed as a bundle: + +- Landing Zone, requires: + - IaC Foundations + - Pipelines + - Network Topology + +In practice, this means that if you want to install Gruntwork Landing Zone, you will need to install the full bundle of IaC Foundations, Pipelines, Landing Zone, and Network Topology. + +## Component layers + +It can also be helpful to think of components in "layers." + +### Layer 1: Foundations + +The first layer is the most foundational because all other infrastructure is built on top of it. It includes: + +- Landing Zone +- IaC Foundations +- Pipelines +- Network Topology + +You might also consider adding other components not yet supported by Gruntwork like FinOps or secrets management. + +### Layer 2: Applications + +Once the layer-1 components are in place, the following components can be added: + +- Running Apps + +In general, when you build new infrastructure, you first need your AWS accounts and their account baselines, a network topology, your IaC foundational patterns, and a Pipeline to deploy everything. Once those infrastructure foundations are in place, you are ready to add apps, data pipelines, and more. diff --git a/_docs-sources/foundations/pipelines/aws-authentication.md b/_docs-sources/foundations/pipelines/aws-authentication.md new file mode 100644 index 0000000000..1097063a8e --- /dev/null +++ b/_docs-sources/foundations/pipelines/aws-authentication.md @@ -0,0 +1,72 @@ +# Authenticating to AWS accounts + +Pipelines can intelligently figure out which AWS account to authenticate to based on the infrastructure changes proposed in your Pull Request. + +## How Pipelines authenticates to AWS + +To perform the actions that Pipelines detects, each AWS account needs to assume an AWS IAM Role using [OIDC](https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/configuring-openid-connect-in-amazon-web-services). To explain OIDC at a high level, AWS considers GitHub an "identity provider," trusts GitHub's request for a temporary IAM Role, and then issues AWS credentials that are valid for the duration of a GitHub Action workflow. + +Every time you create a new AWS account, you need to update the AWS OIDC configuration to create an IAM role for this account and allow it to be assumed by GitHub. When you use [Gruntwork Landing Zone](../landing-zone), this role is automatically created when [adding a new AWS account](../landing-zone/add-aws-account.md). + +## How Pipelines knows what AWS account to authenticate to + +Pipelines assumes that each top level directory in your `infrastructure-live` repository maps to a single AWS account, excluding the directory used for [module defaults](../iac-foundations/module_defaults). Each account-mapped directory must have an entry in the `/accounts.yml` file with a key matching the directory name and containing key/value pairs for the AWS account ID and the account's root user email address. + +For example, the following entry in `accounts.yml` would be mapped to a directory called `my-cool-account` in your `infrastructure-live` repository. + +```yml title=accounts.yml +"my-cool-account": + "email": "my-root-account-email@example.com" + "id": "123456789012" +``` + +```bash title="Infrastructure Live" +. +├── accounts.yml +├── _module_defaults +│ └── services +│ └── my-app.hcl +├── my-cool-account +│ └── us-east-1 +│ └── dev +│ └── database +│ └── terragrunt.hcl +``` + +## AWS account authentication when creating new AWS accounts + +:::note + +This section contains some advanced topics pertaining to Pipelines, how it differentiates between types of changes, and how it handles planning and applying changes. + +From the perspective of the end user, you will still only need one step in your CI job. Pipelines knows how to do the rest. + +::: + +Pipelines can handle two main types of infrastructure-change events: + +1. Adding/changing/deleting Terragrunt files +2. Creating new AWS accounts + +For the first type of infrastructure change (add/change/delete Terragrunt files) Pipelines authenticates to the specific AWS account where the affected resources live. For the second type of change (creating new AWS accounts), Pipelines uses the Gruntwork Landing Zone Management Account. + +### Gruntwork Landing Zone Management Account + +Gruntwork Landing Zone is built on top of AWS Control Tower, and AWS Control Tower requires that you create new AWS accounts using the [Control Tower Management AWS Account](https://docs.aws.amazon.com/controltower/latest/userguide/how-control-tower-works.html#what-is-mgmt). + +When new AWS account requests are approved and the AWS account is created, Pipelines will create a Pull Request that contains an account baseline. Since Pipelines does not yet have access to the AWS account, the baseline will be applied by authenticating to the Management Account. + +When applying the baseline and provisioning the Pipelines role in the new child account, Pipelines will first assume the management account Pipelines role, then will assume an automatically provisioned role in the child account (this process is known as [role chaining](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_terms-and-concepts.html)). Once Pipelines has temporary credentials in the new account, it will run a plan or apply to provision the requested resources. After the initial baseline and Pipelines role has been applied in the child account, all subsequent events for that account will be handled by the Pipelines role directly in the child account. + +:::note + +The AWS IAM Role for the management account must have permissions to provision new AWS Accounts (via AWS Control Tower) and assume a role in all child accounts. + +::: + +### Child Accounts + +Each child account (e.g., `dev`, `stage`, `prod,` etc.) contains an AWS IAM role that Pipelines can assume from GitHub Actions using OIDC. This role is automatically provisioned as a part of the [account baseline process](../landing-zone/add-aws-account). Once this role is in place in the child account, users may submit Pull Requests to add/modify/delete resources in the child account. + +When a Pull Request is created or synchronized, or a push to the `main` branch occurs, Pipelines will detect the changes, map them to the new account, assume the role in the child account, then run a `terragrunt plan` or `terragrunt apply` job (plan for PRs, apply for pushes to main). + diff --git a/_docs-sources/foundations/pipelines/index.md b/_docs-sources/foundations/pipelines/index.md new file mode 100644 index 0000000000..6b646ab1a2 --- /dev/null +++ b/_docs-sources/foundations/pipelines/index.md @@ -0,0 +1,9 @@ +# Pipelines for Account Factory + +**Gruntwork Pipelines is a framework for securely and responsibly rolling out infrastructure changes to your AWS environments using GitOps workflows.** We often refer to Gruntwork Pipelines as simply "Pipelines." + +These docs cover how Gruntwork Pipelines works specifically in the context of other DevOps components. For a more general overview of Gruntwork Pipelines, see the [dedicated documentation](/pipelines/overview). + +## Gruntwork Pipelines and Gruntwork Landing Zone + +In addition to rolling out infrastructure changes, Gruntwork Pipelines can also manage requests to create new AWS accounts as part of [Gruntwork Landing Zone](../../foundations/landing-zone) diff --git a/_docs-sources/foundations/running-apps/index.md b/_docs-sources/foundations/running-apps/index.md index 8fc403034a..066048a215 100644 --- a/_docs-sources/foundations/running-apps/index.md +++ b/_docs-sources/foundations/running-apps/index.md @@ -1,6 +1,17 @@ # About Running Apps -The Running Apps component is focused on enabling your organization's developers to run their apps and workloads. +The Running Apps component is focused on: + +- Enabling your organization's developers to run their containerized apps +- Enabling your organization's developers to run individual Lambda functions +- Adopting patterns for managing multiple clusters across many teams + +## What's included +- **Strategy.** For EKS, Gruntwork has an opinionated GitOps workflow we call "GruntOps." +- **IaC Modules.** Running Apps includes access to several IaC modules relating to EKS, ECS, and individual Lambda functions. +- **Tooling.** We make use of Kubernetes ecosystem tooling like Karpenter and Argo, in addition to our own tooling to improve the UX for ECS and Lambda. +- **Setup.** Our modules include documentation on how to configure an EKS or ECS cluster. +- **Updates.** Gruntwork publishes regular updates to EKS, ECS, and Lambda modules, and you can use [Patcher](/pipelines/overview/) to automatically update to the latest version of these modules. ## Available approaches @@ -37,8 +48,6 @@ We support invoking Lambda functions across a range of scenarios: For running complex Serverless applications, we recommend the [Serverless Framework](https://www.serverless.com/), which is available separately via Serverless, Inc. -## Out-of-the-box setup - -A Gruntwork subject matter expert in EKS, ECS, or Lambda will help you design a strategy for deploying apps using your preferred methodology, and provide support while you configure the Gruntwork modules to implement your strategy. +### Missing functionality -If you are looking to add functionality not yet supported by Gruntwork, you can either add this on your own, or we will consider those requests for our roadmap or as part of a separate, paid engagement. +If you are looking to add functionality not yet supported by Gruntwork, you can either [contribute a pull request](http://localhost:3000/library/usage/contributing), add the functionality on your own, work with Gruntwork as part of paid engagement, or request we add it to our roadmap. diff --git a/_docs-sources/pipelines/overview/index.md b/_docs-sources/pipelines/overview/index.md index 16cd9477da..a789c19fb1 100644 --- a/_docs-sources/pipelines/overview/index.md +++ b/_docs-sources/pipelines/overview/index.md @@ -1,18 +1,20 @@ # What is Gruntwork Pipelines? -Gruntwork Pipelines is a framework for securely deploying infrastructure as code to your AWS environments using GitOps workflows. Gruntwork Pipelines runs as a binary and series of steps in your CI system, determining what _actions_ need to be taken, in which _environments_, based on the [_infrastructure changes_](#infrastructure-change) that occurred. +**Gruntwork Pipelines is a framework for securely and responsibly rolling out infrastructure changes to your AWS environments using GitOps workflows.** We often refer to Gruntwork Pipelines as simply "Pipelines." + +Gruntwork Pipelines features first-class support for Terragrunt, runs on top of GitHub Actions, and uses a Pull Request-centric workflow. This means that all information about a proposed infrastructure change is added as comments to the applicable Pull Request, and that you apply the infrastructure change by taking action in the Pull Request. + +Gruntwork Pipelines runs as a binary and series of steps in GitHub Actions, determining what _actions_ need to be taken, in which _environments_, based on the [_infrastructure changes_](#infrastructure-change) that occurred. ## Common Terms -Gruntwork Pipelines uses specific terminology to describe code changes and operations that occur as a result of to changes. This section will familiarize you with the terminology used throughout the Gruntwork Pipelines documentation. +Gruntwork Pipelines uses specific terminology to describe code changes and operations that occur as a result of changes. This section will familiarize you with the terminology used throughout the Gruntwork Pipelines documentation. ### Infrastructure change When you edit any infrastructure-as-code that needs to be "applied" to your cloud account (e.g. AWS or GCP), you are making an _infrastructure change_. We sometime call these an "infra-change" for short. -Infra-changes can involve updates to OpenTF, Terraform, or Terragrunt code, or any other type of file that represents a desired state of your infrastructure and that needs to be somehow applied. A classic example is changing some variables on an instance of a Terraform module. By changing these variables you are essentially requesting that some API calls eventually be made to AWS to update your infrastructure in some way (in this case, by running `terraform apply` at some point). - -Gruntwork Pipelines assumes that infra-changes are committed via git, usually by first opening a Pull Request. When you open a Pull Request, you are essentially "proposing" an infra-change. +Infra-changes can involve updates to Terraform/OpenTofu or Terragrunt code, or any other type of file that represents a desired state of your infrastructure and that needs to be somehow applied. A classic example is changing some variables on an instance of a Terraform module. Gruntwork Pipelines assumes that infra-changes are committed via git, usually by first opening a Pull Request. When you open a Pull Request, you are essentially "proposing" an infra-change. ### Infrastructure change set diff --git a/docs/foundations/accounts/index.md b/docs/foundations/accounts/index.md deleted file mode 100644 index b9a39d4e29..0000000000 --- a/docs/foundations/accounts/index.md +++ /dev/null @@ -1,9 +0,0 @@ -# Account Foundations Placeholder - - - diff --git a/docs/foundations/accounts/manage-accounts.md b/docs/foundations/accounts/manage-accounts.md deleted file mode 100644 index e1cd855575..0000000000 --- a/docs/foundations/accounts/manage-accounts.md +++ /dev/null @@ -1,67 +0,0 @@ -import Admonition from "@theme/Admonition" - -# Manage your accounts - -Gruntwork's Control Tower integration provides an IaC-based (infrastructure as code) approach to many of your account management needs. Operations within those accounts can be grouped according to whether they may be performed in IaC, the AWS Console, or either. _When operations may be performed in either location, we strongly recommend using IaC._ - -## Prerequisites - -- An AWS account with AWS Control Tower set up -- Access to an IAM User or Role with administrative permissions to AWS Control Tower - -## How to manage your accounts - -Below you'll find a table with common account operations and the Gruntwork recommendation for if the operation should be done using IaC or in the AWS Console. When both options are available, using IaC is strongly recommended. You can explore more documentation for each operation by clicking on the operation name in the table. - -| Management Operation | Terraform (IaC) | AWS Console (ClickOps) | -| ------------------------------------------------------------------------------------------------------------------------------------------- | ---------------- | ---------------------- | -| [Create a new Organization Unit](https://docs.aws.amazon.com/controltower/latest/userguide/create-new-ou.html) (OU) | ❌ | ✅ | -| [Request a new account](./add-account.md) | ✅ | ❌ | -| [Create a new account](./add-account.md) | ✅ | ❌ | -| [Removing an account](./manage-accounts.md#removing-an-account) | ✅ | ❌ | -| [Renaming an account](https://docs.aws.amazon.com/controltower/latest/userguide/change-account-name.html) | ❌ | ✅ | -| [Update root account e-mail address](https://docs.aws.amazon.com/accounts/latest/reference/manage-acct-update-root-user.html) | ❌ | ✅ | -| [Modify account controls](https://docs.aws.amazon.com/controltower/latest/userguide/enable-controls-on-ou.html) | ❌ | ✅ | -| [Moving an account to a new Organizational Unit](./manage-accounts.md#un-managing-an-account) | ✅ (recommended) | ✅ (discouraged) | -| [Update account admin user in Account Access IAM Identity Center](./manage-accounts.md#un-managing-an-account) | ✅ (recommended) | ✅ (discouraged) | -| Granting additional users access to accounts in AWS IAM Identity Center | ✅ (recommended) | ✅ (discouraged) | - - -## Removing an account - -1. Create a branch and delete the directory in your `infrastructure-live` repository that corresponds to the account you would like to remove. - - -

This will delete all resources in the account. Make sure you are deleting the correct directory.

- - -Push your changes and create a PR. Pipelines will detect that an account is deleted. Verify that all expected resources will be removed in the `plan` output. - -Once you have confirmed everything looks as expected, merge the PR. - -1. Create a branch and delete the account request file in `_new_account_requests` - -This will de-provision the product in AWS Service Catalog, but will not delete the account. - -Push your changes and create a PR. Pipelines will detect that the account should be removed, which can be verified in the `plan` output. - -Once you have confirmed everything looks as expected, merge the PR. After `apply` runs, the account status should be `Suspended`. - -## Updating the account request - -You may update some attributes attributes of an AWS Account by modifying the the account request file in `_new_account_requests`. See below for steps to update each attribute. - -Start by creating a new branch that will contain your changes. - -1. Update the name of the OU by modifying the `organizational_unit_name` key -1. Updating the account admin user by modifying the `sso_user_first_name`, `sso_user_last_name`, and `sso_user_email` keys to the new users first name, last name, and email. - -After you have made your modifications, push your branch and create a pull request. Gruntwork Pipelines will detect the account changes and run a `plan` operation. Review the output of the `plan` to confirm the output is as expected. Once confirmed, merge the PR to apply the changes. - - - diff --git a/docs/foundations/ci-cd/index.md b/docs/foundations/ci-cd/index.md deleted file mode 100644 index 1fa1332e65..0000000000 --- a/docs/foundations/ci-cd/index.md +++ /dev/null @@ -1,76 +0,0 @@ -# Pipelines for Account Factory - -[Gruntwork Pipelines](../../pipelines/overview/) is a framework for deploying Infrastructure as Code changes and managing requests for and creation of AWS accounts, with first class support for [Terragrunt](https://terragrunt.gruntwork.io/). Pipelines enables you to use your preferred CI tool to detect changes to Infrastructure as Code Infrastructure Units (IUs — individual modules containing `terragrunt.hcl` files) and securely run `terragrunt plan` and `terragrunt apply` jobs on each IU. - -Pipelines runs as a step in your CI tool, determining the types of changes that were made (e.g., adding, changing, or deleting a module) and the action to take based on whether your change was made in a Pull Request (PR) or in a commit to your main branch (e.g., the PR has been merged). As an example, creating a pull request with changes that add a new IU containing an AWS RDS instance in your `dev` account would detect the new module and run a `terragrunt plan` in the directory containing the IU. - -:::note Supported CI systems - -GitHub Actions is currently the only CI system supported by Gruntwork Pipelines. Support for additional CI systems will be added in the future — let us know your preferred tool. In the meantime, our legacy version of Gruntwork Pipelines remains available and can be configured for the CI system of your choice. - -::: - -## Components of Pipelines - -Pipelines is a single binary that runs as a step in your CI tool. To perform the actions that Pipelines detects, each account will need an AWS IAM Role that allows GitHub Actions to assume it using OIDC. This role is automatically provisioned in new accounts that are provisioned when [adding an account](../accounts/add-account.md) using the Account Factory. - -Pipelines assumes that each top level directory in your `infrastructure-live` repository maps to a single AWS account, excluding the `_envcommon` directory. Each account mapped directory must have an entry in `accounts.yml` with a key matching the directory name and containing key/value pairs for the account ID and the e-mail for the root user of the account. - -For example, the following entry in `accounts.yml` would be mapped to a directory called `my-cool-account` in your `infrastructure-live` repository. - -```yml title=accounts.yml -"my-cool-account": - "email": "my-root-account-email@example.com" - "id": "123456789012" -``` - -```bash title="Infrastructure Live" -. -├── accounts.yml -├── _envcommon -│ └── services -│ └── my-app.hcl -├── my-cool-account -│ └── us-east-1 -│ └── dev -│ └── database -│ └── terragrunt.hcl -``` - -## Account Specific Pipelines - -:::note - -This section contains some advanced topics pertaining to Pipelines, how it differentiates between types of changes, and how it handles planning and applying changes. - -From the perspective of the end user, you will still only need one step in your CI job. Pipelines knows how to do the rest. - -::: - -Pipelines can handle two main types of events — additions, changes, or deletions of Terragrunt files, and creating new AWS accounts. With the exception of account creation, adding, changing, or deleting Terragrunt files (aka Infrastructure Units) is handled by the pipeline for each specific account. Provisioning new accounts and applying baselines in them is handled exclusively by the management account. - -### Management Account - -Provisioning AWS accounts must be handled by your Management AWS Account (e.g., your AWS Control Tower management account). When account requests are approved and the account is created, Pipelines will create a PR that contains an account baseline. Since Pipelines doesn’t have access to the account yet, the baseline will be applied using the Management Account Pipeline. - -When applying the baseline and provisioning the Pipelines role in the new child account, Pipelines will first assume the management account Pipelines role, then will assume an automatically provisioned role in the child account (this process is known as [role chaining](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_terms-and-concepts.html)). Once Pipelines has temporary credentials in the new account, it will run a plan or apply to provision the requested resources. After the initial baseline and Pipelines role has been applied in the child account, all subsequent events for that account will be handled by the Pipelines role directly in the child account. - -:::note - -The AWS IAM Role for the management account must have permissions to provision new AWS Accounts (via AWS Control Tower) and assume a role in all child accounts. - -::: - -### Child Accounts - -Each child account (e.g., `dev`, `stage`, `prod,` etc.) contains an AWS IAM role that Pipelines can assume from GitHub Actions using OIDC. This role is automatically provisioned as a part of the [account baseline process](../accounts/add-account#4-review-and-merge-the-account-baseline-pr). Once this role is in place in the child account, users may submit pull requests with new Infrastructure Units in the child account. - -When a Pull Request is created or synchronized, or a push to main occurs, Pipelines will detect the changes, map them to the new account, assume the role in the child account, then run a `terragrunt plan` or `terragrunt apply` job (plan for PRs, apply for pushes to main). - - - diff --git a/docs/foundations/iac/folder_structure.md b/docs/foundations/iac-foundations/folder-structure.md similarity index 100% rename from docs/foundations/iac/folder_structure.md rename to docs/foundations/iac-foundations/folder-structure.md diff --git a/docs/foundations/iac-foundations/index.md b/docs/foundations/iac-foundations/index.md new file mode 100644 index 0000000000..a17138e9c9 --- /dev/null +++ b/docs/foundations/iac-foundations/index.md @@ -0,0 +1,35 @@ +# About IaC Foundations + +The IaC Foundations component is focused on: + +- Teaching you the considerations to think about when coming up with your foundational Terraform/OpenTofu and Terragrunt patterns +- Giving you a fully configured set of git repositories with an initial folder structure + +## What's included + +- **Strategy.** We recommend core patterns your Terragrunt and Terraform/OpenTofu git repo needs to incorporate to scale effectively. +- **IaC Modules.** No IaC modules are needed for this component. +- **Tooling.** We recommend Terragrunt to effectively use Terraform/OpenTofu at scale. +- **Setup.** We grant you access to a sophisticated git repo template that includes customization options and generates your repo code. +- **Updates.** We publish ongoing updates to IaC foundational patterns and will write [patches](/patcher) if applicable to adopt those changes. + +## IaC foundations considerations + +When creating your initial Terraform/OpenTofu and Terragrunt infrastructure, there are a number of patterns you need to consider how to solve, including: + +- How to create the backend (e.g. S3 bucket) for storing Terraform state +- How to structure your folder hierarchy +- How to handle tagging and labels +- Whether to use branches per environment +- How to handle global variables +- How to handle module default values + +This component includes either pre-baked implementations that address these considerations, or written guidance on how to incorporate them yourself. + + + diff --git a/docs/foundations/iac-foundations/initial-setup.md b/docs/foundations/iac-foundations/initial-setup.md new file mode 100644 index 0000000000..97069aa607 --- /dev/null +++ b/docs/foundations/iac-foundations/initial-setup.md @@ -0,0 +1,35 @@ +# Initial setup + +To set up IaC Foundations, we use three pre-configured git repository templates that include best practices and also allow for customization. + +For each repository below, navigate to the template repository and select **Use this template** -> **Create a new Repository**. This will initiate repository creation. You should select your org as the owner, add a description if you like, make sure you are creating a **private** repo, and click **Create repository**. + +The repository template will be created, and you can follow the instructions in the `README` to bootstrap your IaC Foundations. Gruntwork is available to assist with questions around other patterns as they arise. + +### Infrastructure Live Template + +_[https://github.com/gruntwork-io/infrastructure-live-template](https://github.com/gruntwork-io/infrastructure-live-template)_ + +This template creates an infrastructure-live repository with scaffolding for a best practices Terragrunt configuration, including patterns for module defaults, global variables, and account baselines. It also configures Gruntwork Pipelines, which is easy to remove if you don't want it. + +### Infrastructure Modules Template + +_[https://github.com/gruntwork-io/infrastructure-modules-template](https://github.com/gruntwork-io/infrastructure-modules-template)_ + +This template creates an empty infrastructure-modules repository that will be used to store Terraform/OpenTofu modules that your organization has authored and intends to use within your organization. + +### Infrastructure Pipelines Template + +_[https://github.com/gruntwork-io/infrastructure-pipelines-template](https://github.com/gruntwork-io/infrastructure-pipelines-template)_ + +This template is only necessary if you plan on implementing [Gruntwork Pipelines](../pipelines). + + + + + diff --git a/docs/foundations/iac/module_defaults/defining.md b/docs/foundations/iac-foundations/module_defaults/defining.md similarity index 100% rename from docs/foundations/iac/module_defaults/defining.md rename to docs/foundations/iac-foundations/module_defaults/defining.md diff --git a/docs/foundations/iac/module_defaults/index.md b/docs/foundations/iac-foundations/module_defaults/index.md similarity index 100% rename from docs/foundations/iac/module_defaults/index.md rename to docs/foundations/iac-foundations/module_defaults/index.md diff --git a/docs/foundations/iac/module_defaults/usage.md b/docs/foundations/iac-foundations/module_defaults/usage.md similarity index 100% rename from docs/foundations/iac/module_defaults/usage.md rename to docs/foundations/iac-foundations/module_defaults/usage.md diff --git a/docs/foundations/iac/index.md b/docs/foundations/iac/index.md deleted file mode 100644 index 523f6fa840..0000000000 --- a/docs/foundations/iac/index.md +++ /dev/null @@ -1,45 +0,0 @@ -# About IaC Foundations - -The IaC Foundations component is focused on: - -- Teaching you the considerations to think about when coming up with your foundational Terraform and Terragrunt patterns -- Giving you a fully configured set of git repositories with an initial folder structure - -## IaC foundations considerations - -When creating your initial Terraform and Terragrunt and infrastructure, there are a number of patterns you need to consider how to solve. These include: - -- How to create the backend (e.g. S3 bucket) for storing Terraform state -- How to structure your folder hierarchy -- How to handle tagging and labels -- Whether to use branches per environment -- How to handle global variables -- How to handle module default values - -## Initial setup - -We provide 3 template repos to subscribers that implement best practices regarding the considerations above. - -For each of the repositories below, navigate to the template repository and select **Use this template** -> **Create a new Repository**. -This will initiate repository creation. You should select your org as the owner, add a description if you like, make sure you are creating a **private** repo, and click **Create repository**. - -The repository template will be created, and you can follow the instructions in the README to bootstrap your IaC Foundations. - -### [Infrastructure Live Template](https://github.com/gruntwork-io/infrastructure-live-template) -This [template](https://github.com/gruntwork-io/infrastructure-live-template) creates an infrastructure live repository with scaffolding for common module defaults, account baselines, and Gruntwork Pipelines - -### [Infrastructure Modules Template](https://github.com/gruntwork-io/infrastructure-modules-template) -This [template](https://github.com/gruntwork-io/infrastructure-modules-template) creates an empty infrastructure modules repository for populating shared modules within your organization. - -### [Infrastructure Pipelines Template](https://github.com/gruntwork-io/infrastructure-pipelines-template) - -This [template](https://github.com/gruntwork-io/infrastructure-pipelines-template) is only necessary if you plan on implementing [Gruntwork Pipelines](../ci-cd). - - - - diff --git a/_docs-sources/foundations/accounts/add-account.md b/docs/foundations/landing-zone/add-aws-account.md similarity index 80% rename from _docs-sources/foundations/accounts/add-account.md rename to docs/foundations/landing-zone/add-aws-account.md index b4e51e52e7..5026db9dbf 100644 --- a/_docs-sources/foundations/accounts/add-account.md +++ b/docs/foundations/landing-zone/add-aws-account.md @@ -1,6 +1,6 @@ -# Add a new account +# Add a new AWS account -This document provides instructions for provisioning a new account in a Control Tower managed organization using Gruntwork. The described workflow gives you the flexibility to require approval for all new account requests in accordance with the permissions configured in your repository. +This document provides instructions for provisioning a new AWS account using Gruntwork Landing Zone. The described workflow gives you the flexibility to require approval for all new AWS account requests in accordance with the permissions configured in your git repository. ## Prerequisites @@ -8,9 +8,9 @@ Before proceeding, ensure you have an `infrastructure-live` repository which con - The [`control-tower-multi-account-factory` module](https://GitHub.com/gruntwork-io/terraform-aws-control-tower/tree/main/modules/landingzone/control-tower-multi-account-factory) configured in the root account -- An installation of [Gruntwork Pipelines](https://LINK-TO-VALID-DOC) +- An installation of [Gruntwork Pipelines](/pipelines/overview) -## 1. Create an account request file +## 1. Create an AWS account request file To initiate the process, create an `account-.yml` file in the `_new-account-requests` folder located in the root of your `infrastructure-live` repository. This file will be used to create a pull request and add the new account to your organization. The file should have the following format: @@ -40,7 +40,7 @@ requested_by: :::note -If you have configured GitHub Actions in your `infrastructure-live` repository with an [Account Factory workflow](https://LINK-TO-VALID-DOC), you can invoke that workflow via the GitHub UI or programmatically. This workflow will automatically create the account request file and open a pull request on your behalf. After doing so, jump to [step 3](#3-review-and-merge-the-account-request-pr). +If you have configured GitHub Actions in your `infrastructure-live` repository with a Gruntwork Landing Zone Account Factory workflow, you can invoke that workflow via the GitHub UI or programmatically. This workflow will automatically create the account request file and open a pull request on your behalf. After doing so, jump to [step 3](#3-review-and-merge-the-account-request-pr). ::: @@ -48,20 +48,8 @@ If you have configured GitHub Actions in your `infrastructure-live` repository w Next, create a pull request containing the new account request file. This action will trigger the Gruntwork pipeline to `terragrunt plan` the new account and update the pull request with the plan output. -:::caution - -Currently the output is still only available in the `infrastructure-pipelines` repository’ GitHub Actions logs. This will be updated shortly to be available in the pull request and this notice will be removed. - -::: - ## 3. Review and merge the account request PR -:::caution - -Currently the output is still only available in the `infrastructure-pipelines` repository’ GitHub Actions logs. This will be updated shortly to be available in the pull request and this notice will be removed. - -::: - Review the `plan` output in the pull request. Once satisfied, merge the pull request to trigger creation of the new account. Gruntwork Pipelines will run `terragrunt apply` to create the new account in Control Tower. :::tip @@ -88,3 +76,11 @@ After the pipeline for the new account request completes, the new account will e - A new OIDC role required to install the baseline and enable Gruntwork Pipelines to run within the new account When you merge this pull request, Gruntwork Pipelines will automatically deploy the Gruntwork account baselines into the new account and provision a role that Pipelines can assume to deploy resources into this account. Once this process completes, you may access your account and leverage Gruntwork Pipelines to make any further changes to the infrastructure in your new account. + + + diff --git a/_docs-sources/foundations/accounts/enable-control-tower.md b/docs/foundations/landing-zone/enable-control-tower.md similarity index 90% rename from _docs-sources/foundations/accounts/enable-control-tower.md rename to docs/foundations/landing-zone/enable-control-tower.md index 92c6179ce4..65e2b067c4 100644 --- a/_docs-sources/foundations/accounts/enable-control-tower.md +++ b/docs/foundations/landing-zone/enable-control-tower.md @@ -1,19 +1,11 @@ # Enable Control Tower -Enabling Control Tower is the first step in getting started with Gruntwork Account Foundations. It must be completed +Enabling Control Tower is the first step in getting started with Gruntwork Landing Zone. It must be completed before infrastructure as code is generated for your infrastructure-live repository. -:::info - -[AWS Control Tower](https://docs.aws.amazon.com/controltower/latest/userguide/what-is-control-tower.html) is an AWS service -that orchestrates accounts, handling creation, deletion, security policies, and more. It serves as the base layer of -Gruntwork Account Foundations. - -::: - ## Prerequisites -In order to enable control tower you will need the resources described in [Prerequisites.](/foundations/accounts/prerequisites) +In order to enable AWS Control Tower you will need the resources described in [Prerequisites](./prerequisites). ## Enable AWS Control Tower @@ -107,7 +99,7 @@ Control Tower Creation will take around an hour to complete ## Initial Configuration Now that Control Tower is enabled in your root account, there are a few configuration changes that need to be made to -prepare the landing zone for Gruntwork Account Foundations. +prepare for Gruntwork Landing Zone. 1. Navigate to the [AWS Control Tower Organization Dashboard](https://console.aws.amazon.com/controltower/home/organization) 1. [Turn off the default VPC created for new accounts](https://docs.aws.amazon.com/controltower/latest/userguide/configure-without-vpc.html#create-without-vpc). Gruntwork VPCs will be created for each account using terraform. @@ -132,4 +124,12 @@ Control Tower is now configured! Next you should consider: - [Configuring IAM Identity Center](https://docs.aws.amazon.com/singlesignon/latest/userguide/get-started-choose-identity-source.html) for Access Control. - [Configuring any controls or SCPs](https://docs.aws.amazon.com/controltower/latest/userguide/controls.html) your organization requires. -- [Set up your Gruntwork IaC Foundations](/foundations/iac) +- [Setting up your Gruntwork IaC Foundations](../iac-foundations) + + + diff --git a/docs/foundations/landing-zone/index.md b/docs/foundations/landing-zone/index.md new file mode 100644 index 0000000000..4f4e86cac8 --- /dev/null +++ b/docs/foundations/landing-zone/index.md @@ -0,0 +1,50 @@ +# About Landing Zone + +The Landing Zone component is focused on: + +- Creating an initial best-practice, AWS multi-account setup +- Putting in place a system for vending new AWS accounts (commonly known as an "account factory") +- Installing secure AWS account baselines compliant with the [CIS AWS Foundations Benchmark](https://www.cisecurity.org/benchmark/amazon_web_services). + +Gruntwork Landing Zone is intended to be deployed as your new paradigm for creating and managing AWS accounts. Once set up, you can import existing AWS accounts. + +## What's included + +- **Strategy.** Gruntwork has designed an approach that extends AWS Control Tower to support the full set of needs around AWS account management with Terraform/OpenTofu and GitHub. +- **IaC Modules.** Gruntwork Landing Zone includes modules to configure Control Tower, configure AWS IAM Identity Center, and install AWS account baselines that meet the CIS AWS Foundations Benchmark. +- **Tooling.** Gruntwork Landing Zone makes use of Gruntwork Pipelines and GitHub Action to support the account request and review process. +- **Setup.** Installing Gruntwork Landing Zone is fully documented. +- **Updates.** Gruntwork publishes regular updates to all Landing Zone modules, including support for the latest versions of the CIS AWS Foundations Benchmark, and ongoing updates to our Landing Zone workflows. + +## Extending AWS Control Tower + +Gruntwork Landing Zone builds on AWS Control Tower to add important new functionality. + +### What is AWS Control Tower? + +AWS Control Tower helps you set up and govern AWS accounts, and gives you a centralized way to manage compliance and configuration across your AWS accounts. But Control Tower is fundamentally a Platform-as-a-Service (PaaS) paradigm where you use "ClickOps" to create and configure AWS accounts. That stands in contrast to the vision behind IaC tools like Terraform, which are built on the philosophy that all Cloud resources are described in code. It can also be challenging to customize Control Tower and maintain it over time. + +### Additional functionality + +Gruntwork Landing Zone directly addresses these limitations by extending Control Tower with the following functionality: + +1. Configure and customize new AWS accounts using Terraform/OpenTofu +1. Have all new AWS accounts meet the CIS AWS Foundations Benchmark out of the box +1. Request new AWS accounts using a simple YAML file, GitHub Actions, or any system that can trigger a GitHub Action (e.g. ServiceNow) +1. Review and customize all new AWS account requests using GitHub Pull Request functionality +1. Define a customized account baseline unique to your organization +1. Enable AWS account baselines to stay up to date automatically using [Patcher](./patcher) + +## Greenfield vs. brownfield + +Gruntwork Landing Zone is optimized for situations where you are creating a "new world" infrastructure. Once your new world is stable, you can then import existing AWS accounts into Gruntwork Landing Zone, and migrate existing resources as needed. + +While it possible to migrate an existing Control Tower deployment into Gruntwork Landing Zone, we have not yet defined a formal process for this and we do not currently recommend this. That being said, other Gruntwork DevOps components can still be installed alongside your existing non-Gruntwork Landing Zone solution. + + + diff --git a/docs/foundations/landing-zone/manage-accounts.md b/docs/foundations/landing-zone/manage-accounts.md new file mode 100644 index 0000000000..f7ea6548c4 --- /dev/null +++ b/docs/foundations/landing-zone/manage-accounts.md @@ -0,0 +1,75 @@ +# Managing your AWS accounts + +Over time you will need to run various operations on your AWS accounts such as requesting new accounts, creating new accounts, renaming accounts, etc. In Gruntwork Landing Zone, some AWS account management operations should only be done using IaC, some can only be done using ClickOps, and some can be done using either. + +In this page, we review which mode (IaC or ClickOps) to use for each AWS account operation. _When operations may be performed using either mode, we strongly recommend using IaC._ + +## Prerequisites + +This page applies to users who are: +- Actively running Gruntwork Landing Zone +- Have access to an AWS user or IAM Role with administrative permissions to AWS Control Tower + +## When to use IaC vs. ClickOps + +Below you'll find a table with common AWS account operations and the Gruntwork recommendation for if the operation should be done using IaC or in the AWS Console. When both options are available, using IaC is strongly recommended. You can explore more documentation for each operation by clicking on the operation name in the table. + +- ✅ means that the operation should or can only be done using the given mode. +- ❌ means that the operation should not or cannot be done using the given mode. + +| Management Operation | Terraform (IaC) | AWS Console (ClickOps) | +|-------------------------------------------------------------------------------------------------------------------------------| ---------------- | ---------------------- | +| [Create a new Organization Unit](https://docs.aws.amazon.com/controltower/latest/userguide/create-new-ou.html) (OU) | ❌ | ✅ | +| Request a new account | ✅ | ❌ | +| Create a new account | ✅ | ❌ | +| [Remove an account](#remove-an-aws-account) | ✅ | ❌ | +| [Rename an account](https://docs.aws.amazon.com/controltower/latest/userguide/change-account-name.html) | ❌ | ✅ | +| [Update root account e-mail address](https://docs.aws.amazon.com/accounts/latest/reference/manage-acct-update-root-user.html) | ❌ | ✅ | +| [Modify account controls](https://docs.aws.amazon.com/controltower/latest/userguide/enable-controls-on-ou.html) | ❌ | ✅ | +| Moving an account to a new Organizational Unit | ✅ (recommended) | ✅ (discouraged) | +| Update account admin user in Account Access IAM Identity Center | ✅ (recommended) | ✅ (discouraged) | +| Granting additional users access to accounts in AWS IAM Identity Center | ✅ (recommended) | ✅ (discouraged) | + + +## AWS account management operations + +### Remove an AWS account + +This operation removes an AWS account from AWS Control Tower but does _not_ delete the AWS account. + +1. Create a branch and delete the directory in your `infrastructure-live` repository that corresponds to the account you would like to remove. + + :::warning + This will delete all resources in the account. Make sure you are deleting the correct directory. + ::: + + Push your changes and create a PR. Pipelines will detect that an account is deleted. Verify that all expected resources will be removed in the `plan` output. + + Once you have confirmed everything looks as expected, merge the PR. + +1. Create a branch and delete the account request file in `_new_account_requests`. + + This will de-provision the product in AWS Service Catalog, but will not delete the account. + + Push your changes and create a PR. Pipelines will detect that the account should be removed, which can be verified in the `plan` output. + + Once you have confirmed everything looks as expected, merge the PR. After `apply` runs, the account status should be `Suspended`. + +### Update the new AWS account request + +You may update some attributes of an AWS Account by modifying the account request file in `_new_account_requests`. See below for steps to update each attribute. + +Start by creating a new branch that will contain your changes. + +1. Update the name of the OU by modifying the `organizational_unit_name` key +1. Updating the account admin user by modifying the `sso_user_first_name`, `sso_user_last_name`, and `sso_user_email` keys to the new users first name, last name, and email. + +After you have made your modifications, push your branch and create a pull request. Gruntwork Pipelines will detect the account changes and run a `plan` operation. Review the output of the `plan` to confirm the output is as expected. Once confirmed, merge the PR to apply the changes. + + + diff --git a/_docs-sources/foundations/accounts/prerequisites.md b/docs/foundations/landing-zone/prerequisites.md similarity index 82% rename from _docs-sources/foundations/accounts/prerequisites.md rename to docs/foundations/landing-zone/prerequisites.md index 14e5753f43..7272f2c2d3 100644 --- a/_docs-sources/foundations/accounts/prerequisites.md +++ b/docs/foundations/landing-zone/prerequisites.md @@ -1,6 +1,14 @@ # Prerequisites -In order to enable control tower you will need the following: +The first step to using Gruntwork Landing Zone is to use AWS Control Tower to create a new multi-account setup, which requires that you meet the prerequisites below. + +## Gruntwork Pipelines + +In order to get the full functionality of Gruntwork Landing Zone, you must use [Gruntwork Pipelines](/pipelines/overview/). + +## Enabling AWS Control Tower + +To use AWS Control Tower you must first enable it, which requires that you have the following: 1. A new AWS Account and a user with administrator permissions. (We recommend using an IAM user with admin permissions rather than the root user) @@ -15,7 +23,7 @@ In order to enable control tower you will need the following: Your home Region is the AWS Region in which you'll run most of your workloads or store most of your data. It **cannot** be changed after you've set up your AWS Control Tower landing zone. For more information about how to choose a home Region, see [Administrative tips for landing zone setup](https://docs.aws.amazon.com/controltower/latest/userguide/tips-for-admin-setup.html). ::: -## KMS Key Creation +### KMS Key Creation :::info For more help setting up KMS see the AWS docs: [Guidance for KMS keys](https://docs.aws.amazon.com/en_us/controltower/latest/userguide//kms-guidance.html) ::: @@ -72,3 +80,14 @@ For more help setting up KMS see the AWS docs: [Guidance for KMS keys](https://d 1. Click `Save Changes` +## Next steps + +Now that you've met the prerequisites, it's time to enable Control Tower. + + + diff --git a/docs/foundations/overview/index.md b/docs/foundations/overview/index.md index 3e6732059d..1212c02533 100644 --- a/docs/foundations/overview/index.md +++ b/docs/foundations/overview/index.md @@ -1,40 +1,37 @@ # DevOps Components -Gruntwork DevOps Foundations is made up of a collection of _DevOps components._ A DevOps component is an essential element of modern infrastructure for which Gruntwork has pre-built IaC modules and pre-written documentation to reflect our recommended best practices and support fast setup. +**Gruntwork DevOps Foundations is a collection of _DevOps components_ that you use as building blocks to assemble your own best-practice infrastructure.** -## Available components - -The Gruntwork DevOps components include: +In a modern cloud infrastructure there are many component parts, ranging from the infrastructure pipeline to secrets management to FinOps to how you deploy apps. Setting up and managing each component requires that you understand the core infrastructure need, develop a strategy for addressing it, implement a solution, and then maintain it forever. Doing this on your own is expensive and error-prone, so Gruntwork DevOps components are designed to "pre-solve" all of these issues by including: -1. **AWS Account Foundations.** Set up a best-practice multi-account setup, easily create new AWS accounts, and set secure account baselines. -1. **IaC Foundations.** Set up the foundational Terraform and Terragrunt coding patterns that your team can scale with -1. **CI/CD Foundations.** Roll out an infrastructure change in a way that meets the needs of your organization. -1. **Network Topology.** Protect your internal cloud resources from external access. -1. **Running apps.** Run your apps on EKS or ECS in a best-practices way. +- Pre-written recommendations on strategy +- A collection of pre-written IaC modules with extensive documentation +- A tool that directly solves the underlying need +- A streamlined approach to implementing the component in your environment +- A commitment by Gruntwork to update the component to match the latest best practices -## Working with components +When you set up a new DevOps component, you also have access to guidance from Gruntwork subject matter experts to make sure the component is applied correctly in your environment. -Some components can be installed independently, and some components work best bundled together. In addition, some components depend on others being in place. In general, we have identified two "layers" of components: - -### Layer 1 +## Available components -The first layer is the most foundational because all other infrastructure is built on top of it. It includes: +There are 5 DevOps components available today. -- AWS Account Foundations -- IaC foundations -- CI/CD Foundations -- Network Topology +1. **[Landing Zone.](../landing-zone)** Set up a best-practice AWS multi-account setup, easily create new AWS accounts, and set secure account baselines. +1. **[IaC Foundations.](../iac-foundations)** Set up the foundational Terraform and Terragrunt coding patterns that enable your team to scale. +1. **[Pipelines.](../pipelines)** Roll out an infrastructure change in a way that meets the needs of your organization. +1. **[Network Topology.](../network-topology)** Protect your internal cloud resources from external access. +1. **[Running Apps.](../running-apps)** Run your apps on EKS, ECS, or Lambda in a best-practices way. -### Layer 2 +All DevOps components are focused on AWS and Terraform/OpenTofu. We may add support for additional technologies in the future. -Once the layer-1 components are in place, the following components can be added: +## Building your own components -- Running apps +The Gruntwork DevOps components implement a meaningful portion of a modern cloud infrastructure, but not 100% of it. We expect you to build on top of the Gruntwork DevOps components by adding your own solutions to build out your full infrastructure. diff --git a/docs/foundations/overview/setup-order.md b/docs/foundations/overview/setup-order.md new file mode 100644 index 0000000000..9739489adf --- /dev/null +++ b/docs/foundations/overview/setup-order.md @@ -0,0 +1,54 @@ +# Setup order + +Some components can be installed on their own, while other components are inter-dependent and can only be installed as a bundle. + +## Installable independently + +The following components can each be installed on their own, without a dependency on any other component: + +- IaC Foundations +- Pipelines +- Network Topology +- Running Apps + +## Installable as bundles + +The following components can only be installed as a bundle: + +- Landing Zone, requires: + - IaC Foundations + - Pipelines + - Network Topology + +In practice, this means that if you want to install Gruntwork Landing Zone, you will need to install the full bundle of IaC Foundations, Pipelines, Landing Zone, and Network Topology. + +## Component layers + +It can also be helpful to think of components in "layers." + +### Layer 1: Foundations + +The first layer is the most foundational because all other infrastructure is built on top of it. It includes: + +- Landing Zone +- IaC Foundations +- Pipelines +- Network Topology + +You might also consider adding other components not yet supported by Gruntwork like FinOps or secrets management. + +### Layer 2: Applications + +Once the layer-1 components are in place, the following components can be added: + +- Running Apps + +In general, when you build new infrastructure, you first need your AWS accounts and their account baselines, a network topology, your IaC foundational patterns, and a Pipeline to deploy everything. Once those infrastructure foundations are in place, you are ready to add apps, data pipelines, and more. + + + diff --git a/docs/foundations/pipelines/aws-authentication.md b/docs/foundations/pipelines/aws-authentication.md new file mode 100644 index 0000000000..221a0097db --- /dev/null +++ b/docs/foundations/pipelines/aws-authentication.md @@ -0,0 +1,80 @@ +# Authenticating to AWS accounts + +Pipelines can intelligently figure out which AWS account to authenticate to based on the infrastructure changes proposed in your Pull Request. + +## How Pipelines authenticates to AWS + +To perform the actions that Pipelines detects, each AWS account needs to assume an AWS IAM Role using [OIDC](https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/configuring-openid-connect-in-amazon-web-services). To explain OIDC at a high level, AWS considers GitHub an "identity provider," trusts GitHub's request for a temporary IAM Role, and then issues AWS credentials that are valid for the duration of a GitHub Action workflow. + +Every time you create a new AWS account, you need to update the AWS OIDC configuration to create an IAM role for this account and allow it to be assumed by GitHub. When you use [Gruntwork Landing Zone](../landing-zone), this role is automatically created when [adding a new AWS account](../landing-zone/add-aws-account.md). + +## How Pipelines knows what AWS account to authenticate to + +Pipelines assumes that each top level directory in your `infrastructure-live` repository maps to a single AWS account, excluding the directory used for [module defaults](../iac-foundations/module_defaults). Each account-mapped directory must have an entry in the `/accounts.yml` file with a key matching the directory name and containing key/value pairs for the AWS account ID and the account's root user email address. + +For example, the following entry in `accounts.yml` would be mapped to a directory called `my-cool-account` in your `infrastructure-live` repository. + +```yml title=accounts.yml +"my-cool-account": + "email": "my-root-account-email@example.com" + "id": "123456789012" +``` + +```bash title="Infrastructure Live" +. +├── accounts.yml +├── _module_defaults +│ └── services +│ └── my-app.hcl +├── my-cool-account +│ └── us-east-1 +│ └── dev +│ └── database +│ └── terragrunt.hcl +``` + +## AWS account authentication when creating new AWS accounts + +:::note + +This section contains some advanced topics pertaining to Pipelines, how it differentiates between types of changes, and how it handles planning and applying changes. + +From the perspective of the end user, you will still only need one step in your CI job. Pipelines knows how to do the rest. + +::: + +Pipelines can handle two main types of infrastructure-change events: + +1. Adding/changing/deleting Terragrunt files +2. Creating new AWS accounts + +For the first type of infrastructure change (add/change/delete Terragrunt files) Pipelines authenticates to the specific AWS account where the affected resources live. For the second type of change (creating new AWS accounts), Pipelines uses the Gruntwork Landing Zone Management Account. + +### Gruntwork Landing Zone Management Account + +Gruntwork Landing Zone is built on top of AWS Control Tower, and AWS Control Tower requires that you create new AWS accounts using the [Control Tower Management AWS Account](https://docs.aws.amazon.com/controltower/latest/userguide/how-control-tower-works.html#what-is-mgmt). + +When new AWS account requests are approved and the AWS account is created, Pipelines will create a Pull Request that contains an account baseline. Since Pipelines does not yet have access to the AWS account, the baseline will be applied by authenticating to the Management Account. + +When applying the baseline and provisioning the Pipelines role in the new child account, Pipelines will first assume the management account Pipelines role, then will assume an automatically provisioned role in the child account (this process is known as [role chaining](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_terms-and-concepts.html)). Once Pipelines has temporary credentials in the new account, it will run a plan or apply to provision the requested resources. After the initial baseline and Pipelines role has been applied in the child account, all subsequent events for that account will be handled by the Pipelines role directly in the child account. + +:::note + +The AWS IAM Role for the management account must have permissions to provision new AWS Accounts (via AWS Control Tower) and assume a role in all child accounts. + +::: + +### Child Accounts + +Each child account (e.g., `dev`, `stage`, `prod,` etc.) contains an AWS IAM role that Pipelines can assume from GitHub Actions using OIDC. This role is automatically provisioned as a part of the [account baseline process](../landing-zone/add-aws-account). Once this role is in place in the child account, users may submit Pull Requests to add/modify/delete resources in the child account. + +When a Pull Request is created or synchronized, or a push to the `main` branch occurs, Pipelines will detect the changes, map them to the new account, assume the role in the child account, then run a `terragrunt plan` or `terragrunt apply` job (plan for PRs, apply for pushes to main). + + + + diff --git a/docs/foundations/pipelines/index.md b/docs/foundations/pipelines/index.md new file mode 100644 index 0000000000..e8f00bab24 --- /dev/null +++ b/docs/foundations/pipelines/index.md @@ -0,0 +1,17 @@ +# Pipelines for Account Factory + +**Gruntwork Pipelines is a framework for securely and responsibly rolling out infrastructure changes to your AWS environments using GitOps workflows.** We often refer to Gruntwork Pipelines as simply "Pipelines." + +These docs cover how Gruntwork Pipelines works specifically in the context of other DevOps components. For a more general overview of Gruntwork Pipelines, see the [dedicated documentation](/pipelines/overview). + +## Gruntwork Pipelines and Gruntwork Landing Zone + +In addition to rolling out infrastructure changes, Gruntwork Pipelines can also manage requests to create new AWS accounts as part of [Gruntwork Landing Zone](../../foundations/landing-zone) + + + diff --git a/docs/foundations/running-apps/index.md b/docs/foundations/running-apps/index.md index bf97060f38..8d71cba596 100644 --- a/docs/foundations/running-apps/index.md +++ b/docs/foundations/running-apps/index.md @@ -1,6 +1,17 @@ # About Running Apps -The Running Apps component is focused on enabling your organization's developers to run their apps and workloads. +The Running Apps component is focused on: + +- Enabling your organization's developers to run their containerized apps +- Enabling your organization's developers to run individual Lambda functions +- Adopting patterns for managing multiple clusters across many teams + +## What's included +- **Strategy.** For EKS, Gruntwork has an opinionated GitOps workflow we call "GruntOps." +- **IaC Modules.** Running Apps includes access to several IaC modules relating to EKS, ECS, and individual Lambda functions. +- **Tooling.** We make use of Kubernetes ecosystem tooling like Karpenter and Argo, in addition to our own tooling to improve the UX for ECS and Lambda. +- **Setup.** Our modules include documentation on how to configure an EKS or ECS cluster. +- **Updates.** Gruntwork publishes regular updates to EKS, ECS, and Lambda modules, and you can use [Patcher](/pipelines/overview/) to automatically update to the latest version of these modules. ## Available approaches @@ -37,16 +48,14 @@ We support invoking Lambda functions across a range of scenarios: For running complex Serverless applications, we recommend the [Serverless Framework](https://www.serverless.com/), which is available separately via Serverless, Inc. -## Out-of-the-box setup - -A Gruntwork subject matter expert in EKS, ECS, or Lambda will help you design a strategy for deploying apps using your preferred methodology, and provide support while you configure the Gruntwork modules to implement your strategy. +### Missing functionality -If you are looking to add functionality not yet supported by Gruntwork, you can either add this on your own, or we will consider those requests for our roadmap or as part of a separate, paid engagement. +If you are looking to add functionality not yet supported by Gruntwork, you can either [contribute a pull request](http://localhost:3000/library/usage/contributing), add the functionality on your own, work with Gruntwork as part of paid engagement, or request we add it to our roadmap. diff --git a/docs/pipelines/overview/index.md b/docs/pipelines/overview/index.md index 00c7718b9a..8089a49f76 100644 --- a/docs/pipelines/overview/index.md +++ b/docs/pipelines/overview/index.md @@ -1,18 +1,20 @@ # What is Gruntwork Pipelines? -Gruntwork Pipelines is a framework for securely deploying infrastructure as code to your AWS environments using GitOps workflows. Gruntwork Pipelines runs as a binary and series of steps in your CI system, determining what _actions_ need to be taken, in which _environments_, based on the [_infrastructure changes_](#infrastructure-change) that occurred. +**Gruntwork Pipelines is a framework for securely and responsibly rolling out infrastructure changes to your AWS environments using GitOps workflows.** We often refer to Gruntwork Pipelines as simply "Pipelines." + +Gruntwork Pipelines features first-class support for Terragrunt, runs on top of GitHub Actions, and uses a Pull Request-centric workflow. This means that all information about a proposed infrastructure change is added as comments to the applicable Pull Request, and that you apply the infrastructure change by taking action in the Pull Request. + +Gruntwork Pipelines runs as a binary and series of steps in GitHub Actions, determining what _actions_ need to be taken, in which _environments_, based on the [_infrastructure changes_](#infrastructure-change) that occurred. ## Common Terms -Gruntwork Pipelines uses specific terminology to describe code changes and operations that occur as a result of to changes. This section will familiarize you with the terminology used throughout the Gruntwork Pipelines documentation. +Gruntwork Pipelines uses specific terminology to describe code changes and operations that occur as a result of changes. This section will familiarize you with the terminology used throughout the Gruntwork Pipelines documentation. ### Infrastructure change When you edit any infrastructure-as-code that needs to be "applied" to your cloud account (e.g. AWS or GCP), you are making an _infrastructure change_. We sometime call these an "infra-change" for short. -Infra-changes can involve updates to OpenTF, Terraform, or Terragrunt code, or any other type of file that represents a desired state of your infrastructure and that needs to be somehow applied. A classic example is changing some variables on an instance of a Terraform module. By changing these variables you are essentially requesting that some API calls eventually be made to AWS to update your infrastructure in some way (in this case, by running `terraform apply` at some point). - -Gruntwork Pipelines assumes that infra-changes are committed via git, usually by first opening a Pull Request. When you open a Pull Request, you are essentially "proposing" an infra-change. +Infra-changes can involve updates to Terraform/OpenTofu or Terragrunt code, or any other type of file that represents a desired state of your infrastructure and that needs to be somehow applied. A classic example is changing some variables on an instance of a Terraform module. Gruntwork Pipelines assumes that infra-changes are committed via git, usually by first opening a Pull Request. When you open a Pull Request, you are essentially "proposing" an infra-change. ### Infrastructure change set @@ -30,6 +32,6 @@ Gruntwork is responsible for adding support for a growing library of Pipelines A diff --git a/sidebars/foundations.js b/sidebars/foundations.js index dc368a6a8c..79b3f26b5c 100644 --- a/sidebars/foundations.js +++ b/sidebars/foundations.js @@ -19,17 +19,23 @@ const sidebar = [ type: "doc", id: "foundations/overview/index", }, + "foundations/overview/setup-order" ], }, { - label: "AWS Account Foundations", + label: "Landing Zone", type: "category", collapsed: false, items: [ - "foundations/accounts/prerequisites", - "foundations/accounts/enable-control-tower", - "foundations/accounts/add-account", - "foundations/accounts/manage-accounts", + { + label: "Overview", + type: "doc", + id: "foundations/landing-zone/index", + }, + "foundations/landing-zone/prerequisites", + "foundations/landing-zone/enable-control-tower", + "foundations/landing-zone/add-aws-account", + "foundations/landing-zone/manage-accounts", ], }, { @@ -40,9 +46,10 @@ const sidebar = [ { label: "Overview", type: "doc", - id: "foundations/iac/index", + id: "foundations/iac-foundations/index", }, - "foundations/iac/folder_structure", + "foundations/iac-foundations/initial-setup", + "foundations/iac-foundations/folder-structure", { label: "Module Defaults", type: "category", @@ -51,32 +58,33 @@ const sidebar = [ { label: "What are Module Defaults?", type: "doc", - id: "foundations/iac/module_defaults/index", + id: "foundations/iac-foundations/module_defaults/index", }, { label: "Defining Module Defaults", type: "doc", - id: "foundations/iac/module_defaults/defining", + id: "foundations/iac-foundations/module_defaults/defining", }, { label: "Using Module Defaults", type: "doc", - id: "foundations/iac/module_defaults/usage", + id: "foundations/iac-foundations/module_defaults/usage", }, ], }, ], }, { - label: "CI/CD Foundations", + label: "Pipelines", type: "category", collapsed: false, items: [ { label: "Overview", type: "doc", - id: "foundations/ci-cd/index", + id: "foundations/pipelines/index", }, + "foundations/pipelines/aws-authentication" ] }, { @@ -92,7 +100,7 @@ const sidebar = [ ] }, { - label: "Running apps", + label: "Running Apps", type: "category", collapsed: false, items: [