diff --git a/_docs-sources/reference/services/intro/deploy-new-infrastructure.md b/_docs-sources/reference/services/intro/deploy-new-infrastructure.md index 3447beac35..e879c37038 100644 --- a/_docs-sources/reference/services/intro/deploy-new-infrastructure.md +++ b/_docs-sources/reference/services/intro/deploy-new-infrastructure.md @@ -277,7 +277,7 @@ Some of the services in the Gruntwork Service Catalog require you to build an [A (AMI)](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html) to install and configure the software that will run on EC2 instances. These services define and manage the AMI as code using [Packer](https://www.packer.io/) templates. -For example, the [`eks-cluster`](https://github.com/gruntwork-io/terraform-aws-service-catalog/tree/master/modules/services/eks-cluster) service defines an +For example, the [`eks-workers`](https://github.com/gruntwork-io/terraform-aws-service-catalog/tree/master/modules/services/eks-workers) service defines an [`eks-node-al2.pkr.hcl`](https://github.com/gruntwork-io/terraform-aws-service-catalog/blob/master/modules/services/eks-workers/eks-node-al2.pkr.hcl) Packer template that can be used to create an AMI for the Kubernetes worker nodes. This Packer template uses the [EKS optimized AMI](https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-ami.html) as its base, which already has Docker, @@ -285,6 +285,12 @@ kubelet, and the AWS IAM Authenticator installed, and on top of that, it install want on an EC2 instance in production, such as tools for gathering metrics, log aggregation, intrusion prevention, and so on. +The packer templates are provided as `hcl` files in each service module folder, and follow the naming convention of: + +``` +-.pkr.hcl +``` + Below are instructions on how to build an AMI using these Packer templates. We'll be using the [`eks-node-al2.pkr.hcl`](https://github.com/gruntwork-io/terraform-aws-service-catalog/blob/master/modules/services/eks-workers/eks-node-al2.pkr.hcl) Packer template as an example. @@ -346,10 +352,44 @@ Below are instructions on how to build an AMI using these Packer templates. We'l 1. **Build**. Now you can build an AMI as follows: ```bash - packer build -var-file=eks-vars.json eks-node-al2.json + packer build -var-file=eks-vars.json eks-node-al2.pkr.hcl ``` -1. **Deploy**. At the end of the build, Packer will output the ID of your new AMI. Typically, you deploy this AMI by - plugging the AMI ID into some Terraform code and using Terraform to deploy it: e.g., plugging in the EKS worker node - AMI ID into the `cluster_instance_ami` input variable of the [`eks-cluster` Terraform - module](https://github.com/gruntwork-io/terraform-aws-service-catalog/tree/master/modules/services/eks-cluster). +#### How to deploy newly built AMIs? + +Once you build the AMI, the next step is to deploy it to your infrastructure. Each service that requires an AMI offers +two configuration inputs for selecting the AMI, and you must pick one: + +- `*_ami` (e.g., the [`cluster_instance_ami` input + variable](https://github.com/gruntwork-io/terraform-aws-service-catalog/blob/v0.72.0/modules/services/eks-workers/variables.tf#L185-L188) + in the `eks-workers` module) +- `*_ami_filters` (e.g., the [`cluster_instance_ami_filters` input + variable](https://github.com/gruntwork-io/terraform-aws-service-catalog/blob/v0.72.0/modules/services/eks-workers/variables.tf#L190-L204) + in the `eks-workers` module) + +The two approaches are mutually exclusive. When specifying both, `*_ami` is always used and the input to +`*_ami_filters` is ignored. + +The `*_ami` input variable can be used to directly specify the AMI to use. When using this input, the value should be +set to the AMI ID that is returned by the packer call. It should be in the format `ami-`. + +The `*_ami_filters` input variable takes in an AMI filter expression that can be used for dynamically looking up a +prebuilt AMI. The supported filters for the lookup can be obtained from the [describe-images command +description](https://docs.aws.amazon.com/cli/latest/reference/ec2/describe-images.html) in the AWS CLI reference. The +most commonly used filters will be: + +- `name`: Filter by the name of the AMI. Note that this supports unix glob expressions (e.g., `*-eks-node` will match + any image with the suffix `-eks-node` in the name). +- `tag:`: Filter by the given tag key. Note that `` can be any tag key that is applied to the AMI. For + example, to search for AMIs with the tag `service-catalog-module = eks-workers`, you can use the following filter: + + cluster_instance_ami_filters = { + owners = ["self"] + filters = [{ + name = "tag:service-catalog-module" + values = ["eks-workers"] + }] + } + + Note that tags are only visible in the account that owns the AMI. Even if you share the AMI in the packer template, + the AMI tags will not be visible when you query it from a shared account. diff --git a/_docs-sources/reference/services/intro/make-changes-to-your-infrastructure.md b/_docs-sources/reference/services/intro/make-changes-to-your-infrastructure.md index f7a64ce151..c63b5569d9 100644 --- a/_docs-sources/reference/services/intro/make-changes-to-your-infrastructure.md +++ b/_docs-sources/reference/services/intro/make-changes-to-your-infrastructure.md @@ -183,7 +183,7 @@ _(Documentation coming soon. If you need help with this ASAP, please contact [su 1. **Build**. Once you've made your changes, you can re-run `packer build` to build a new AMI: ```bash - packer build -var-file=eks-vars.json eks-node-al2.json + packer build -var-file=eks-vars.json eks-node-al2.pkr.hcl ``` 1. **Deploy**. Once you've built a new AMI, you'll need to [make changes to your Terraform code and deploy diff --git a/_docs-sources/reference/services/intro/patterns-used-in-the-service-catalog.md b/_docs-sources/reference/services/intro/patterns-used-in-the-service-catalog.md deleted file mode 100644 index b26594aaee..0000000000 --- a/_docs-sources/reference/services/intro/patterns-used-in-the-service-catalog.md +++ /dev/null @@ -1,95 +0,0 @@ -# Patterns used in the Service Catalog - -## How to build AMIs for the Service Catalog - -Some services depend on EC2 instances to host the underlying service. For example, when deploying an ECS cluster, you -need to provision worker nodes that will run the Docker Containers of the ECS services that are deployed on the cluster. -For any service that depends on EC2 instances, the Service Catalog will provide [packer](https://www.packer.io/) -templates that can be used to build the AMIs. These packer templates define the AMIs as code, so that they can be -built from scratch in a consistent manner. - -The packer templates are provided as `json` files in each service module folder, and follow the naming convention of: - -``` --.json -``` - -For example, the packer template for ECS workers is the file -[`ecs-node-al2.json`](https://github.com/gruntwork-io/terraform-aws-service-catalog/blob/master/modules/services/ecs-cluster/ecs-node-al2.json), -providing an Amazon Linux 2 based AMI. - -You can directly build any packer template by calling `packer` on it. The following walks through the steps for building -the ECS node AMI against Service Catalog version `v0.30.0`: - -1. If you haven't already, download and install packer from https://www.packer.io/. - -1. Clone the Service Catalog repository to a temporary directory (if you don't already have a local copy). Make sure to - checkout the version tag of the Service Catalog repository for which you wish to base your AMI on: - - git clone git@github.com:gruntwork-io/terraform-aws-service-catalog.git - (cd terraform-aws-service-catalog && git checkout v0.30.0) - -1. Change your working directory to the service module folder where the packer template is located: - - cd terraform-aws-service-catalog/modules/services/ecs-cluster - -1. Inspect the packer template and identify the variables you wish to configure. Define all the variables you wish to - configure in a [packer var file](https://www.packer.io/docs/templates/legacy_json_templates/user-variables#from-a-file). - -1. Set the environment variable `GITHUB_OAUTH_TOKEN` in your shell to a [Github Personal Access - Token](https://docs.github.com/en/github/authenticating-to-github/creating-a-personal-access-token) for a user that - has access to the Gruntwork Infrastructure as Code library. - -1. Authenticate to the AWS account where you wish to host the AMI. Refer to the post [A Comprehensive Guide to - Authenticating to AWS on the Command - Line](https://blog.gruntwork.io/a-comprehensive-guide-to-authenticating-to-aws-on-the-command-line-63656a686799) for - information on how to authenticate to AWS in your shell. - -1. Once you are authenticated, invoke `packer` with your var file to build the AMI: - - packer build -var-file /path/to/var-file.json ./ecs-node-al2.json - -## How to deploy newly built AMIs? - -Once you build the AMI, the next step is to deploy it to your infrastructure. Each service that requires an AMI offers -two configuration inputs for selecting the AMI, and you must pick one: - -- `*_ami` (e.g., the [`cluster_instance_ami` input - variable](https://github.com/gruntwork-io/terraform-aws-service-catalog/blob/v0.29.0/modules/services/ecs-cluster/variables.tf#L26-29) - in the `ecs-cluster` module) -- `*_ami_filters` (e.g., the [`cluster_instance_ami_filters` input - variable](https://github.com/gruntwork-io/terraform-aws-service-catalog/blob/v0.29.0/modules/services/ecs-cluster/variables.tf#L31-44) - in the `ecs-cluster` module) - -The two approaches are mutually exclusive. When specifying both, `*_ami` is always used and the input to -`*_ami_filters` is ignored. - -The `*_ami` input variable can be used to directly specify the AMI to use. When using this input, the value should be -set to the AMI ID that is returned by the packer call. It should be in the format `ami-`. - -The `*_ami_filters` input variable takes in an AMI filter expression that can be used for dynamically looking up a -prebuilt AMI. The supported filters for the lookup can be obtained from the [describe-images command -description](https://docs.aws.amazon.com/cli/latest/reference/ec2/describe-images.html) in the AWS CLI reference. The -most commonly used filters will be: - -- `name`: Filter by the name of the AMI. Note that this supports unix glob expressions (e.g., `*-ecs-node` will match - any image with the suffix `-ecs-node` in the name). -- `tag:`: Filter by the given tag key. Note that `` can be any tag key that is applied to the AMI. For - example, to search for AMIs with the tag `service-catalog-module = ecs-cluster`, you can use the following filter: - - ```hcl - cluster_instance_ami_filters = { - owners = ["self"] - filters = [{ - name = "tag:service-catalog-module" - values = ["ecs-cluster"] - }] - } - ``` - - :::note - - Tags are only visible in the account that owns the AMI. Even if you share the AMI in the packer template, - the AMI tags will not be visible when you query it from a shared account. - - ::: diff --git a/docs/reference/services/intro/deploy-new-infrastructure.md b/docs/reference/services/intro/deploy-new-infrastructure.md index b9eeee0852..00e32b8e45 100644 --- a/docs/reference/services/intro/deploy-new-infrastructure.md +++ b/docs/reference/services/intro/deploy-new-infrastructure.md @@ -277,7 +277,7 @@ Some of the services in the Gruntwork Service Catalog require you to build an [A (AMI)](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html) to install and configure the software that will run on EC2 instances. These services define and manage the AMI as code using [Packer](https://www.packer.io/) templates. -For example, the [`eks-cluster`](https://github.com/gruntwork-io/terraform-aws-service-catalog/tree/master/modules/services/eks-cluster) service defines an +For example, the [`eks-workers`](https://github.com/gruntwork-io/terraform-aws-service-catalog/tree/master/modules/services/eks-workers) service defines an [`eks-node-al2.pkr.hcl`](https://github.com/gruntwork-io/terraform-aws-service-catalog/blob/master/modules/services/eks-workers/eks-node-al2.pkr.hcl) Packer template that can be used to create an AMI for the Kubernetes worker nodes. This Packer template uses the [EKS optimized AMI](https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-ami.html) as its base, which already has Docker, @@ -285,6 +285,12 @@ kubelet, and the AWS IAM Authenticator installed, and on top of that, it install want on an EC2 instance in production, such as tools for gathering metrics, log aggregation, intrusion prevention, and so on. +The packer templates are provided as `hcl` files in each service module folder, and follow the naming convention of: + +``` +-.pkr.hcl +``` + Below are instructions on how to build an AMI using these Packer templates. We'll be using the [`eks-node-al2.pkr.hcl`](https://github.com/gruntwork-io/terraform-aws-service-catalog/blob/master/modules/services/eks-workers/eks-node-al2.pkr.hcl) Packer template as an example. @@ -346,15 +352,49 @@ Below are instructions on how to build an AMI using these Packer templates. We'l 1. **Build**. Now you can build an AMI as follows: ```bash - packer build -var-file=eks-vars.json eks-node-al2.json + packer build -var-file=eks-vars.json eks-node-al2.pkr.hcl ``` -1. **Deploy**. At the end of the build, Packer will output the ID of your new AMI. Typically, you deploy this AMI by - plugging the AMI ID into some Terraform code and using Terraform to deploy it: e.g., plugging in the EKS worker node - AMI ID into the `cluster_instance_ami` input variable of the [`eks-cluster` Terraform - module](https://github.com/gruntwork-io/terraform-aws-service-catalog/tree/master/modules/services/eks-cluster). +#### How to deploy newly built AMIs? + +Once you build the AMI, the next step is to deploy it to your infrastructure. Each service that requires an AMI offers +two configuration inputs for selecting the AMI, and you must pick one: + +- `*_ami` (e.g., the [`cluster_instance_ami` input + variable](https://github.com/gruntwork-io/terraform-aws-service-catalog/blob/v0.72.0/modules/services/eks-workers/variables.tf#L185-L188) + in the `eks-workers` module) +- `*_ami_filters` (e.g., the [`cluster_instance_ami_filters` input + variable](https://github.com/gruntwork-io/terraform-aws-service-catalog/blob/v0.72.0/modules/services/eks-workers/variables.tf#L190-L204) + in the `eks-workers` module) + +The two approaches are mutually exclusive. When specifying both, `*_ami` is always used and the input to +`*_ami_filters` is ignored. + +The `*_ami` input variable can be used to directly specify the AMI to use. When using this input, the value should be +set to the AMI ID that is returned by the packer call. It should be in the format `ami-`. + +The `*_ami_filters` input variable takes in an AMI filter expression that can be used for dynamically looking up a +prebuilt AMI. The supported filters for the lookup can be obtained from the [describe-images command +description](https://docs.aws.amazon.com/cli/latest/reference/ec2/describe-images.html) in the AWS CLI reference. The +most commonly used filters will be: + +- `name`: Filter by the name of the AMI. Note that this supports unix glob expressions (e.g., `*-eks-node` will match + any image with the suffix `-eks-node` in the name). +- `tag:`: Filter by the given tag key. Note that `` can be any tag key that is applied to the AMI. For + example, to search for AMIs with the tag `service-catalog-module = eks-workers`, you can use the following filter: + + cluster_instance_ami_filters = { + owners = ["self"] + filters = [{ + name = "tag:service-catalog-module" + values = ["eks-workers"] + }] + } + + Note that tags are only visible in the account that owns the AMI. Even if you share the AMI in the packer template, + the AMI tags will not be visible when you query it from a shared account. diff --git a/docs/reference/services/intro/make-changes-to-your-infrastructure.md b/docs/reference/services/intro/make-changes-to-your-infrastructure.md index 1d2cbd7e13..05dc642a90 100644 --- a/docs/reference/services/intro/make-changes-to-your-infrastructure.md +++ b/docs/reference/services/intro/make-changes-to-your-infrastructure.md @@ -183,7 +183,7 @@ _(Documentation coming soon. If you need help with this ASAP, please contact [su 1. **Build**. Once you've made your changes, you can re-run `packer build` to build a new AMI: ```bash - packer build -var-file=eks-vars.json eks-node-al2.json + packer build -var-file=eks-vars.json eks-node-al2.pkr.hcl ``` 1. **Deploy**. Once you've built a new AMI, you'll need to [make changes to your Terraform code and deploy @@ -191,5 +191,5 @@ _(Documentation coming soon. If you need help with this ASAP, please contact [su diff --git a/docs/reference/services/intro/patterns-used-in-the-service-catalog.md b/docs/reference/services/intro/patterns-used-in-the-service-catalog.md deleted file mode 100644 index 3f9abbebbd..0000000000 --- a/docs/reference/services/intro/patterns-used-in-the-service-catalog.md +++ /dev/null @@ -1,100 +0,0 @@ -# Patterns used in the Service Catalog - -## How to build AMIs for the Service Catalog - -Some services depend on EC2 instances to host the underlying service. For example, when deploying an ECS cluster, you -need to provision worker nodes that will run the Docker Containers of the ECS services that are deployed on the cluster. -For any service that depends on EC2 instances, the Service Catalog will provide [packer](https://www.packer.io/) -templates that can be used to build the AMIs. These packer templates define the AMIs as code, so that they can be -built from scratch in a consistent manner. - -The packer templates are provided as `json` files in each service module folder, and follow the naming convention of: - -``` --.json -``` - -For example, the packer template for ECS workers is the file -[`ecs-node-al2.json`](https://github.com/gruntwork-io/terraform-aws-service-catalog/blob/master/modules/services/ecs-cluster/ecs-node-al2.json), -providing an Amazon Linux 2 based AMI. - -You can directly build any packer template by calling `packer` on it. The following walks through the steps for building -the ECS node AMI against Service Catalog version `v0.30.0`: - -1. If you haven't already, download and install packer from https://www.packer.io/. - -1. Clone the Service Catalog repository to a temporary directory (if you don't already have a local copy). Make sure to - checkout the version tag of the Service Catalog repository for which you wish to base your AMI on: - - git clone git@github.com:gruntwork-io/terraform-aws-service-catalog.git - (cd terraform-aws-service-catalog && git checkout v0.30.0) - -1. Change your working directory to the service module folder where the packer template is located: - - cd terraform-aws-service-catalog/modules/services/ecs-cluster - -1. Inspect the packer template and identify the variables you wish to configure. Define all the variables you wish to - configure in a [packer var file](https://www.packer.io/docs/templates/legacy_json_templates/user-variables#from-a-file). - -1. Set the environment variable `GITHUB_OAUTH_TOKEN` in your shell to a [Github Personal Access - Token](https://docs.github.com/en/github/authenticating-to-github/creating-a-personal-access-token) for a user that - has access to the Gruntwork Infrastructure as Code library. - -1. Authenticate to the AWS account where you wish to host the AMI. Refer to the post [A Comprehensive Guide to - Authenticating to AWS on the Command - Line](https://blog.gruntwork.io/a-comprehensive-guide-to-authenticating-to-aws-on-the-command-line-63656a686799) for - information on how to authenticate to AWS in your shell. - -1. Once you are authenticated, invoke `packer` with your var file to build the AMI: - - packer build -var-file /path/to/var-file.json ./ecs-node-al2.json - -## How to deploy newly built AMIs? - -Once you build the AMI, the next step is to deploy it to your infrastructure. Each service that requires an AMI offers -two configuration inputs for selecting the AMI, and you must pick one: - -- `*_ami` (e.g., the [`cluster_instance_ami` input - variable](https://github.com/gruntwork-io/terraform-aws-service-catalog/blob/v0.29.0/modules/services/ecs-cluster/variables.tf#L26-29) - in the `ecs-cluster` module) -- `*_ami_filters` (e.g., the [`cluster_instance_ami_filters` input - variable](https://github.com/gruntwork-io/terraform-aws-service-catalog/blob/v0.29.0/modules/services/ecs-cluster/variables.tf#L31-44) - in the `ecs-cluster` module) - -The two approaches are mutually exclusive. When specifying both, `*_ami` is always used and the input to -`*_ami_filters` is ignored. - -The `*_ami` input variable can be used to directly specify the AMI to use. When using this input, the value should be -set to the AMI ID that is returned by the packer call. It should be in the format `ami-`. - -The `*_ami_filters` input variable takes in an AMI filter expression that can be used for dynamically looking up a -prebuilt AMI. The supported filters for the lookup can be obtained from the [describe-images command -description](https://docs.aws.amazon.com/cli/latest/reference/ec2/describe-images.html) in the AWS CLI reference. The -most commonly used filters will be: - -- `name`: Filter by the name of the AMI. Note that this supports unix glob expressions (e.g., `*-ecs-node` will match - any image with the suffix `-ecs-node` in the name). -- `tag:`: Filter by the given tag key. Note that `` can be any tag key that is applied to the AMI. For - example, to search for AMIs with the tag `service-catalog-module = ecs-cluster`, you can use the following filter: - - ```hcl - cluster_instance_ami_filters = { - owners = ["self"] - filters = [{ - name = "tag:service-catalog-module" - values = ["ecs-cluster"] - }] - } - ``` - - :::note - - Tags are only visible in the account that owns the AMI. Even if you share the AMI in the packer template, - the AMI tags will not be visible when you query it from a shared account. - - ::: - - - diff --git a/sidebars/api-reference.js b/sidebars/api-reference.js index ab64eed9a9..ff4fca3e1a 100644 --- a/sidebars/api-reference.js +++ b/sidebars/api-reference.js @@ -28,7 +28,6 @@ const apiSidebars = { "reference/services/intro/deploy-new-infrastructure", "reference/services/intro/make-changes-to-your-infrastructure", "reference/services/intro/create-your-own-service-catalog", - "reference/services/intro/patterns-used-in-the-service-catalog", ], }, {