Skip to content

Commit

Permalink
Subnet counts (#54)
Browse files Browse the repository at this point in the history
* Added new variables to allow the number of public or private subnets to be specified.

* Added option to create 0 to many subnets for each subnet type (public or private). Added Example

* set the dynamic subnet length to match the number of provided az's if any provided

* updated availability_zone variable description

* Added region back in, updated the description to show that there is a nested aws provider within the module

* Added region back in, updated the description to show that there is a nested aws provider within the module

* Created availability_zones.tf and moves availability zone data and locals in to it. Moved the public and private subnet locals into public.tf and private.tf.

* Fixed a miscalculated value for labeling the nat instances by az

* removed outdated comment
  • Loading branch information
Jamie-BitFlight authored and osterman committed Jun 20, 2019
1 parent 90ca5cf commit d7b8237
Show file tree
Hide file tree
Showing 12 changed files with 222 additions and 41 deletions.
21 changes: 14 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -162,23 +162,26 @@ Available targets:
|------|-------------|:----:|:-----:|:-----:|
| additional_tag_map | Additional tags for appending to each tag map | map | `<map>` | no |
| attributes | Any extra attributes for naming these resources | list | `<list>` | no |
| availability_zones | List of Availability Zones where subnets will be created | list | - | yes |
| cidr_block | Base CIDR block which will be divided into subnet CIDR blocks (e.g. `10.0.0.0/16`) | string | - | yes |
| availability_zones | List of Availability Zones where subnets will be created. When none provided, all availability zones will be used up to the number provided in the public_subnet_count and/or private_subnet_count, and then will be reused if the number of subnets requested is more than the number of availability zones | list | `<list>` | no |
| cidr_block | Base CIDR block which will be divided into subnet CIDR blocks (e.g. `10.0.0.0/16`), or pass in the vpc_id to use the CIDR from the VPC | string | `` | no |
| context | The context output from an external label module to pass to the label modules within this module | map | `<map>` | no |
| delimiter | Delimiter to be used between `namespace`, `stage`, `name` and `attributes` | string | `-` | no |
| environment | The environment name if not using stage | string | `` | no |
| igw_id | Internet Gateway ID the public route table will point to (e.g. `igw-9c26a123`) | string | - | yes |
| label_order | The naming order of the id output and Name tag | list | `<list>` | no |
| map_public_ip_on_launch | Instances launched into a public subnet should be assigned a public IP address | string | `true` | no |
| max_subnet_count | Sets the maximum amount of subnets to deploy. 0 will deploy a subnet for every provided availablility zone (in `availability_zones` variable) within the region | string | `0` | no |
| max_subnet_count | The maximum number of subnets to deploy. 0 for none, -1 to match the number of az's in the region, or a specific number | string | `-1` | no |
| name | Solution name, e.g. 'app' or 'jenkins' | string | `` | no |
| namespace | Namespace, which could be your organization name or abbreviation, e.g. 'eg' or 'cp' | string | `` | no |
| nat_gateway_enabled | Flag to enable/disable NAT Gateways to allow servers in the private subnets to access the Internet | string | `true` | no |
| nat_instance_enabled | Flag to enable/disable NAT Instances to allow servers in the private subnets to access the Internet | string | `false` | no |
| nat_instance_type | NAT Instance type | string | `t3.micro` | no |
| private_network_acl_id | Network ACL ID that will be added to private subnets. If empty, a new ACL will be created | string | `` | no |
| private_subnet_count | Sets the amount of private subnets to deploy. -1 will deploy a subnet for every availablility zone within the region, 0 will deploy no subnets. The AZ's supplied will be cycled through to create the subnets | string | `-1` | no |
| public_network_acl_id | Network ACL ID that will be added to public subnets. If empty, a new ACL will be created | string | `` | no |
| public_subnet_count | Sets the amount of public subnets to deploy. -1 will deploy a subnet for every availablility zone within the region, 0 will deploy no subnets. The AZ's supplied will be cycled through to create the subnets | string | `-1` | no |
| regex_replace_chars | Regex to replace chars with empty string in `namespace`, `environment`, `stage` and `name`. By default only hyphens, letters and digits are allowed, all other chars are removed | string | `/[^a-zA-Z0-9-]/` | no |
| region | The region to pass to the AWS provider nested in this module. | string | - | yes |
| stage | Stage, e.g. 'prod', 'staging', 'dev', or 'test' | string | `` | no |
| subnet_type_tag_key | Key for subnet type tag to provide information about the type of subnets, e.g. `cpco.io/subnet/type=private` or `cpco.io/subnet/type=public` | string | `cpco.io/subnet/type` | no |
| subnet_type_tag_value_format | This is using the format interpolation symbols to allow the value of the subnet_type_tag_key to be modified. | string | `%s` | no |
Expand All @@ -191,6 +194,8 @@ Available targets:
| Name | Description |
|------|-------------|
| availability_zones | List of Availability Zones where subnets were created |
| availability_zones_private | List of private Availability Zones where subnets were created |
| availability_zones_public | List of public Availability Zones where subnets were created |
| nat_gateway_ids | IDs of the NAT Gateways created |
| nat_instance_ids | IDs of the NAT Instances created |
| private_route_table_ids | IDs of the created private route tables |
Expand Down Expand Up @@ -338,25 +343,27 @@ Check out [our other projects][github], [follow us on twitter][twitter], [apply

### Contributors

| [![Erik Osterman][osterman_avatar]][osterman_homepage]<br/>[Erik Osterman][osterman_homepage] | [![Andriy Knysh][aknysh_avatar]][aknysh_homepage]<br/>[Andriy Knysh][aknysh_homepage] | [![Sergey Vasilyev][s2504s_avatar]][s2504s_homepage]<br/>[Sergey Vasilyev][s2504s_homepage] | [![Vladimir][SweetOps_avatar]][SweetOps_homepage]<br/>[Vladimir][SweetOps_homepage] | [![Konstantin B][comeanother_avatar]][comeanother_homepage]<br/>[Konstantin B][comeanother_homepage] | [![dcowan-vestmark][dcowan-vestmark_avatar]][dcowan-vestmark_homepage]<br/>[dcowan-vestmark][dcowan-vestmark_homepage] | [![Ivan Pinatti][ivan-pinatti_avatar]][ivan-pinatti_homepage]<br/>[Ivan Pinatti][ivan-pinatti_homepage] | [![Oscar Sullivan][osulli_avatar]][osulli_homepage]<br/>[Oscar Sullivan][osulli_homepage] |
|---|---|---|---|---|---|---|---|
| [![Erik Osterman][osterman_avatar]][osterman_homepage]<br/>[Erik Osterman][osterman_homepage] | [![Andriy Knysh][aknysh_avatar]][aknysh_homepage]<br/>[Andriy Knysh][aknysh_homepage] | [![Jamie Nelson][Jamie-BitFlight_avatar]][Jamie-BitFlight_homepage]<br/>[Jamie Nelson][Jamie-BitFlight_homepage] | [![Sergey Vasilyev][s2504s_avatar]][s2504s_homepage]<br/>[Sergey Vasilyev][s2504s_homepage] | [![Vladimir][SweetOps_avatar]][SweetOps_homepage]<br/>[Vladimir][SweetOps_homepage] | [![Konstantin B][comeanother_avatar]][comeanother_homepage]<br/>[Konstantin B][comeanother_homepage] | [![Ivan Pinatti][ivan-pinatti_avatar]][ivan-pinatti_homepage]<br/>[Ivan Pinatti][ivan-pinatti_homepage] | [![Oscar Sullivan][osulli_avatar]][osulli_homepage]<br/>[Oscar Sullivan][osulli_homepage] | [![dcowan-vestmark][dcowan-vestmark_avatar]][dcowan-vestmark_homepage]<br/>[dcowan-vestmark][dcowan-vestmark_homepage] |
|---|---|---|---|---|---|---|---|---|

[osterman_homepage]: https://github.com/osterman
[osterman_avatar]: https://github.com/osterman.png?size=150
[aknysh_homepage]: https://github.com/aknysh
[aknysh_avatar]: https://github.com/aknysh.png?size=150
[Jamie-BitFlight_homepage]: https://github.com/Jamie-BitFlight
[Jamie-BitFlight_avatar]: https://github.com/Jamie-BitFlight.png?size=150
[s2504s_homepage]: https://github.com/s2504s
[s2504s_avatar]: https://github.com/s2504s.png?size=150
[SweetOps_homepage]: https://github.com/SweetOps
[SweetOps_avatar]: https://github.com/SweetOps.png?size=150
[comeanother_homepage]: https://github.com/comeanother
[comeanother_avatar]: https://github.com/comeanother.png?size=150
[dcowan-vestmark_homepage]: https://github.com/dcowan-vestmark
[dcowan-vestmark_avatar]: https://github.com/dcowan-vestmark.png?size=150
[ivan-pinatti_homepage]: https://github.com/ivan-pinatti
[ivan-pinatti_avatar]: https://github.com/ivan-pinatti.png?size=150
[osulli_homepage]: https://github.com/osulli
[osulli_avatar]: https://github.com/osulli.png?size=150
[dcowan-vestmark_homepage]: https://github.com/dcowan-vestmark
[dcowan-vestmark_avatar]: https://github.com/dcowan-vestmark.png?size=150



Expand Down
6 changes: 4 additions & 2 deletions README.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -98,16 +98,18 @@ contributors:
github: "osterman"
- name: "Andriy Knysh"
github: "aknysh"
- name: "Jamie Nelson"
github: "Jamie-BitFlight"
- name: "Sergey Vasilyev"
github: "s2504s"
- name: "Vladimir"
github: "SweetOps"
- name: "Konstantin B"
github: "comeanother"
- name: "dcowan-vestmark"
github: "dcowan-vestmark"
- name: "Ivan Pinatti"
github: "ivan-pinatti"
- name: "Oscar Sullivan"
github: "osulli"
- name: "dcowan-vestmark"
github: "dcowan-vestmark"

34 changes: 34 additions & 0 deletions availability_zones.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
data "aws_availability_zones" "available" {}

locals {
## If the variable availability_zones is empty, use the list provided by data.aws_availability_zones.available.names
## Otherwise use the zones listed in availability_zones
az_name_map = {
"0" = ["${data.aws_availability_zones.available.names}"]
"1" = ["${var.availability_zones}"]
}

availability_zones = "${local.az_name_map[signum(length(var.availability_zones))]}"

## This selects one of the lists based on the signum() interpolation
az_map = {
"-1" = ["${data.aws_availability_zones.available.names}"]
"1" = ["${local.availability_zones}"]
"0" = []
}

## Select the az from the list using a function like `local.availability_zones_public[count.index % length(local.availability_zones_public)]`
availability_zones_public = "${local.az_map[signum(local.public_subnet_count)]}"
availability_zones_private = "${local.az_map[signum(local.private_subnet_count)]}"
}

## This should be depreciated in the future.
locals {
max_subnets_map = {
"-1" = "${length(local.availability_zones)}"
"0" = "0"
"1" = "${var.max_subnet_count}"
}

max_subnet_count = "${local.max_subnets_map[signum(var.max_subnet_count)]}"
}
11 changes: 8 additions & 3 deletions docs/terraform.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,23 +4,26 @@
|------|-------------|:----:|:-----:|:-----:|
| additional_tag_map | Additional tags for appending to each tag map | map | `<map>` | no |
| attributes | Any extra attributes for naming these resources | list | `<list>` | no |
| availability_zones | List of Availability Zones where subnets will be created | list | - | yes |
| cidr_block | Base CIDR block which will be divided into subnet CIDR blocks (e.g. `10.0.0.0/16`) | string | - | yes |
| availability_zones | List of Availability Zones where subnets will be created. When none provided, all availability zones will be used up to the number provided in the public_subnet_count and/or private_subnet_count, and then will be reused if the number of subnets requested is more than the number of availability zones | list | `<list>` | no |
| cidr_block | Base CIDR block which will be divided into subnet CIDR blocks (e.g. `10.0.0.0/16`), or pass in the vpc_id to use the CIDR from the VPC | string | `` | no |
| context | The context output from an external label module to pass to the label modules within this module | map | `<map>` | no |
| delimiter | Delimiter to be used between `namespace`, `stage`, `name` and `attributes` | string | `-` | no |
| environment | The environment name if not using stage | string | `` | no |
| igw_id | Internet Gateway ID the public route table will point to (e.g. `igw-9c26a123`) | string | - | yes |
| label_order | The naming order of the id output and Name tag | list | `<list>` | no |
| map_public_ip_on_launch | Instances launched into a public subnet should be assigned a public IP address | string | `true` | no |
| max_subnet_count | Sets the maximum amount of subnets to deploy. 0 will deploy a subnet for every provided availablility zone (in `availability_zones` variable) within the region | string | `0` | no |
| max_subnet_count | The maximum number of subnets to deploy. 0 for none, -1 to match the number of az's in the region, or a specific number | string | `-1` | no |
| name | Solution name, e.g. 'app' or 'jenkins' | string | `` | no |
| namespace | Namespace, which could be your organization name or abbreviation, e.g. 'eg' or 'cp' | string | `` | no |
| nat_gateway_enabled | Flag to enable/disable NAT Gateways to allow servers in the private subnets to access the Internet | string | `true` | no |
| nat_instance_enabled | Flag to enable/disable NAT Instances to allow servers in the private subnets to access the Internet | string | `false` | no |
| nat_instance_type | NAT Instance type | string | `t3.micro` | no |
| private_network_acl_id | Network ACL ID that will be added to private subnets. If empty, a new ACL will be created | string | `` | no |
| private_subnet_count | Sets the amount of private subnets to deploy. -1 will deploy a subnet for every availablility zone within the region, 0 will deploy no subnets. The AZ's supplied will be cycled through to create the subnets | string | `-1` | no |
| public_network_acl_id | Network ACL ID that will be added to public subnets. If empty, a new ACL will be created | string | `` | no |
| public_subnet_count | Sets the amount of public subnets to deploy. -1 will deploy a subnet for every availablility zone within the region, 0 will deploy no subnets. The AZ's supplied will be cycled through to create the subnets | string | `-1` | no |
| regex_replace_chars | Regex to replace chars with empty string in `namespace`, `environment`, `stage` and `name`. By default only hyphens, letters and digits are allowed, all other chars are removed | string | `/[^a-zA-Z0-9-]/` | no |
| region | The region to pass to the AWS provider nested in this module. | string | - | yes |
| stage | Stage, e.g. 'prod', 'staging', 'dev', or 'test' | string | `` | no |
| subnet_type_tag_key | Key for subnet type tag to provide information about the type of subnets, e.g. `cpco.io/subnet/type=private` or `cpco.io/subnet/type=public` | string | `cpco.io/subnet/type` | no |
| subnet_type_tag_value_format | This is using the format interpolation symbols to allow the value of the subnet_type_tag_key to be modified. | string | `%s` | no |
Expand All @@ -33,6 +36,8 @@
| Name | Description |
|------|-------------|
| availability_zones | List of Availability Zones where subnets were created |
| availability_zones_private | List of private Availability Zones where subnets were created |
| availability_zones_public | List of public Availability Zones where subnets were created |
| nat_gateway_ids | IDs of the NAT Gateways created |
| nat_instance_ids | IDs of the NAT Instances created |
| private_route_table_ids | IDs of the created private route tables |
Expand Down
86 changes: 86 additions & 0 deletions example/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
## This example demonstrates it use as if it was being used
## for some Spinnaker (spinnaker.io) deployment

module "dynamic_subnets" {
source = "./.."
context = "${module.label.context}"
tags = "${merge(module.label.tags, local.subnet_tags)}"
region = "${data.aws_region.current.name}"
availability_zones = ["${data.aws_region.current.name}a", "${data.aws_region.current.name}b"] // Optional list of AZ's to restrict it to
vpc_id = "${module.vpc.vpc_id}"
igw_id = "${module.vpc.igw_id}"
public_subnet_count = "2" // Two public zones for the load balancers
private_subnet_count = "4" // Four private zones for the
map_public_ip_on_launch = "true"

## You can use nat_gateway_enabled or nat_instance_enabled
## It creates one nat instance per public subnet.
## So if you want to exclude the public subnet by setting the public_subnet_count to 0
## You will neet to use the nat_gateway_enabled option.
nat_instance_enabled = "true"

nat_gateway_enabled = "false"

## Optionally customising a tag based on whether it is public or private
## will format like this: "immutable_metadata": {"purpose": \"public-subnet\"}"
subnet_type_tag_key = "immutable_metadata"

subnet_type_tag_value_format = "{\"purpose\": \"%s-subnet\"}" // The %s gets replaced with 'public' on public subnets and 'private' on private subnets
}

## VPC module doesn't have the latest version of null_label
## module integrated with it at the time of this example being
## written so no context variable here.
module "vpc" {
source = "git::https://github.com/cloudposse/terraform-aws-vpc.git?ref=tags/0.4.1"
namespace = "${module.label.namespace}"
stage = "${module.label.environment}"
name = "${module.label.name}"
attributes = ["${module.label.attributes}"]
delimiter = "${module.label.delimiter}"
tags = "${module.label.tags}"
cidr_block = "${var.vpc_cidr}"
}

data "aws_region" "current" {}

module "label" {
source = "git::https://github.com/cloudposse/terraform-null-label.git?ref=tags/0.11.1"
namespace = "cp"
environment = "prod"
delimiter = "-"
name = "spinnaker"

tags = {
"ManagedBy" = "Terraform"
"ModuleBy" = "CloudPosse"
}
}

variable "eks_cluster_name" {
description = "The name of the EKS cluster that will be utilising this network(if any)"
default = "my-main-eks-cluster"
}

variable "vpc_cidr" {
default = "10.0.0.0/16"
}

locals {
# Spinnaker subnet tags
subnet_tags = {
"kubernetes.io/cluster/${var.eks_cluster_name}" = "shared"
"kubernetes.io/role/elb" = ""
"kubernetes.io/role/internal-elb" = ""
}
}

provider "aws" {
version = "~> 2.12"
region = "us-west-2"
skip_requesting_account_id = true # this can be tricky
skip_get_ec2_platforms = true
skip_metadata_api_check = true
skip_region_validation = true
skip_credentials_validation = true
}
3 changes: 1 addition & 2 deletions main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
# Any non-beta version >= 2.12.0 and < 2.13.0, e.g. 2.12.X
provider "aws" {
version = "~> 2.12.0"
region = "${var.region}"
}

# Terraform
Expand All @@ -15,5 +16,3 @@ terraform {
data "aws_vpc" "default" {
id = "${var.vpc_id}"
}

data "aws_availability_zones" "available" {}
8 changes: 4 additions & 4 deletions nat-gateway.tf
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
module "nat_label" {
source = "git::https://github.com/cloudposse/terraform-null-label.git?ref=tags/0.11.1"
context = "${module.label.context}"
attributes = "${distinct(compact(concat(module.label.attributes,list("nat"))))}"
attributes = "${distinct(compact(concat(module.label.attributes,list("nat", "gateway"))))}"
}

locals {
nat_gateways_count = "${var.nat_gateway_enabled == "true" ? length(var.availability_zones) : 0}"
nat_gateways_count = "${var.nat_gateway_enabled == "true" ? local.private_subnet_count : 0}"
}

resource "aws_eip" "default" {
count = "${local.nat_gateways_count}"
vpc = true
tags = "${merge(module.private_label.tags, map("Name",format("%s%s%s", module.private_label.id, var.delimiter, replace(element(var.availability_zones, count.index),"-",var.delimiter))))}"
tags = "${merge(module.private_label.tags, map("Name",format("%s%s%s", module.private_label.id, var.delimiter, replace(local.availability_zones_private[count.index % length(local.availability_zones_private)],"-",var.delimiter))))}"

lifecycle {
create_before_destroy = true
Expand All @@ -22,7 +22,7 @@ resource "aws_nat_gateway" "default" {
count = "${local.nat_gateways_count}"
allocation_id = "${element(aws_eip.default.*.id, count.index)}"
subnet_id = "${element(aws_subnet.public.*.id, count.index)}"
tags = "${merge(module.nat_label.tags, map("Name",format("%s%s%s", module.nat_label.id, var.delimiter, replace(element(var.availability_zones, count.index),"-",var.delimiter))))}"
tags = "${merge(module.nat_label.tags, map("Name",format("%s%s%s", module.nat_label.id, var.delimiter, replace(local.availability_zones_private[count.index % length(local.availability_zones_private)],"-",var.delimiter))))}"

lifecycle {
create_before_destroy = true
Expand Down
Loading

0 comments on commit d7b8237

Please sign in to comment.