Terraform AWS Connectivity Module

Description

The purpose of this module is to provision the necessary resources to establish connectivity between a transit gateway and other VPCs, accounts, and on-premises networks, as well as provision a baseline for network topology and security. Currently this module can setup the requirement for

• Inspection VPC: provision the necessary resources and routing to inspect traffic between the transit gateway and spoke VPCs.
• Trusted Layout: here the routing is broken into two routing domains, trusted and untrusted. All traffic within the environment have the ability to route to the trusted domain attachments and back, but traffic between those networks located in the untrusted domain is forbidden.
• Egress VPC: using either one of the above, this module can setup the requirements for an egress VPC to route traffic to the internet.
• Ingress VPC: using either one of the above, this module can setup the requirements for an ingress VPC to route traffic from the internet, to the tenant VPCs.

Usage

module "example" {
  source  = "appvia/<NAME>/aws"
  version = "0.0.1"

  name                       = var.name
  description                = "The transit gateway fot all accounts within this region"
  amazon_side_asn            = var.asn
  enable_dns_support         = true
  enable_external_principals = true
  enable_multicast_support   = true
  enable_vpn_ecmp_support    = true
  tags                       = var.tags

  connectivity_config = {
    inspection = {
      inspection_tgw_attachment_id = "tgw-attach-111111"
    }
  }
}

Support Layouts

Currently the module supports the following layouts:

Inspection Layout

The inspection layout is intended to be used in collaboration with an Inspection VPC, filtering all traffic between the spokes, and depending if enabled, all traffic outbound to the internet or inbound via an ingress VPC.

module "connectivity" {
  source  = "appvia/connectivity/aws"
  version = "0.1.7""

  # insert variables here
  connectivity_config = {
    inspection = {
      # The transit gateway attachment (naturally a chicken and egg problem here, so the attachment is optional)
      attachment_id            = module.firewall.attachment_id
      # OR you can provision the vpc for the inspection vpc module to consume
      network = {
        availability_zones     = 2
        vpc_cidr               = "100.64.0.0/23"
        name                   = "inspection"
        private_subnet_netmask = 24
        public_subnet_netmask  = 24
      }
    }
  }
}

By adding the optional of egress, another VPC can be provisioned containing outbound nat gateways to route traffic to the internet.

module "connectivity" {
  source  = "appvia/connectivity/aws"
  version = "0.0.2"

  services = {
    egress = {
      network = {
        availability_zones = 2
        ipam_pool_id       = module.ipam_pool.id
        name               = "egress"
        vpc_netmask        = 24
      }
    }
  }

  # insert variables here
  connectivity_config = {
    inspection = {
      network = {
        availability_zones = 2
        vpc_cidr           = "100.64.0.0/21"
        name               = "inspection"
        private_netmask    = 24
        public_netmask     = 24
      }
    }
  }
}

Notes:

• The transit gateway must be configured so that the default association table is the inbound routing table; alls spokes are immediately connected to the inspection VPC.
• The transit gateway must be configured so that the default propagation table is the spokes routing table; all traffic returning from the inspection VPC is routed to the correct spoke.
• If egress is enabled, a default route is added to the return routing table to route traffic to the egress VPC.
• If ingress is enabled, the spokes acts like any other spoke with all traffic being routed to the inspection VPC.

Trusted Layout

The trusted layout uses transit gateway routing tables to create two distinct routing domains:

• Trusted: can see all networks and can route traffic to and from them.
• Untrusted: can only see the trusted networks, and cannot route traffic to or from any other untrusted network.

The most common usage pattern here is to place resources such as CI, monitoring, logging, remote access within the trusted domain, with all other accounts falling into untrusted. Any other connectivity requirements between the accounts must use alternative methods to communicate; for example AWS Private Links

Notes:

• The transit gateway must be configured so that the default association table is workloads routing table, hence all new attachments are placed in the untrusted routing table.
• The transit gateway must be configured so that the default propagation table is the trusted routing table, this ensures that all traffic has a route from trusted to untrusted.
• Adding a new trusted requires manual intervention, i.e the network is automatically added to the untrusted routing table, manually deleted, and then the attachment id added to the trusted attachments variable.
• Any trusted attachments are automatically added to the untrusted routing table, to ensure bi-directional routing.

Egress VPC

By adding a var.connectivity_config.egress object, the module will provision the necessary resources to route traffic to the internet via a shared egress VPC. Routing within the choose network layout (inspection, or trusted) is automatically provisioned accordingly.

module "connectivity" {
  source  = "appvia/connectivity/aws"
  version = "0.0.2"

  connectivity_config = {
    egress = {
      network = {
        availability_zones = 2
        ipam_pool_id       = var.ipam_pool_id
        name               = "egress"
        private_netmask    = 28
        vpc_netmask        = 24
      }
    }
  }
}

Ingress VPC

By adding a var.connectivity_config.ingress object, the module will provision the necessary resources to route traffic from the internet to the tenant VPCs. Routing within the choose network layout (inspection, or trusted) is automatically provisioned accordingly. Note, this module does not provisioned the load balancers and or WAF devices depicted in the diagram; purely the VPC and connectivity.

module "connectivity" {
  source  = "appvia/connectivity/aws"
  version = "0.0.2"

  connectivity_config = {
    ingress = {
      network = {
        availability_zones = 2
        ipam_pool_id       = var.ipam_pool_id
        name               = "ingress"
        private_netmask    = 24
        public_netmask     = 22
        vpc_netmask        = 21
      }
    }
  }
}

Private Endpoints

Ensuring all traffic is private and does not traverse the internet is a common requirement. By adding the var.connectivity_config.endpoints object, the module will provision the necessary resources to route traffic to the internet via a shared endpoints VPC. Routing within the choose network layout (inspection, or trusted) is automatically provisioned accordingly.

Take a look at the endpoints module to see how it works, and the prerequisites required on the consumer side i.e associating the resolvers rule sets with the spoke vpc.

module "connectivity" {
  source  = "appvia/connectivity/aws"
  version = "0.0.2"

  connectivity_config = {
    endpoints = {
      # A collection of private endpoints to provision
      services = {
        ec2 = {
          service = "ec2"
        },
        ec2messages = {
          service = "ec2messages"
        },
        ssm = {
          service = "ssm"
        },
        ssmmessages = {
          service = "ssmmessages"
        },
        logs = {
          service = "logs"
        },
        kms = {
          service = "kms"
        },
        secretsmanager = {
          service = "secretsmanager"
        },
        s3 = {
          service = "s3"
        },
      }
      # Configuration for sharing the resolver rule sets with the spoke vpcs
      sharing = {
        ram_principals = var.ram_principals
      }
      # Configuration for the endpoints vpc
      network = {
        availability_zones = 2
        ipam_pool_id       = var.ipam_pool_id
        name               = "endpoints"
        private_netmask    = 24
        public_netmask     = 22
        vpc_netmask        = 21
      }
    }

  }
}

Update Documentation

The terraform-docs utility is used to generate this README. Follow the below steps to update:

1. Make changes to the .terraform-docs.yml file
2. Fetch the terraform-docs binary (https://terraform-docs.io/user-guide/installation/)
3. Run terraform-docs markdown table --output-file ${PWD}/README.md --output-mode inject .

Requirements

Name	Version
terraform	>= 1.0.0
aws	>= 5.0.0

Providers

Name	Version
aws	>= 5.0.0

Modules

Name	Source	Version
dns	appvia/dns/aws	1.2.0
dns_vpc	appvia/network/aws	0.3.0
egress_vpc	appvia/network/aws	0.3.0
endpoints	appvia/private-endpoints/aws	0.2.4
endpoints_vpc	appvia/network/aws	0.3.0
ingress_vpc	appvia/network/aws	0.3.0
inspection_vpc	appvia/network/aws	0.3.0
share_prefixes	./modules/prefix_share	n/a
tgw	terraform-aws-modules/transit-gateway/aws	2.12.2

Resources

Name	Type
aws_ec2_managed_prefix_list.prefixes	resource
aws_ec2_transit_gateway_route.inspection_egress	resource
aws_ec2_transit_gateway_route.inspection_inbound	resource
aws_ec2_transit_gateway_route.trusted_default	resource
aws_ec2_transit_gateway_route.trusted_route_table	resource
aws_ec2_transit_gateway_route_table.inspection_return	resource
aws_ec2_transit_gateway_route_table.trusted	resource
aws_ec2_transit_gateway_route_table.trusted_core	resource
aws_ec2_transit_gateway_route_table_association.inspection_egress	resource
aws_ec2_transit_gateway_route_table_association.inspection_endpoints	resource
aws_ec2_transit_gateway_route_table_association.inspection_inbound	resource
aws_ec2_transit_gateway_route_table_association.inspection_ingress	resource
aws_ec2_transit_gateway_route_table_association.trusted	resource
aws_ec2_transit_gateway_route_table_association.trusted_egress	resource
aws_ec2_transit_gateway_route_table_association.trusted_endpoints	resource
aws_ec2_transit_gateway_route_table_association.trusted_ingress	resource
aws_ec2_transit_gateway_route_table_propagation.inspection_egress	resource
aws_ec2_transit_gateway_route_table_propagation.inspection_endpoints	resource
aws_ec2_transit_gateway_route_table_propagation.inspection_ingress	resource
aws_ec2_transit_gateway_route_table_propagation.trusted	resource
aws_ec2_transit_gateway_route_table_propagation.trusted_egress	resource
aws_ec2_transit_gateway_route_table_propagation.trusted_endpoints	resource
aws_ec2_transit_gateway_route_table_propagation.trusted_ingress	resource
aws_ec2_transit_gateway_route_table_propagation.untrusted	resource
aws_ram_principal_association.associations	resource
aws_ram_resource_association.prefixes	resource
aws_ram_resource_share.prefixes	resource
aws_caller_identity.current	data source
aws_region.current	data source

Inputs

Name	Description	Type	Default	Required
amazon_side_asn	The ASN for the transit gateway.	number	n/a	yes
connectivity_config	The type of connectivity options for the transit gateway.	object({ inspection_with_all = optional(object({ # The name of the inbound route table. Defaults to 'inbound'. network = optional(object({ # Defines the configuration for the inspection network. availability_zones = number # The number of availablity zones to use for the inspection network. Required. Must match the # number of availability zones you use in the organization, due to symmetric routing requirements. name = optional(string, "inspection") # The name of the inspection network. Defaults to 'inspection'. private_netmask = optional(number, 24) # The netmask to use for the private network. Defaults to 24 vpc_cidr = optional(string, "100.64.0.0/21") # The CIDR block to use for the VPC. Defaults to carrier-grade NAT space. }), null) return_route_table_name = optional(string, "inspection-return") }), null) trusted = optional(object({ # Defines the configuration for the trusted routing trusted_attachments = optional(list(string), []) # The list of transit gateway attachments to trust e.g can see all the other untrusted networks. Defaults to an empty list. trusted_route_table_name = optional(string, "trusted") # The name of the trusted route table. Defaults to 'trusted'. trusted_core_route_table_name = optional(string, "trusted-core") }), null) })	n/a	yes
description	The description of the transit gateway to provision.	string	n/a	yes
tags	A map of tags to add to all resources.	map(string)	n/a	yes
enable_dns_support	Whether DNS support is enabled.	bool	true	no
enable_external_principals	Whether to enable external principals in the RAM share.	bool	true	no
enable_multicast_support	Whether multicast support is enabled.	bool	false	no
enable_vpn_ecmp_support	Whether VPN Equal Cost Multipath Protocol support is enabled.	bool	false	no
name	The name of the transit gateway to provision.	string	"tgw"	no
prefix_lists	Provides the ability to provision prefix lists, and share them with other accounts.	list(object({ name = string entry = list(object({ address_family = optional(string, "IPv4") cidr = string description = string max_entries = number })) }))	[]	no
prefix_ram_principals	The list of organizational units or accounts to share the prefix lists with.	list(string)	[]	no
ram_share_name	The name of the RAM share to create for the transit gateway.	string	"tgw-ram-share"	no
ram_share_principals	The list of organizational units or accounts to share the transit gateway with.	list(string)	[]	no
services	A collection of features and services associated with this connectivity domain.	object({ egress = optional(object({ network = object({ # Defines the configuration for an egress network. availability_zones = optional(number, 2) # The number of availablity zones to use for the egress network. Defaults to 2. ipam_pool_id = optional(string, null) # The ID of the IPAM pool to use for the egress network. Defaults to null. name = optional(string, "egress") # The name of the egress network. Defaults to 'egress'. private_netmask = optional(number, 28) # The netmask to use for the private network. Defaults to 28. public_netmask = optional(number, 28) # The netmask to use for the public network. Defaults to 28. vpc_cidr = optional(string, null) # The CIDR block to use for the VPC. Defaults to null, required when not using IPAM vpc_netmask = optional(string, null) # The netmask to use for the VPC. Defaults to null, required when using IPAM }) }), null) dns = optional(object({ # The list of organizational units or accounts to share the domain rule with. resolver_name = optional(string, "dns-resolver") # Defines the configuration for the endpoints network. network = object({ # Defines the configuration for the endpoints network. availability_zones = optional(number, 2) # The number of availablity zones to use for the endpoints network. Defaults to 2. ipam_pool_id = optional(string, null) # The ID of the IPAM pool to use for the endpoints network. Defaults to null. name = optional(string, "central-dns") # The name of the endpoints network. Defaults to 'endpoints'. private_netmask = optional(number, 24) # The netmask to use for the private network. Defaults to 24, ensure space for enough aws services. vpc_cidr = optional(string, null) # The CIDR block to use for the VPC. Defaults to null, required when not using IPAM vpc_netmask = optional(string, null) # The netmask to use for the VPC. Defaults to null, required when using IPAM }), domain_rules = optional(list(object({ ram_share_name = optional(string, "central-dns") # The name of the domain rule - this is mapped to the resource share name ram_principals = optional(map(string), {}) # The name of the resolver to use. Defaults to 'dns-resolver'. rules = list(object({ name = string # The name of the rule - the ram share name is domain.name + "-" + rule.name # The list of domain rules to apply to the domain. domain = string # The domain to apply the rule to. targets = optional(list(string), []) # The list of targets to apply the rule to - defaults to local resolver. })) })), []) }), null) endpoints = optional(object({ # Defines the configuration for the endpoints network. network = object({ # Defines the configuration for the endpoints network. availability_zones = optional(number, 2) # The number of availablity zones to use for the endpoints network. Defaults to 2. ipam_pool_id = optional(string, null) # The ID of the IPAM pool to use for the endpoints network. Defaults to null. name = optional(string, "endpoints") # The name of the endpoints network. Defaults to 'endpoints'. private_netmask = optional(number, 24) # The netmask to use for the private network. Defaults to 24, ensure space for enough aws services. vpc_cidr = optional(string, null) # The CIDR block to use for the VPC. Defaults to null, required when not using IPAM vpc_netmask = optional(string, null) # The netmask to use for the VPC. Defaults to null, required when using IPAM }) sharing = optional(object({ # Defines the configuration for the sharing network via AWS RAM principals = optional(list(string), []) # The list of organizational units or accounts to share the endpoints resolvers rules with. Defaults to an empty list. }), null) services = optional(map(object({ # Defines the configuration for the private endpoints in the shared network. private_dns_enabled = optional(bool, true) # Whether private DNS is enabled. Defaults to true. service_type = optional(string, "Interface") # The type of service, i.e. Gateway or Interface. Defaults to 'Interface' service = string # The name of the service i.e. ec2, ec2messages, ssm, ssmmessages, logs, kms, secretsmanager, s3.awsamazon.com policy = optional(string, null) # An optional IAM policy to use for the endpoint. Defaults to null. })), { ec2messages = { service = "ec2messages" }, ssm = { service = "ssm" }, ssmmessages = { service = "ssmmessages" }, }) }), null) ingress = optional(object({ # Defines the configuration for the ingress network. network = object({ # Defines the configuration for the ingress network. availability_zones = optional(number, 2) # The number of availablity zones to use for the ingress network. Defaults to 2. ipam_pool_id = optional(string, null) # The ID of the IPAM pool to use for the ingress network. Defaults to null. name = optional(string, "ingress") # The name of the ingress network. Defaults to 'ingress'. private_netmask = number # The netmask to use for the private network. Required, ensure space for enough aws services. public_netmask = number # The netmask to use for the public network. Required, ensure space for enough aws services. vpc_cidr = optional(string, null) # The CIDR block to use for the VPC. Defaults to null, required when not using IPAM vpc_netmask = optional(string, null) # The netmask to use for the VPC. Defaults to null, required when using IPAM }) }), null) })	{}	no

Outputs

Name	Description
account_id	The AWS account ID.
connectivity_type	The type of connectivity for the transit gateway.
egress_vpc_id	The ID of the VPC that is used for egress traffic.
egress_vpc_id_rt_attributes_by_type_by_az	The route table attributes of the egress VPC.
egress_vpc_private_subnet_attributes_by_az	The attributes of the egress VPC.
egress_vpc_public_subnet_attributes_by_az	The attributes of the egress VPC.
endpoints_vpc_id	The ID of the VPC that is used for endpoint traffic.
endpoints_vpc_id_rt_attributes_by_type_by_az	The route table attributes of the endpoints VPC.
endpoints_vpc_private_subnet_attributes_by_az	The attributes of the endpoints VPC.
ingress_vpc_id	The ID of the VPC that is used for ingress traffic.
ingress_vpc_id_rt_attributes_by_type_by_az	The route table attributes of the ingress VPC.
ingress_vpc_private_subnet_attributes_by_az	The attributes of the ingress VPC.
ingress_vpc_public_subnet_attributes_by_az	The attributes of the ingress VPC.
inspection_route_inbound_table_id	The ID of the inbound route table for inspection.
inspection_vpc_id	The ID of the VPC that is used for inspection traffic.
inspection_vpc_id_rt_attributes_by_type_by_az	The route table attributes of the inspection VPC.
inspection_vpc_private_subnet_attributes_by_az	The attributes of the inspection VPC.
inspection_vpc_public_subnet_attributes_by_az	The attributes of the inspection VPC.
region	The AWS region in which the resources are created.
transit_gateway_id	The ID of the transit gateway.
trusted_core_route_table_id	The ID of the trusted core route table.
trusted_route_table_id	The ID of the trusted route table.
workloads_route_table_id	The ID of the workloads route table.