Skip to content

CMS-Enterprise/batcave-tf-cluster

Repository files navigation

Launch template with managed groups example

This is EKS example using workers custom launch template with managed groups feature in two different ways:

  • Using a defined existing launch template created outside module
  • Using dlaunch template which will be created by module with user customization

See the official documentation for more details.

Usage

To run this example you need to execute:

$ terraform init
$ terraform plan
$ terraform apply

Note that this example may create resources which cost money. Run terraform destroy when you don't need these resources.

Requirements

Name Version
terraform >= 0.13.1
aws >= 5.0
kubectl ~> 1.14.0
kubernetes >= 2.0
local >= 1.4
null >= 3.0
random >= 2.1

Providers

Name Version
aws >= 5.0
kubectl ~> 1.14.0
null >= 3.0

Modules

Name Source Version
eks terraform-aws-modules/eks/aws 20.8.5
eks_managed_node_groups terraform-aws-modules/eks/aws//modules/eks-managed-node-group 20.8.5
vpc_cni_irsa terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks 5.33

Resources

Name Type
aws_autoscaling_attachment.eks_managed_node_groups_alb_attachment resource
aws_autoscaling_attachment.eks_managed_node_groups_proxy_attachment resource
aws_autoscaling_attachment.eks_managed_node_groups_shared_attachment resource
aws_eks_access_entry.cluster_admin resource
aws_eks_access_entry.cluster_admin_readonly resource
aws_eks_access_entry.delete_ebs_volume resource
aws_eks_access_policy_association.cluster_admin resource
aws_eks_access_policy_association.cluster_admin_readonly resource
aws_eks_access_policy_association.delete_ebs_volume resource
aws_iam_policy.cloudwatch_logs resource
aws_iam_policy.node_policy resource
aws_iam_policy.ssm_managed_instance resource
aws_iam_role.cosign resource
aws_iam_role.eks_node resource
aws_iam_role_policy_attachment.additional resource
aws_iam_role_policy_attachment.cloudwatch_logs resource
aws_iam_role_policy_attachment.ebs_csi_driver resource
aws_iam_role_policy_attachment.eks_additional resource
aws_iam_role_policy_attachment.eks_cloudwatch_plolicy_attachment resource
aws_iam_role_policy_attachment.eks_custom_node_policy_attachment resource
aws_iam_role_policy_attachment.eks_node_policies resource
aws_iam_role_policy_attachment.eks_ssm_managed_instance resource
aws_iam_role_policy_attachment.ssm_managed_instance resource
aws_iam_role_policy_attachment.ssm_patching_policy_attachment resource
aws_iam_service_linked_role.autoscaling resource
aws_kms_key.eks resource
aws_lb.batcave_alb resource
aws_lb.batcave_alb_proxy resource
aws_lb.batcave_alb_shared resource
aws_lb_listener.batcave_alb_http resource
aws_lb_listener.batcave_alb_https resource
aws_lb_listener.batcave_alb_proxy_http resource
aws_lb_listener.batcave_alb_proxy_https resource
aws_lb_listener.batcave_alb_shared_http resource
aws_lb_listener.batcave_alb_shared_https resource
aws_lb_listener_rule.batcave_alb__proxy_https resource
aws_lb_listener_rule.batcave_alb_https resource
aws_lb_listener_rule.batcave_alb_shared_https resource
aws_lb_target_group.batcave_alb_http resource
aws_lb_target_group.batcave_alb_https resource
aws_lb_target_group.batcave_alb_proxy_https resource
aws_lb_target_group.batcave_alb_shared_https resource
aws_security_group.batcave_alb resource
aws_security_group.batcave_alb_proxy resource
aws_security_group.batcave_alb_shared resource
aws_security_group_rule.allow_all_nodes_to_other_nodes resource
aws_security_group_rule.allow_ingress_additional_prefix_lists resource
aws_security_group_rule.batcave_alb_egress resource
aws_security_group_rule.batcave_alb_ingress_cidrs_http resource
aws_security_group_rule.batcave_alb_ingress_cidrs_https resource
aws_security_group_rule.batcave_alb_ingress_pl_http resource
aws_security_group_rule.batcave_alb_ingress_pl_https resource
aws_security_group_rule.batcave_alb_proxy_egress resource
aws_security_group_rule.batcave_alb_proxy_ingress_cidrs_http resource
aws_security_group_rule.batcave_alb_proxy_ingress_cidrs_https resource
aws_security_group_rule.batcave_alb_proxy_ingress_pl_http resource
aws_security_group_rule.batcave_alb_proxy_ingress_pl_https resource
aws_security_group_rule.batcave_alb_shared_egress resource
aws_security_group_rule.batcave_alb_shared_ingress_cidrs_http resource
aws_security_group_rule.batcave_alb_shared_ingress_cidrs_https resource
aws_security_group_rule.batcave_alb_shared_ingress_pl_http resource
aws_security_group_rule.batcave_alb_shared_ingress_pl_https resource
aws_security_group_rule.eks_node_ingress_alb_proxy resource
aws_security_group_rule.eks_node_ingress_alb_shared resource
aws_security_group_rule.https-tg-ingress resource
aws_security_group_rule.https-vpc-ingress resource
aws_wafv2_web_acl_association.batcave_alb_shared_cms_waf_assoc resource
aws_wafv2_web_acl_association.cms_waf_assoc resource
aws_wafv2_web_acl_association.cms_waf_priv_assoc resource
kubectl_manifest.batcave_namespace resource
null_resource.kubernetes_requirements resource
aws_acm_certificate.acm_certificate data source
aws_ami.eks_ami data source
aws_caller_identity.current data source
aws_eks_cluster_auth.cluster data source
aws_iam_policy.ssm_patching_policy data source
aws_iam_policy_document.cloudwatch_logs data source
aws_iam_policy_document.node_policy data source
aws_kms_alias.sops data source
aws_partition.current data source
aws_security_groups.delete_ebs_volumes_lambda_security_group data source
aws_wafv2_web_acl.cms_waf data source

Inputs

Name Description Type Default Required
acm_cert_base_domain Base domain of the certificate used for the ALB Proxy string "" no
admin_principal_arns List of principal_arns that require admin access to the cluster list(string) [] no
admin_readonly_principal_arns List of principal_arns that require admin readonly access to the cluster list(string) [] no
alb_deletion_protection Enable/Disable ALB deletion protection for both ALBs bool false no
alb_drop_invalid_header_fields Indicates whether HTTP headers with header fields that are not valid are removed by the load balancer (true) or routed to targets (false). The default is false. Elastic Load Balancing requires that message header names contain only alphanumeric characters and hyphens. Only valid for Load Balancers of type application bool true no
alb_idle_timeout Default idle request timeout for the ALB string "60" no
alb_private_tags Additional private ALB tags map(any) null no
alb_proxy_ingress_cidrs List of CIDR blocks allowed to access the ALB Proxy; used to restrict public access to a certain set of IPs list(string) [] no
alb_proxy_ingress_prefix_lists List of Prefix List IDs allowed to access the ALB Proxy; used to restrict public access to a certain set of IPs list(string) [] no
alb_proxy_is_internal If the ALB Proxy should be using internal ips. Defaults to false, because the reason for ALB proxy existing is typically to make it accessible over the Internet bool false no
alb_proxy_restricted_hosts A list of allowable host for proxy alb set(string) [] no
alb_proxy_subnets List of subnet ids for the ALB Proxy to be deployed into list(string) [] no
alb_public_tags Additional public ALB tags map(any) null no
alb_restricted_hosts A list of allowable host for private alb set(string) [] no
alb_shared_ingress_cidrs List of CIDR blocks allowed to access the ALB Proxy; used to restrict public access to a certain set of IPs list(string) [] no
alb_shared_ingress_prefix_lists List of Prefix List IDs allowed to access the ALB Proxy; used to restrict public access to a certain set of IPs list(string) [] no
alb_shared_is_internal If the ALB in the shared subnet should be using internal ips. Defaults to false, because the reason for this ALB existing is to make it accessible over the Internet bool false no
alb_shared_restricted_hosts A list of allowable host for shared alb. Defaults to deny to ensure this load balancer is configured correctly. set(string)
[
"deny-by-default.example.com"
]
no
alb_shared_subnets List of subnet ids for the ALB in the shared subnet list(string) [] no
alb_ssl_security_policy ALB SSL Security Policy string "ELBSecurityPolicy-TLS13-1-2-Res-2021-06" no
alb_subnets_by_zone n/a map(string) n/a yes
ami_date n/a string "" no
ami_regex_override Overrides default AMI lookup regex, which grabs latest AMI matching cluster_version by default string "" no
autoscaling_group_tags Tags to apply to all autoscaling groups created map(any) {} no
cluster_additional_sg_prefix_lists n/a list(string) n/a yes
cluster_enabled_log_types A list of the desired control plane logging to enable. For more information, see Amazon EKS Control Plane Logging documentation (https://docs.aws.amazon.com/eks/latest/userguide/control-plane-logs.html) list(string)
[
"api",
"audit",
"authenticator",
"controllerManager",
"scheduler"
]
no
cluster_name n/a string n/a yes
cluster_security_group_additional_rules Map of security group rules to attach to the cluster security group, as you cannot change cluster security groups without replacing the instance map(any) {} no
cluster_service_cidr n/a string "172.20.0.0/16" no
cluster_version n/a string "1.28" no
create_alb_proxy Create an Application Load Balancer proxy to live in front of the K8s ALB and act as a proxy from the public Internet bool false no
create_alb_shared Creaes an ALB in the shared subnet bool false no
create_cosign_iam_role Flag to create Cosign IAM role bool false no
custom_node_policy_arns Custom node policy arns set(string) [] no
custom_node_pools n/a any {} no
delete_ebs_volume_role_arn principal_arn for delete ebs volume role string "" no
enable_cluster_creator_admin_permissions Grants the user who created the cluster admin permissions bool true no
enable_eks_managed_nodes Enables eks managed nodes bool false no
enable_hoplimit Enables a IMDSv2 hop limit of 1 on all nodes. Defaults to false bool false no
enable_self_managed_nodes Enables self managed nodes bool true no
enable_ssm_patching Enables Systems Manager to patch nodes bool false no
environment n/a string "dev" no
force_update_version Force update version bool true no
general_node_pool General node pool, required for hosting core services any
{
"desired_size": 3,
"instance_type": "c5.2xlarge",
"labels": {
"general": "true"
},
"max_size": 5,
"min_size": 2,
"taints": {}
}
no
grant_delete_ebs_volumes_lambda_access When set to true, a cluster role and permissions will be created to grant the delete-ebs-volumes Lambda access to the PersistentVolumes API. bool false no
host_subnets Override the ec2 instance subnets. By default, they are launche in private_subnets, just like the EKS control plane. list(any) [] no
iam_role_path n/a string "/delegatedadmin/developer/" no
iam_role_permissions_boundary n/a string "arn:aws:iam::373346310182:policy/cms-cloud-admin/developer-boundary-policy" no
instance_tags Instance custom tags map(any) null no
logging_bucket Name of the S3 bucket to send load balancer access logs. string null no
node_https_ingress_cidr_blocks List of CIDR blocks to allow into the node over the HTTPs port list(string)
[
"10.0.0.0/8",
"100.0.0.0/8"
]
no
node_schedule_shutdown_cron The cron expression for the shutdown schedule. If left empty, the cluster will not be stopped. Overrides node_schedule_shutdown_hour string "" no
node_schedule_shutdown_hour The hour of the day (0-23) the cluster should be shutdown. If left empty, the cluster will not be stopped. Will run every day otherwise. number -1 no
node_schedule_startup_cron The cron expression for the startup schedule. If left empty, the cluster will not be restarted after shutdown. Overrides node_schedule_startup_hour string "" no
node_schedule_startup_hour The hour of the day (0-23) the cluster should be restarted. If left empty, the cluster will not be restarted after shutdown. Will run every weekday otherwise. number -1 no
node_schedule_timezone The timezone of the schedule. Ex: 'America/New_York', 'America/Chicago', 'America/Denver', 'America/Los_Angeles', 'Pacific/Honolulu' See: https://www.joda.org/joda-time/timezones.html string "America/New_York" no
openid_connect_audiences OpenID Connect Audiences list(string) [] no
private_subnets n/a list(any) n/a yes
ssm_iam_patching_policy SSM IAM policy for patching string "cms-cloud-ssm-iam-policy-v3" no
ssm_tag_patch_group SSM Patching group for instances. For more information: https://cloud.cms.gov/patching-prerequisites string "AL2" no
ssm_tag_patch_window SSM Patching window for instances. For more information: https://cloud.cms.gov/patching-prerequisites string "ITOPS-Wave1-Non-Mktplc-DevTestImpl-MW" no
tags Global resource tags to apply to all resources map(any) null no
vpc_cidr_blocks List of VPC CIDR blocks list(string) n/a yes
vpc_id n/a string n/a yes

Outputs

Name Description
batcave_alb_proxy_dns DNS value of ALB created for proxying request
batcave_alb_shared_dns DNS value of ALB created for proxying requests through an ALB in the shared subnet
batcave_lb_dns DNS value of NLB created for routing traffic to apps
cloudwatch_log_group_arn Arn of cloudwatch log group created
cloudwatch_log_group_name Name of cloudwatch log group created
cluster_arn The Amazon Resource Name (ARN) of the cluster
cluster_certificate_authority_data Base64 encoded certificate data required to communicate with the cluster
cluster_endpoint Endpoint for EKS control plane.
cluster_iam_role_arn IAM role ARN of the EKS cluster
cluster_iam_role_name IAM role name of the EKS cluster
cluster_iam_role_unique_id Stable and unique string identifying the IAM role
cluster_id [deprecated, use cluster_name] The name of the EKS cluster. Will block on cluster creation until the cluster is really ready
cluster_name The name of the EKS cluster. Will block on cluster creation until the cluster is really ready
cluster_oidc_issuer_url The URL on the EKS cluster for the OpenID Connect identity provider
cluster_platform_version Platform version for the cluster
cluster_primary_security_group_id Cluster security group that was created by Amazon EKS for the cluster. Managed node groups use this security group for control-plane-to-data-plane communication. Referred to as 'Cluster security group' in the EKS console
cluster_security_group_arn Amazon Resource Name (ARN) of the cluster security group
cluster_security_group_id Security group ids attached to the cluster control plane.
cluster_status Status of the EKS cluster. One of CREATING, ACTIVE, DELETING, FAILED
cluster_version The version of the cluster being deployed
cosign_iam_role_arn n/a
eks_managed_node_group ARNs of all self managed node groups created
fargate_profiles Map of attribute maps for all EKS Fargate Profiles created
node_security_group_arn Amazon Resource Name (ARN) of the node shared security group
node_security_group_id ID of the node shared security group
oidc_provider_arn The ARN of the OIDC Provider if enable_irsa = true
private_alb_security_group_id The Security Group that controls access to the private ALB
self_managed_node_groups Map of attribute maps for all self managed node groups created
worker_security_group_id n/a