Modernisation Platform Terraform Loadbalancer Module with Access Logs enabled

A Terraform module that creates an application loadbalancer (with loadbalancer security groups) or network loadbalancer in AWS with logging enabled, s3 to store logs and Athena DB to query logs.

An s3 bucket name can be provided in the module by adding the existing_bucket_name variable and adding the bucket name. Otherwise, if no bucket exists one will be created and no variable needs to be set in the module. Application loadbalancers and network loadbalancers do not log to the same S3 bucket location. If you're using existing buckets they also need to have specific permissions applied to them. See the External buckets section for more information.

Either pass in existing security group(s) to attach to the load balancer using the security_groups variable, or define loadbalancer_ingress_rules and loadbalancer_egress_rules variables to create a new security group within the module.

If using the module to create the security group, you can use locals to define the rules for the loadbalancer_ingress_rules and loadbalancer_egress_rules variables as in the below example.

locals {
  loadbalancer_ingress_rules = {
    "cluster_ec2_lb_ingress" = {
      description     = "Cluster EC2 loadbalancer ingress rule"
      from_port       = 8080
      to_port         = 8080
      protocol        = "tcp"
      cidr_blocks     = []
      security_groups = []
    },
    "cluster_ec2_bastion_ingress" = {
      description     = "Cluster EC2 bastion ingress rule"
      from_port       = 3389
      to_port         = 3389
      protocol        = "tcp"
      cidr_blocks     = []
      security_groups = []
    }
  }
  loadbalancer_egress_rules = {
    "cluster_ec2_lb_egress" = {
      description     = "Cluster EC2 loadbalancer egress rule"
      from_port       = 443
      to_port         = 443
      protocol        = "tcp"
      cidr_blocks     = ["0.0.0.0/0"]
      security_groups = []
    }
  }
}

Loadbalancer target groups and listeners need to be created separately.

The use of "aws_glue_catalog_table" resources for application and network loadbalancers means that logs appearing in the S3 bucket will be available to query via Athena without having to carry out any manual Athena config steps.

Module created S3 access_logs bucket

By default the loadbalancer will set up an access_logs bucket for you, unless you set access_logs = false initially for testing or some other reason. Setting this back to true after the lb has been deployed will then create the bucket for you. The reason for the 'depends_on' here is that without the module.s3-bucket resource being created first, the module.lb resource will fail with a validation error.

  depends_on = [
    module.s3-bucket
  ]

External buckets

If you decide to use externally created buckets they need to have been created and have appropriate permissions applied to them BEFORE access_logs = true and existing_bucket_name values are added to the lb code. If you add these values before the bucket is created you will get an error because the lb module will run a check to see if the s3 bucket is writeable and if it is not it will fail.

So to use external_bucket_name the deployment steps are:

1. Set access_logs = false in the lb create code & create the lb
2. Create the bucket - making sure the appropriate permissions are applied
3. Set existing_bucket_name in the lb create code as your-bucket-name-GUID

External bucket permissions

For simplicity the bucket can be created with the following policy attached to it. This applies whether the loadbalancer is an "application" or "network" loadbalancer. This uses the bucket_policy_v2 implementation using the s3_bucket module:

  public-lb-logs-bucket = {
    sse_algorithm = "AES256" # required for Network Loadbalancers
    bucket_policy_v2 = [
      {
        effect = "Allow"
        actions = [
          "s3:PutObject",
        ]
        principals = {
          identifiers = ["arn:aws:iam::652711504416:root"]
          type        = "AWS"
        }
      },
      {
        effect = "Allow"
        actions = [
          "s3:PutObject"
        ]
        principals = {
          identifiers = ["delivery.logs.amazonaws.com"]
          type        = "Service"
        }

        conditions = [
          {
            test     = "StringEquals"
            variable = "s3:x-amz-acl"
            values   = ["bucket-owner-full-control"]
          }
        ]
      },
      {
        effect = "Allow"
        actions = [
          "s3:GetBucketAcl"
        ]
        principals = {
          identifiers = ["delivery.logs.amazonaws.com"]
          type        = "Service"
        }
      }
    ]
    iam_policies = module.baseline_presets.s3_iam_policies
  }

If you want to see exactly what policies are needed for each then refer to NLB Requirements and ALB Requirements

Network Loadbalancer caveats

• Access logs are created only if the load balancer has a TLS listener and they contain information only about TLS requests.
• Network loadbalancers only support SSE-S3 encryption for access logs, not aws:kms (AWS managed keys).
• They can support customer managed keys but this is not currently supported by this module.
• No "verify bucket permissions" test file is created in the relevant bucket, only that the terraform apply step will fail with a validation error if the permissions and the bucket encryption parameters are not correct.

Application Loadbalancer caveats

• It's worth noting that Application LB's will create a test file in the S3 bucket to verify that the bucket permissions are correct.

Usage

module "lb-access-logs-enabled" {
  source = "github.com/ministryofjustice/modernisation-platform-terraform-loadbalancer"

  providers = {
    # Here we use the default provider for the S3 bucket module, buck replication is disabled but we still
    # Need to pass the provider to the S3 bucket module
    aws.bucket-replication = aws
  }
  vpc_all                             = "${local.vpc_name}-${local.environment}"
  #existing_bucket_name               = "my-bucket-name"
  application_name                    = local.application_name
  public_subnets                      = [data.aws_subnet.public_az_a.id,data.aws_subnet.public_az_b.id,data.aws_subnet.public_az_c.id]
  loadbalancer_ingress_rules          = local.loadbalancer_ingress_rules
  tags                                = local.tags
  account_number                      = local.environment_management.account_ids[terraform.workspace]
  region                              = local.app_data.accounts[local.environment].region
  enable_deletion_protection          = false
  idle_timeout                        = 60
}

Requirements

Name	Version
terraform	>= 1.0.1
aws	~> 4.0

Providers

Name	Version
aws	~> 4.0
template	n/a

Modules

Name	Source	Version
s3-bucket	github.com/ministryofjustice/modernisation-platform-terraform-s3-bucket	v6.1.1

Resources

Name	Type
aws_athena_database.lb-access-logs	resource
aws_athena_named_query.main	resource
aws_athena_workgroup.lb-access-logs	resource
aws_lb.loadbalancer	resource
aws_security_group.lb	resource
aws_elb_service_account.default	data source
aws_iam_policy_document.bucket_policy	data source
aws_region.current	data source
aws_vpc.shared	data source
template_file.lb-access-logs	data source

Inputs

Name	Description	Type	Default	Required
account_number	Account number of current environment	string	n/a	yes
application_name	Name of application	string	n/a	yes
enable_deletion_protection	If true, deletion of the load balancer will be disabled via the AWS API. This will prevent Terraform from deleting the load balancer.	bool	n/a	yes
existing_bucket_name	The name of the existing bucket name. If no bucket is provided one will be created for them.	string	""	no
force_destroy_bucket	A boolean that indicates all objects (including any locked objects) should be deleted from the bucket so that the bucket can be destroyed without error. These objects are not recoverable.	bool	false	no
idle_timeout	The time in seconds that the connection is allowed to be idle.	string	n/a	yes
loadbalancer_egress_rules	Security group egress rules for the loadbalancer	map(object({ description = string from_port = number to_port = number protocol = string security_groups = list(string) cidr_blocks = list(string) }))	n/a	yes
loadbalancer_ingress_rules	Security group ingress rules for the loadbalancer	map(object({ description = string from_port = number to_port = number protocol = string security_groups = list(string) cidr_blocks = list(string) }))	n/a	yes
public_subnets	Public subnets	list(string)	n/a	yes
region	AWS Region where resources are to be created	string	n/a	yes
tags	Common tags to be used by all resources	map(string)	n/a	yes
vpc_all	The full name of the VPC (including environment) used to create resources	string	n/a	yes

Outputs

Name	Description
athena_db	n/a
load_balancer	n/a
security_group	n/a

Looking for issues?

If you're looking to raise an issue with this module, please create a new issue in the Modernisation Platform repository.

Requirements

Name	Version
terraform	>= 1.0.1
aws	~> 5.0

Providers

Name	Version
aws	~> 5.0

Modules

Name	Source	Version
s3-bucket	github.com/ministryofjustice/modernisation-platform-terraform-s3-bucket	568694e50e03630d99cb569eafa06a0b879a1239

Resources

Name	Type
aws_athena_database.lb-access-logs	resource
aws_athena_workgroup.lb-access-logs	resource
aws_glue_catalog_table.application_lb_logs	resource
aws_glue_catalog_table.network_lb_logs	resource
aws_iam_policy.glue_s3	resource
aws_iam_role.glue	resource
aws_iam_role_policy_attachment.glue_s3	resource
aws_iam_role_policy_attachment.glue_service	resource
aws_lb.loadbalancer	resource
aws_lb_target_group.this	resource
aws_lb_target_group_attachment.this	resource
aws_security_group.lb	resource
aws_elb_service_account.default	data source
aws_iam_policy_document.bucket_policy	data source
aws_iam_policy_document.glue_assume	data source
aws_iam_policy_document.glue_s3	data source
aws_vpc.shared	data source

Inputs

Name	Description	Type	Default	Required
access_logs	A boolean that determines whether to have access logs	bool	true	no
account_number	Account number of current environment	string	n/a	yes
application_name	Name of application	string	n/a	yes
dns_record_client_routing_policy	(optional) Indicates how traffic is distributed among network load balancer Availability Zones only. Possible values are any_availability_zone (client DNS queries are resolved among healthy LB IP addresses across all LB Availability Zones), partial_availability_zone_affinity (85 percent of client DNS queries will favor load balancer IP addresses in their own Availability Zone, while the remaining queries resolve to any healthy zone) and availability_zone_affinity (Client DNS queries will favor load balancer IP address in their own Availability Zone).	string	"any_availability_zone"	no
enable_cross_zone_load_balancing	A boolean that determines whether cross zone load balancing is enabled. In application load balancers this feature is always enabled and cannot be disabled. In network and gateway load balancers this feature is disabled by default but can be enabled.	bool	false	no
enable_deletion_protection	If true, deletion of the load balancer will be disabled via the AWS API. This will prevent Terraform from deleting the load balancer.	bool	n/a	yes
existing_bucket_name	The name of the existing bucket name. If no bucket is provided one will be created for them.	string	""	no
force_destroy_bucket	A boolean that indicates all objects (including any locked objects) should be deleted from the bucket so that the bucket can be destroyed without error. These objects are not recoverable.	bool	false	no
idle_timeout	The time in seconds that the connection is allowed to be idle.	string	null	no
internal_lb	A boolean that determines whether the load balancer is internal or internet-facing.	bool	false	no
lb_target_groups	Map of load balancer target groups, where key is the name	map(object({ port = optional(number) attachment_port = optional(number) deregistration_delay = optional(number) health_check = optional(object({ enabled = optional(bool) interval = optional(number) healthy_threshold = optional(number) matcher = optional(string) path = optional(string) port = optional(number) timeout = optional(number) unhealthy_threshold = optional(number) })) stickiness = optional(object({ enabled = optional(bool) type = string cookie_duration = optional(number) cookie_name = optional(string) })) }))	{}	no
load_balancer_type	application or network	string	"application"	no
loadbalancer_egress_rules	Create new security group with these egress rules for the loadbalancer. Or use the security_groups var to attach existing group(s)	map(object({ description = string from_port = number to_port = number protocol = string security_groups = list(string) cidr_blocks = list(string) }))	{}	no
loadbalancer_ingress_rules	Create new security group with these ingress rules for the loadbalancer. Or use the security_groups var to attach existing group(s)	map(object({ description = string from_port = number to_port = number protocol = string security_groups = list(string) cidr_blocks = list(string) }))	{}	no
public_subnets	Badly named variable, use subnets instead. Keeping for backward compatibility	list(string)	[]	no
region	AWS Region where resources are to be created	string	n/a	yes
s3_versioning	A boolean that determines whether s3 will have versioning	bool	true	no
security_groups	List of existing security group ids to attach to the load balancer. You can use this instead of loadbalancer_ingress_rules,loadbalancer_egress_rules vars	list(string)	null	no
subnets	List of subnet IDs. Typically use private subnet for internal LBs and public for public LBs	list(string)	[]	no
tags	Common tags to be used by all resources	map(string)	n/a	yes
vpc_all	The full name of the VPC (including environment) used to create resources	string	n/a	yes

Outputs

Name	Description
athena_db	n/a
lb_target_groups	n/a
load_balancer	n/a
load_balancer_arn	n/a
load_balancer_dns_name	n/a
load_balancer_zone_id	n/a
security_group	n/a