Azure Kubernetes Service

This Terraform module creates an Azure Kubernetes Service.

Non-exhaustive feature list, most of them can be overridden:

Cluster created with User Assigned identity, and related role assignments are managed, for better dependencies lifecycle management
Default latest stable Kubernetes version used at creation
Cluster and containers logs sent to Log Analytics Workspace or Storage Account
Kube audit logs disabled by default for cost purpose
Cluster is private by default
Virtual Network integration
Azure CNI Overlay network mode by default, support for Azure CNI, Cilium & Kubelet. More in the Azure documentation
Azure CSI KeyVault enabled by default
Azure Policies enabled by default
Calico network policy enabled by default
Workload identities enabled by default
Additional node pools can be configured within the module
Azure naming convention for all resources
Container Insights enabled by default

Why replacing the previous Claranet AKS module

This modules supersedes the previous AKS module. We've built a new module to clean up the technical debt that piled up due to fast-moving AKS product and keep backwards compatibility for existing users. Also, we've decided to remove all Kubernetes resources from this new module as a recommended best practice and for better tooling responsibility segregation.

So this module does less, that why light, but it should do it better.

Global versioning rule for Claranet Azure modules

Module version	Terraform version	AzureRM version
>= 7.x.x	1.3.x	>= 3.0
>= 6.x.x	1.x	>= 3.0
>= 5.x.x	0.15.x	>= 2.0
>= 4.x.x	0.13.x / 0.14.x	>= 2.0
>= 3.x.x	0.12.x	>= 2.0
>= 2.x.x	0.12.x	< 2.0
< 2.x.x	0.11.x	< 2.0

Contributing

If you want to contribute to this repository, feel free to use our pre-commit git hook configuration which will help you automatically update and format some files for you by enforcing our Terraform code module best-practices.

More details are available in the CONTRIBUTING.md file.

Usage

This module is optimized to work with the Claranet terraform-wrapper tool which set some terraform variables in the environment needed by this module. More details about variables set by the terraform-wrapper available in the documentation.

module "azure_region" {
  source  = "claranet/regions/azurerm"
  version = "x.x.x"

  azure_region = var.azure_region
}

module "rg" {
  source  = "claranet/rg/azurerm"
  version = "x.x.x"

  location    = module.azure_region.location
  client_name = var.client_name
  environment = var.environment
  stack       = var.stack
}

module "run" {
  source  = "claranet/run/azurerm"
  version = "x.x.x"

  location       = module.azure_region.location
  location_short = module.azure_region.location_short
  client_name    = var.client_name
  environment    = var.environment
  stack          = var.stack

  resource_group_name = module.rg.resource_group_name

  monitoring_function_enabled = false
}

module "acr" {
  source  = "claranet/acr/azurerm"
  version = "x.x.x"

  location       = module.azure_region.location
  location_short = module.azure_region.location_short
  client_name    = var.client_name
  environment    = var.environment
  stack          = var.stack

  resource_group_name = module.rg.resource_group_name

  sku = "Standard"

  logs_destinations_ids = [module.run.log_analytics_workspace_id]
}

module "vnet" {
  source  = "claranet/vnet/azurerm"
  version = "x.x.x"

  location       = module.azure_region.location
  location_short = module.azure_region.location_short
  client_name    = var.client_name
  environment    = var.environment
  stack          = var.stack

  resource_group_name = module.rg.resource_group_name

  vnet_cidr = ["10.0.0.0/19"]
}

module "nodes_subnet" {
  source  = "claranet/subnet/azurerm"
  version = "x.x.x"

  location_short = module.azure_region.location_short
  client_name    = var.client_name
  environment    = var.environment
  stack          = var.stack

  resource_group_name = module.rg.resource_group_name

  name_suffix = "nodes"

  virtual_network_name = module.vnet.virtual_network_name

  subnet_cidr_list  = ["10.0.0.0/20"]
  service_endpoints = ["Microsoft.Storage", "Microsoft.KeyVault"]
}

module "aks_private_dns_zone" {
  source  = "claranet/private-endpoint/azurerm//modules/private-dns-zone"
  version = "x.x.x"

  environment = var.environment
  stack       = var.stack

  resource_group_name = module.rg.resource_group_name

  private_dns_zone_name      = "privatelink.${module.azure_region.location_cli}.azmk8s.io"
  private_dns_zone_vnets_ids = [module.vnet.virtual_network_id]
}

resource "tls_private_key" "key" {
  algorithm = "RSA"
}

module "containers_logs" {
  source = "claranet/run/azurerm//modules/logs"

  client_name         = var.client_name
  environment         = var.environment
  location            = module.azure_region.location
  location_short      = module.azure_region.location_short
  resource_group_name = module.rg.resource_group_name
  stack               = var.stack

  logs_storage_account_enabled        = false
  log_analytics_workspace_custom_name = "log-aks-containers-${var.environment}-${module.azure_region.location_short}"
}

module "aks" {
  source  = "claranet/aks-light/azurerm"
  version = "x.x.x"

  location       = module.azure_region.location
  location_short = module.azure_region.location_short
  client_name    = var.client_name
  environment    = var.environment
  stack          = var.stack

  resource_group_name = module.rg.resource_group_name

  kubernetes_version = "1.27.3"
  service_cidr       = "10.0.16.0/22"

  nodes_subnet = {
    name                 = module.nodes_subnet.subnet_name
    virtual_network_name = module.vnet.virtual_network_name
  }

  private_cluster_enabled = true
  private_dns_zone_type   = "Custom"
  private_dns_zone_id     = module.aks_private_dns_zone.private_dns_zone_id

  default_node_pool = {
    vm_size         = "Standard_B4ms"
    os_disk_size_gb = 64
  }

  node_pools = [{
    name                = "nodepool1"
    vm_size             = "Standard_B4ms"
    os_disk_type        = "Ephemeral"
    os_disk_size_gb     = 100
    vnet_subnet_id      = module.nodes_subnet.subnet_id
    enable_auto_scaling = true
    min_count           = 1
    max_count           = 10
  }]

  linux_profile = {
    username = "nodeadmin"
    ssh_key  = tls_private_key.key.public_key_openssh
  }

  container_registry_id = module.acr.acr_id

  oms_agent = {
    log_analytics_workspace_id = module.run.log_analytics_workspace_id
  }

  data_collection_rule = {
    custom_log_analytics_workspace_id = module.containers_logs.log_analytics_workspace_id
  }

  maintenance_window = {
    allowed = [{
      day   = "Monday"
      hours = [10, 11, 12, 13, 14]
    }]
  }

  maintenance_window_auto_upgrade = {
    frequency   = "RelativeMonthly"
    interval    = 1
    duration    = 4
    week_index  = "First"
    day_of_week = "Monday"
    start_time  = "10:00"
    utc_offset  = "+02:00"
  }

  logs_destinations_ids = [module.run.log_analytics_workspace_id]
}

Providers

Name	Version
azapi	~> 1.9, < 1.13
azuread	~> 2.31
azurecaf	~> 1.2, >= 1.2.22
azurerm	~> 3.63
null	>= 3.0

Modules

Name	Source	Version
diagnostics	claranet/diagnostic-settings/azurerm	~> 6.5.0

Resources

Name	Type
azapi_update_resource.aks_kubernetes_version	resource
azurerm_kubernetes_cluster.aks	resource
azurerm_kubernetes_cluster_node_pool.node_pools	resource
azurerm_monitor_data_collection_rule.dcr	resource
azurerm_monitor_data_collection_rule_association.dcr	resource
azurerm_role_assignment.aci_assignment	resource
azurerm_role_assignment.aks_kubelet_uai_acr_pull	resource
azurerm_role_assignment.aks_kubelet_uai_nodes_rg_contributor	resource
azurerm_role_assignment.aks_uai_private_dns_zone_contributor	resource
azurerm_role_assignment.aks_uai_route_table_contributor	resource
azurerm_role_assignment.aks_uai_subnets_network_contributor	resource
azurerm_user_assigned_identity.aks_user_assigned_identity	resource
null_resource.kubernetes_version_keeper	resource
azapi_resource.subnet_delegation	data source
azuread_service_principal.aci_identity	data source
azurecaf_name.aks	data source
azurecaf_name.aks_identity	data source
azurecaf_name.aks_nodes_rg	data source
azurecaf_name.dcr	data source
azurerm_kubernetes_service_versions.versions	data source
azurerm_subscription.current	data source

Inputs

Name	Description	Type	Default	Required
aci_subnet_id	ID of the Subnet for ACI virtual-nodes.	`string`	`null`	no
aks_automatic_channel_upgrade	The upgrade channel for this Kubernetes Cluster. Possible values are `patch`, `rapid`, `node-image` and `stable`. Setting this field to `null` sets this value to none.	`string`	`"patch"`	no
aks_http_proxy_settings	Azure Kubernetes Service HTTP proxy settings. URLs must be in format `http(s)://fqdn:port/`. When setting the `no_proxy_list` parameter, the AKS Private Endpoint domain name and the AKS VNet CIDR (or Subnet CIDRs) must be added to the list.	object({ https_proxy_url = optional(string) http_proxy_url = optional(string) trusted_ca = optional(string) no_proxy_list = optional(list(string), []) })	`null`	no
aks_network_mode	Azure Kubernetes Service network mode to use. Only available with Azure CNI.	`string`	`null`	no
aks_network_plugin	Azure Kubernetes Service network plugin to use. Possible names are `azure` and `kubenet`. Possible CNI modes are `None`, `Overlay` and `Cilium` for Azure CNI and `None` for Kubenet. Changing this forces a new resource to be created.	object({ name = optional(string, "azure") cni_mode = optional(string, "overlay") })	`{}`	no
aks_network_policy	Azure Kubernetes Service network policy to use.	`string`	`"calico"`	no
aks_pod_cidr	CIDR used by pods when network plugin is set to `kubenet` or `azure` CNI Overlay.	`string`	`null`	no
aks_route_table_id	Provide an existing Route Table ID when `outbound_type = "userDefinedRouting"`. Only available with Kubenet.	`string`	`null`	no
aks_sku_tier	Azure Kubernetes Service SKU tier. Possible values are Free ou Standard	`string`	`"Standard"`	no
aks_user_assigned_identity_custom_name	Custom name for the AKS user assigned identity resource.	`string`	`null`	no
aks_user_assigned_identity_resource_group_name	Resource Group where to deploy the Azure Kubernetes Service User Assigned Identity resource.	`string`	`null`	no
aks_user_assigned_identity_tags	Tags to add to AKS MSI	`map(string)`	`{}`	no
api_server_authorized_ip_ranges	IP ranges allowed to interact with Kubernetes API for public clusters. See documentation about "0.0.0.0/32" default value : - https://learn.microsoft.com/en-us/azure/aks/api-server-authorized-ip-ranges#allow-only-the-outbound-public-ip-of-the-standard-sku-load-balancer - https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/kubernetes_cluster#public_network_access_enabled Set to `0.0.0.0/0` to wide open (not recommended)	`list(string)`	[ "0.0.0.0/32" ]	no
auto_scaler_profile	Auto Scaler configuration.	object({ balance_similar_node_groups = optional(bool, false) expander = optional(string, "random") max_graceful_termination_sec = optional(number, 600) max_node_provisioning_time = optional(string, "15m") max_unready_nodes = optional(number, 3) max_unready_percentage = optional(number, 45) new_pod_scale_up_delay = optional(string, "10s") scale_down_delay_after_add = optional(string, "10m") scale_down_delay_after_delete = optional(string, "10s") scale_down_delay_after_failure = optional(string, "3m") scan_interval = optional(string, "10s") scale_down_unneeded = optional(string, "10m") scale_down_unready = optional(string, "20m") scale_down_utilization_threshold = optional(number, 0.5) empty_bulk_delete_max = optional(number, 10) skip_nodes_with_local_storage = optional(bool, true) skip_nodes_with_system_pods = optional(bool, true) })	`null`	no
azure_active_directory_rbac	Active Directory role based access control configuration.	object({ managed_integration_enabled = optional(bool, true) service_principal_azure_tenant_id = optional(string) admin_group_object_ids = optional(list(string), []) azure_rbac_enabled = optional(bool, true) service_principal_client_app_id = optional(string) service_principal_server_app_id = optional(string) service_principal_server_app_secret = optional(string) })	`null`	no
azure_policy_enabled	Option to enable Azure Policy add-on.	`bool`	`true`	no
client_name	Client name/account used in naming.	`string`	n/a	yes
container_registry_id	Azure Container Registry ID where Azure Kubernetes Service needs pull access.	`string`	`null`	no
custom_diagnostic_settings_name	Custom name of the diagnostics settings, name will be 'default' if not set.	`string`	`"default"`	no
custom_name	Custom AKS name, generated if not set.	`string`	`""`	no
data_collection_rule	AKS Data Collection Rule configuration.	object({ enabled = optional(bool, true) custom_log_analytics_workspace_id = optional(string) data_streams = optional(list(string), [ "Microsoft-ContainerLog", "Microsoft-ContainerLogV2", "Microsoft-KubeEvents", "Microsoft-KubePodInventory", "Microsoft-InsightsMetrics", "Microsoft-ContainerInventory", "Microsoft-ContainerNodeInventory", "Microsoft-KubeNodeInventory", "Microsoft-KubeServices", "Microsoft-KubePVInventory" ]) namespaces_filter = optional(list(string), [ "kube-system", "gatekeeper-system", "kube-node-lease", "calico-system", ]) namespace_filtering_mode = optional(string, "Exclude") data_collection_interval = optional(string, "5m") container_log_v2_enabled = optional(bool, true) })	`{}`	no
data_collection_rule_custom_name	Custom name for the AKS Data Collection Rule.	`string`	`null`	no
default_node_pool	Default Node Pool configuration.	object({ name = optional(string, "default") type = optional(string, "VirtualMachineScaleSets") vm_size = optional(string, "Standard_D2_v3") os_sku = optional(string, "Ubuntu") os_disk_type = optional(string, "Managed") os_disk_size_gb = optional(number) enable_auto_scaling = optional(bool, false) node_count = optional(number, 1) min_count = optional(number, 1) max_count = optional(number, 10) max_pods = optional(number) node_labels = optional(map(any)) node_taints = optional(list(any)) enable_host_encryption = optional(bool) enable_node_public_ip = optional(bool, false) orchestrator_version = optional(string) zones = optional(list(number), [1, 2, 3]) tags = optional(map(string), {}) temporary_name_for_rotation = optional(string) upgrade_settings = optional(object({ max_surge = optional(string, "10%") }), {}) })	`{}`	no
default_tags_enabled	Option to enable or disable default tags.	`bool`	`true`	no
environment	Project environment.	`string`	n/a	yes
extra_tags	Additional tags to add on resources.	`map(string)`	`{}`	no
http_application_routing_enabled	Whether HTTP Application Routing is enabled.	`bool`	`false`	no
key_vault_secrets_provider	Enable AKS built-in Key Vault secrets provider. If enabled, an identity is created by the AKS itself and exported from this module.	object({ secret_rotation_enabled = optional(bool, true) secret_rotation_interval = optional(string) })	`{}`	no
kubernetes_version	Version of Kubernetes to deploy.	`string`	`null`	no
linux_profile	Username and SSH public key for accessing Linux nodes with SSH.	object({ username = string ssh_key = string })	`null`	no
location	Azure region to use.	`string`	n/a	yes
location_short	Short string for Azure location.	`string`	n/a	yes
logs_categories	Log categories to send to destinations.	`list(string)`	`null`	no
logs_destinations_ids	List of destination resources IDs for logs diagnostic destination. Can be `Storage Account`, `Log Analytics Workspace` and `Event Hub`. No more than one of each can be set. If you want to specify an Azure EventHub to send logs and metrics to, you need to provide a formated string with both the EventHub Namespace authorization send ID and the EventHub name (name of the queue to use in the Namespace) separated by the `pipe` (\|) character.	`list(string)`	n/a	yes
logs_kube_audit_enabled	Whether to include `kube-audit` and `kube-audit-admin` logs from diagnostics settings collection. Enabling this can increase your Azure billing.	`bool`	`false`	no
logs_metrics_categories	Metrics categories to send to destinations.	`list(string)`	`null`	no
maintenance_window	Maintenance window configuration. This is the basic configuration for controlling AKS releases. https://learn.microsoft.com/en-us/azure/aks/planned-maintenance?tabs=azure-cli	object({ allowed = optional(list(object({ day = string hours = list(number) })), []) not_allowed = optional(list(object({ start = string end = string })), []) })	`null`	no
maintenance_window_auto_upgrade	Controls when to perform cluster upgrade whith more finely controlled cadence and recurrence settings compared to the basic one. https://learn.microsoft.com/en-us/azure/aks/planned-maintenance?tabs=azure-cli	object({ frequency = string interval = string duration = number day_of_week = optional(string) day_of_month = optional(string) week_index = optional(string) start_time = string utc_offset = optional(string) start_date = optional(string) not_allowed = optional(list(object({ start = string end = string })), []) })	`null`	no
monitor_metrics	Specifies a Prometheus add-on profile for this Kubernetes Cluster.	object({ annotations_allowed = optional(string, null) labels_allowed = optional(string, null) })	`null`	no
name_prefix	Optional prefix for the generated name.	`string`	`""`	no
name_suffix	Optional suffix for the generated name.	`string`	`""`	no
node_pools	A list of Node Pools to create.	list(object({ name = string vm_size = optional(string, "Standard_D2_v3") os_sku = optional(string, "Ubuntu") os_disk_type = optional(string, "Managed") os_disk_size_gb = optional(number) kubelet_disk_type = optional(string) enable_auto_scaling = optional(bool, false) node_count = optional(number, 1) min_count = optional(number, 1) max_count = optional(number, 10) max_pods = optional(number) node_labels = optional(map(any)) node_taints = optional(list(any)) enable_host_encryption = optional(bool) enable_node_public_ip = optional(bool, false) node_subnet = optional(object({ name = optional(string) virtual_network_name = optional(string) resource_group_name = optional(string) }), {}) pod_subnet = optional(object({ name = optional(string) virtual_network_name = optional(string) resource_group_name = optional(string) }), {}) priority = optional(string) eviction_policy = optional(string) orchestrator_version = optional(string) upgrade_settings = optional(object({ max_surge = optional(string, "10%") }), {}) zones = optional(list(number), [1, 2, 3]) tags = optional(map(string), {}) }))	`[]`	no
nodes_resource_group_name	Name of the Resource Group in which to put Azure Kubernetes Service nodes.	`string`	`null`	no
nodes_subnet	The Subnet used by nodes.	object({ name = string virtual_network_name = string resource_group_name = optional(string) })	n/a	yes
oidc_issuer_enabled	Whether the OIDC issuer URL should be enabled.	`bool`	`true`	no
oms_agent	OMS Agent configuration.	object({ log_analytics_workspace_id = optional(string) msi_auth_for_monitoring_enabled = optional(bool, true) })	n/a	yes
outbound_type	The outbound (egress) routing method which should be used. Possible values are `loadBalancer` and `userDefinedRouting`.	`string`	`"loadBalancer"`	no
pods_subnet	The Subnet containing the pods.	object({ name = optional(string) virtual_network_name = optional(string) resource_group_name = optional(string) })	`{}`	no
private_cluster_enabled	Configure Azure Kubernetes Service as a Private Cluster: https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/kubernetes_cluster#private_cluster_enabled	`bool`	`true`	no
private_cluster_public_fqdn_enabled	Specifies whether a Public FQDN for this Private Cluster should be added: https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/kubernetes_cluster#private_cluster_public_fqdn_enabled	`bool`	`false`	no
private_dns_zone_id	ID of the Private DNS Zone when `private_dns_zone_type = "Custom"`.	`string`	`null`	no
private_dns_zone_role_assignment_enabled	Option to enable or disable Private DNS Zone role assignment.	`bool`	`true`	no
private_dns_zone_type	Set Azure Kubernetes Service private DNS zone if needed and if private cluster is enabled (privatelink..azmk8s.io) - "Custom" : You will have to deploy a private DNS Zone on your own and provide the ID with <private_dns_zone_id> variable - "System" : AKS will manage the Private DNS Zone and creates it in the Node Resource Group - "None" : In case of None you will need to bring your own DNS server and set up resolving, otherwise cluster will have issues after provisioning. https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/kubernetes_cluster#private_dns_zone_id	`string`	`"System"`	no
resource_group_name	Name of the resource group.	`string`	n/a	yes
service_cidr	CIDR used by Kubernetes services (kubectl get svc).	`string`	n/a	yes
stack	Project stack name.	`string`	n/a	yes
vnet_integration	Virtual Network integration configuration.	object({ enabled = optional(bool, false) subnet_id = optional(string) })	`{}`	no
workload_identity_enabled	Whether Azure AD Workload Identity should be enabled for the cluster. `oidc_issuer_enabled` must be set to true to use this feature.	`bool`	`true`	no

Outputs

Name	Description
aks	AKS output object.
apiserver_endpoint	APIServer Endpoint of the Azure Kubernetes Service.
id	ID of the Azure Kubernetes Service.
identity_principal_id	AKS System Managed Identity principal ID.
key_vault_secrets_provider_identity	The User Managed Identity used by the Key Vault secrets provider.
kube_config	Kube configuration of the Azure Kubernetes Service.
kube_config_raw	Raw kubeconfig to be used by kubectl command.
kubelet_user_managed_identity	The Kubelet User Managed Identity used by the Azure Kubernetes Service.
kubernetes_version	Azure Kubernetes Service Kubernetes version.
managed_private_dns_zone_id	ID of the AKS' managed Private DNS Zone.
managed_private_dns_zone_name	Name of the AKS' managed Private DNS Zone.
managed_private_dns_zone_resource_group_name	Resource Group name of the AKS' managed Private DNS Zone.
name	Name of the Azure Kubernetes Service.
node_pools	Map of Azure Kubernetes Service Node Pools attributes.
nodes_resource_group_name	Name of the Resource Group in which Azure Kubernetes Service nodes are deployed.
oidc_issuer_url	The OIDC issuer URL that is associated with the Azure Kubernetes Service.
portal_fqdn	Portal FQDN of the Azure Kubernetes Service.
private_cluster_enabled	Whether private cluster is enabled.
private_fqdn	Private FQDNs of the Azure Kubernetes Service.
public_fqdn	Public FQDN of the Azure Kubernetes Service.
user_managed_identity	The User Managed Identity used by the Azure Kubernetes Service.

Name		Name	Last commit message	Last commit date
Latest commit History 147 Commits
.config		.config
.github		.github
.gitlab/merge_request_templates		.gitlab/merge_request_templates
examples		examples
.commitlintrc.yml		.commitlintrc.yml
.gitignore		.gitignore
.gitlab-ci.yml		.gitlab-ci.yml
.pre-commit-config.yaml		.pre-commit-config.yaml
.releaserc		.releaserc
.tool-versions		.tool-versions
CHANGELOG.md		CHANGELOG.md
CONTRIBUTING.md		CONTRIBUTING.md
LICENSE		LICENSE
NOTICE		NOTICE
README.md		README.md
data.tf		data.tf
locals-naming.tf		locals-naming.tf
locals-tags.tf		locals-tags.tf
locals.tf		locals.tf
outputs.tf		outputs.tf
r-aks-monitoring.tf		r-aks-monitoring.tf
r-aks.tf		r-aks.tf
r-logs.tf		r-logs.tf
r-naming.tf		r-naming.tf
r-node-pools.tf		r-node-pools.tf
r-rbac.tf		r-rbac.tf
renovate.json		renovate.json
terraform.tfvars.ci		terraform.tfvars.ci
variables-aks-light.tf		variables-aks-light.tf
variables-logs.tf		variables-logs.tf
variables-monitoring.tf		variables-monitoring.tf
variables-naming.tf		variables-naming.tf
variables-tags.tf		variables-tags.tf
variables.tf		variables.tf
versions.tf		versions.tf

License

claranet/terraform-azurerm-aks-light

Folders and files

Latest commit

History

Repository files navigation

Azure Kubernetes Service

Why replacing the previous Claranet AKS module

Global versioning rule for Claranet Azure modules

Contributing

Usage

Providers

Modules

Resources

Inputs

Outputs

Related documentation

About

Topics

Resources

License

Stars

Watchers

Forks

Languages