Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions docs/adrs/057-service-identity-and-east-west-zero-trust.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,25 @@ identity ([ADR-041](041-pod-identity-for-tenant-workloads.md)/[ADR-047](047-pod-
and human identity ([ADR-053](053-identity-and-cross-system-authorization-strategy.md)) planes. The mTLS posture
becomes part of the tier's **network** isolation column ([ADR-049](049-tenant-model-team-tenant-zone.md)).

> **Amendment (2026-07-06, #1193) — Phase 1 (transparent encryption) split out and built into the module.**
> Decision #1's *encryption* half is concretized and made buildable now; the *mTLS / SPIFFE* half (decisions
> #2–#4) stays Proposed and parked (regulated-tier only — no regulated tenant exists yet — and it still needs
> the open Cilium-mTLS-vs-SPIRE spike).
>
> - **Backend: WireGuard** (`encryption.type=wireguard`) — in-kernel on the AL2023 6.x nodes (kernel 6.12, no
> key management) and simplest on the platform's **overlay datapath** (tunnel/VXLAN + cluster-pool IPAM),
> which sidesteps the WireGuard caveats that live in native-ENI mode. Cilium stacks it under the VXLAN
> tunnel and auto-adjusts MTU for the added overhead.
> - **Open question resolved: fleet-default, not tier-gated.** Encryption-in-transit is a blanket good and
> decision #3 already puts it in the `standard` tier, so it is enabled cluster-wide rather than per-tier.
> - **Scope: pod-to-pod only** — `node_encryption` (host-to-host) stays off for Phase 1 (it also covers host +
> health-check traffic and is a more invasive later step).
> - **Delivered here:** gated module support in the `cilium` module (`encryption_enabled` / `encryption_type`
> / `node_encryption`, default off). **Enablement is a separate step:** flipping a unit's input triggers a
> **rolling Cilium restart** on the CNI — highest blast radius — so it rolls out **preprod-first**, in a
> maintenance window, from the main checkout, with a **perf-overhead measurement** before the platform
> cluster follows. Status flips to Accepted once applied + verified.

## Context

North-south (ingress) is Gateway API + TLS. East-west is governed by Cilium **L3/L4 NetworkPolicy +
Expand Down
7 changes: 7 additions & 0 deletions infra/modules/cilium/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,10 @@ Deploys Cilium CNI via Helm with a **configurable datapath** and cloud-specific
| `pod_cidr_mask_size` | `24` (256 IPs/node) | _(n/a)_ |
| `egress_masquerade_interfaces` | `""` (all) | `"ens+"` |

## Transparent encryption (in-transit)

East-west zero trust ([ADR-057](../../../docs/adrs/057-service-identity-and-east-west-zero-trust.md) Phase 1). Set `encryption_enabled = true` to encrypt pod traffic on the wire between nodes; `encryption_type` defaults to **WireGuard** (in-kernel on the AL2023 6.x nodes, no key management, simplest on the overlay datapath — Cilium stacks it under the VXLAN tunnel and auto-adjusts MTU). `node_encryption` (host-to-host) stays off for Phase 1. Off by default; enabling is a **rolling Cilium restart**, so roll it out per environment (preprod first) and verify with `cilium status` (`Encryption: Wireguard`). Cryptographic workload identity (Cilium mTLS / SPIFFE) is ADR-057 Phase 2 — not in this module yet.

**Overlay** decouples pod IPs from the VPC (pods draw from `pod_cidr`, encapsulated VXLAN between nodes) — pod density is bounded by `pod_cidr`, not the node subnet. **ENI native** gives pods routable VPC IPs (no encapsulation, VPC-level flow visibility) but density is bounded by the node subnet size and instance ENI limits. Per-cluster `pod_cidr`s must not overlap (keeps the design ClusterMesh-ready). See `infra/docs/08-kubernetes-network-design.md`.

## Usage
Expand Down Expand Up @@ -142,6 +146,8 @@ No modules.
| <a name="input_debug"></a> [debug](#input\_debug) | Debug configuration for Cilium | `any` | <pre>{<br/> "enabled": false<br/>}</pre> | no |
| <a name="input_egress_masquerade_interfaces"></a> [egress\_masquerade\_interfaces](#input\_egress\_masquerade\_interfaces) | Interface glob to masquerade on (e.g. 'ens+' for ENI native). Empty = all interfaces (overlay default). Must be empty when bpf\_masquerade = true. | `string` | `""` | no |
| <a name="input_enable_ipv4_masquerade"></a> [enable\_ipv4\_masquerade](#input\_enable\_ipv4\_masquerade) | Masquerade pod egress to the node IP for traffic leaving the cluster. | `bool` | `true` | no |
| <a name="input_encryption_enabled"></a> [encryption\_enabled](#input\_encryption\_enabled) | Enable Cilium transparent encryption of pod traffic in transit between nodes (ADR-057 Phase 1). Off by default. | `bool` | `false` | no |
| <a name="input_encryption_type"></a> [encryption\_type](#input\_encryption\_type) | Transparent-encryption backend when encryption\_enabled = true. 'wireguard' (platform default — in-kernel, no key management) or 'ipsec'. | `string` | `"wireguard"` | no |
| <a name="input_environment"></a> [environment](#input\_environment) | Environment name (e.g., dev, test, prod) | `string` | `"dev"` | no |
| <a name="input_external_ips_enabled"></a> [external\_ips\_enabled](#input\_external\_ips\_enabled) | Enable ExternalIPs service support | `bool` | `true` | no |
| <a name="input_gateway_api_crd_version"></a> [gateway\_api\_crd\_version](#input\_gateway\_api\_crd\_version) | Gateway API CRD version to install (experimental channel). Set to empty string to skip CRD installation. | `string` | `"v1.3.0"` | no |
Expand Down Expand Up @@ -171,6 +177,7 @@ No modules.
| <a name="input_mtu"></a> [mtu](#input\_mtu) | Override the device MTU. 0 = auto-detect (Cilium subtracts tunnel overhead). Pin to 1500 if jumbo frames cause PMTU black-holing on egress. | `number` | `0` | no |
| <a name="input_namespace"></a> [namespace](#input\_namespace) | Kubernetes namespace to install Cilium into | `string` | `"kube-system"` | no |
| <a name="input_native_routing_cidr"></a> [native\_routing\_cidr](#input\_native\_routing\_cidr) | CIDR that Cilium treats as natively routable (no masquerade within it). Required when routing\_mode = native AND ipam\_mode != eni; ENI mode auto-derives it. | `string` | `""` | no |
| <a name="input_node_encryption"></a> [node\_encryption](#input\_node\_encryption) | Also encrypt node-to-node (host) traffic, not just pod-to-pod. Off for Phase 1 — node encryption also covers host + health-check traffic and is a later, more invasive step. | `bool` | `false` | no |
| <a name="input_node_port_enabled"></a> [node\_port\_enabled](#input\_node\_port\_enabled) | Enable NodePort service support | `bool` | `true` | no |
| <a name="input_operator_prometheus_enabled"></a> [operator\_prometheus\_enabled](#input\_operator\_prometheus\_enabled) | Enable Prometheus metrics for Cilium operator | `bool` | `true` | no |
| <a name="input_operator_resources_limits_cpu"></a> [operator\_resources\_limits\_cpu](#input\_operator\_resources\_limits\_cpu) | CPU limit for Cilium operator | `string` | `"500m"` | no |
Expand Down
2 changes: 2 additions & 0 deletions infra/modules/cilium/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ locals {
var.egress_masquerade_interfaces != "" ? { egressMasqueradeInterfaces = var.egress_masquerade_interfaces } : {},
var.routing_mode == "native" && var.native_routing_cidr != "" ? { ipv4NativeRoutingCIDR = var.native_routing_cidr } : {},
var.mtu > 0 ? { MTU = var.mtu } : {},
# Transparent encryption (ADR-057 Phase 1) — WireGuard/IPsec on the wire between nodes.
var.encryption_enabled ? { encryption = { enabled = true, type = var.encryption_type, nodeEncryption = var.node_encryption } } : {},
)

# Irreducible per-cloud plumbing — NOT the datapath (that's variable-driven above).
Expand Down
32 changes: 32 additions & 0 deletions infra/modules/cilium/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,38 @@ variable "mtu" {
}
}

# ---------------------------------------------------------------------------
# Transparent encryption (in-transit) — ADR-057 Phase 1 (east-west zero trust)
#
# Encrypts pod traffic on the wire between nodes. Off by default; opted into
# per-unit. WireGuard is the platform default: in-kernel on the AL2023 6.x
# nodes (no key management), and simplest on the overlay (tunnel/cluster-pool)
# datapath — Cilium stacks it under the VXLAN tunnel and auto-adjusts MTU for
# the added overhead. Enabling is a rolling Cilium restart; roll out per env.
# ---------------------------------------------------------------------------

variable "encryption_enabled" {
description = "Enable Cilium transparent encryption of pod traffic in transit between nodes (ADR-057 Phase 1). Off by default."
type = bool
default = false
}

variable "encryption_type" {
description = "Transparent-encryption backend when encryption_enabled = true. 'wireguard' (platform default — in-kernel, no key management) or 'ipsec'."
type = string
default = "wireguard"
validation {
condition = contains(["wireguard", "ipsec"], var.encryption_type)
error_message = "encryption_type must be 'wireguard' or 'ipsec'."
}
}

variable "node_encryption" {
description = "Also encrypt node-to-node (host) traffic, not just pod-to-pod. Off for Phase 1 — node encryption also covers host + health-check traffic and is a later, more invasive step."
type = bool
default = false
}

variable "k8s_service_host" {
description = "Kubernetes API server hostname (required for BYOCNI — in-cluster service IP unreachable before CNI exists)"
type = string
Expand Down