1.16.0-pre.1

Summary of Changes

Major Changes:

• Add a readinessProbe to the kvstoremesh container that reports initial synchronization status to support configuring a separate, initial rate-limit to be used while synchronizing. Both clustermesh-apiserver and kvstoremesh now use a high initial rate-limit to decrease start time. (#30361, @thorn3r)
• bpf: introduce encrypted overlay datapath support (#31073, @ldelossa)
• multicast: add CLIs to manage multicast BPF maps (#31355, @harsimran-pabla)
• policy/k8s: Add support for CIDRGroupRef in IngressDeny and EgressDeny (#30933, @pippolo84)
• This adds a new policy field, EnableDefaultDeny, which permits the creation of network polices that do not drop non-matching traffic. (#30572, @squeed)

Minor Changes:

• Add "node-map-max" to allow configuring nodemap size. (#31407, @tommyp1ckles)
• Add helm values.schema.json file for validating supplied values for correct type. (#30631, @ubergesundheit)
• Add line numbers and file names to all metrics in 'cilium-dbg bpf metrics list' (#30972, @ti-mo)
• Add support for ClusterIP service advertisement with BGP Control Plane (#30963, @chaunceyjiang)
• Add support for ExternalIP service advertisement with BGP Control Plane (#31245, @chaunceyjiang)
• agent: add several new flags to control Cilium's datapath events notifications (#30063, @mvisonneau)
• Allow the Host Firewall and IPv6 BPF masquerading to be used together. (#31511, @qmonnet)
• Allows for using AWS SGs in the ingress section of rules. (#30708, @Alex-Waring)
• bgpv1: Add Local internalTrafficPolicy support for ClusterIP advertisements (#31442, @chaunceyjiang)
• bgpv1: BGP Control Plane metrics (#31469, @YutaroHayakawa)
• bugtool: Collect hubble metrics (#31533, @chancez)
• Change Node IPAM to select all nodes if externalTrafficPolicy=Cluster and add nodeipam.cilium.io/match-node-labels annotation (#31406, @MrFreezeex)
• cleanup: Remove deprecated values for KPR (#31286, @sayboras)
• cni: use default logger with timestamps. (#31014, @tommyp1ckles)
• envoy: Add support for exposing Envoy Admin API (#30655, @sayboras)
• feat: Add the http return code to metric api_processed_total (#31227, @vipul-21)
• Fix Cilium default values for EKS when Cilium clustermesh-apiserver LoadBalancer fails to create NLB with AWS Load Balancer Controller with syntax error. (#31329, @oshangalwaduge)
• Fixes a bug where ToFQDN IPs may be garbage collected too early, disrupting existing connections. (#31205, @squeed)
• fqdn: avoid expensive sort/unique of names during GC (#30920, @tklauser)
• GatewayAPI supports to setting the number of trusted loadbalancer hops (#30662, @chaunceyjiang)
• helm: Bump minimum k8s version to v1.21+ (#31648, @sayboras)
• ingress: Allow strict kube-proxy-replacement (#31284, @sayboras)
• Introduce cilium-dbg encrypt flush --stale flag to remove XFRM states and policies with stale node IDs. (#31159, @pchaigno)
• labelsfilter: Always apply Cluster entity specific identity-relevant label (#31178, @soggiest)
• Only detach Cilium-owned legacy XDP programs when XDP is disabled (#31654, @ti-mo)
• pkg/kvstore/allocator: Standardize usage of logfields (#30526, @antonipp)
• Remove helm option enable-remote-node-identity after being deprecated in v1.15. (#31228, @doniacld)
• Support IPv4 fragmentation for service backends. (#31364, @julianwiedmann)
• This allows the initialDelaySeconds option to be configured. This allows users running larger clusters to extend the time it takes for preflight to become ready. (#30495, @chaunceyjiang)
• WG: Improve L7 checks (#31299, @brb)

Bugfixes:

• bpf: use bpf_htons instead of using shift (#31247, @chez-shanpu)
• Cilium allows selecting 'lo' as a device again. (#31200, @bimmlerd)
• cilium-health: Fix broken retry loop in cilium-health-ep controller (#31622, @gandro)
• cni: Allow text-ts log format value (#31686, @sayboras)
• cni: Use batch endpoint deletion API in chaining plugin (#31456, @sayboras)
• envoy: register secret syncer even if only CEC is enabled (#31447, @mhofstetter)
• Fix a bug in the StateDB library that may have caused stale read after write. This may have potentially affected the L2 announcements feature and the node address selection. (#31164, @joamaki)
• Fix a bug that could cause local packet delivery to be skipped, leading to lower performance, when IPsec was enabled and --devices provided. (#31345, @pchaigno)
• Fix a bug where pod label updates are not reflected in endpoint labels in presence of filtered labels. (#31395, @tklauser)
• Fix the logic of the api-server connectivity check for the kubernetes probe (#31019, @tkna)
• fix: Delegated ipam not configure ipv6 if ipv6 disabled in agent (#31104, @tamilmani1989)
• Fixed issue when updated nodes were being reported with unknown connectivity status in health report (#30917, @marseel)
• Fixed issue with assigning 0 nodeID when corresponding bpf map run out of space.
  Potentially it could have impacted connectivity in large clusters (>4k nodes) with IPSec or Mutual Auth enabled.
  Otherwise, it was merely generating unnecessary error log messages. (#31380, @marseel)
• fqdn: Fixed bug that caused DNS Proxy to be overly restrictive on allowed DNS selectors. (#31328, @nathanjsweet)
• gateway-api: Ensure hostname check when set on both the HTTPRoute and the Gateway Listener (#30686, @cjvirtucio87)
• gateway-api: fixed RequestRedirect picks wrong port with multiple listeners (#31361, @chaunceyjiang)
• gateway-api: Retrieve LB service from same namespace (#31271, @sayboras)
• gateway-api: shorten the length of the value of the svc's label. (#31292, @chaunceyjiang)
• helm: Update pod affinity for cilium-envoy (#31150, @sayboras)
• hubble/relay: Fix certificate reloading in PeerManager (#31376, @glrf)
• hubble: fix parsing of invalid HTTP URLs (#31100, @kaworu)
• Hubble: fix traffic direction and is reply when IPSec is enabled (#31211, @kaworu)
• ingress/gateway-api: sort virtual hosts in CEC (#31493, @mhofstetter)
• ingress/gateway-api: stable envoy listener filterchain sort-order (#31572, @mhofstetter)
• k8s/utils: correctly filter out labels in StripPodSpecialLabels (#31421, @tklauser)
• metric: Avoid memory leak/increase in cilium-agent (#31714, @sayboras)
• metrics: Disable prometheus metrics by default (#31144, @joestringer)
• operator: fix errors/warnings metric. (#31214, @tommyp1ckles)
• Updated Kernel parsing to handle single and double digit kernel version as well (#30699, @MeherRushi)

CI Changes:

• Additionally test host firewall + KPR disabled in E2E tests (#30914, @giorio94)
• AKS: avoid overlapping pod and service CIDRs (#31504, @bimmlerd)
• bgpv1: avoid object tracker vs informer race (#31010, @bimmlerd)
• bgpv1: fix Test_PodIPPoolAdvert flakiness (#31365, @rastislavs)
• bgpv2/ci: added watch reactor for bgp cluster config (#31381, @harsimran-pabla)
• bpf: fix go testdata check in ci (#31419, @mhofstetter)
• Checkout the target branch, instead of the default one, on pull_request based GHA test workflows (#31198, @giorio94)
• ci-e2e: Add e2e test with WireGuard + Host Firewall (#31594, @qmonnet)
• ci-e2e: Add matrix for bpf.tproxy and ingress-controller (#31272, @sayboras)
• ci/ipsec: Print more info to debug credentials removal check failures (#31652, @qmonnet)
• ci: Bump lvh-kind ssh-startup-wait-retries (#31387, @YutaroHayakawa)
• ci: check license of third party Go dependencies (#31129, @rolinh)
• ci: fail container scans on vulnerability scan results (#31092, @ferozsalam)
• contrib/scripts: Remove false positives from check-go-testdata.sh (#31089, @dylandreimerink)
• deflake endpointmanager tests (#31488, @bimmlerd)
• Drop legacy and superseded test from the Ginkgo suite (#31411, @giorio94)
• Drop the remaining references to the CILIUM_CLI_MODE environment variable in GHA workflows. (#31199, @giorio94)
• gateway-api: Enable GRPCRoute conformance tests (#31055, @sayboras)
• gh/workflows: Add IPsec key rotation action and use it in ci-eks / ci-ipsec-e2e (#29704, @brb)
• gh: workflows: clarify reference to issue #23283 (#31118, @julianwiedmann)
• gha: disable fail-fast on integration tests (#31420, @giorio94)
• gha: fix coredns logs retrieval in conformance-clustermesh (#31509, @giorio94)
• gha: Remove manual device setting (#31435, @sayboras)
• gha: retrieve additional coredns-related troubleshooting info (#31384, @giorio94)
• introduce ARM github workflows (#31196, @aanm)
• ipam: deepcopy interface resource correctly. (#26998, @tommyp1ckles)
• k8s_install.sh: specify the CNI version (#31182, @aanm)
• loader: fix issue where errors cancelled compile cause error logs. (#30988, @tommyp1ckles)
• Make BPF unit tests reproducible (#31526, @ti-mo)
• Make testdata build output more stable by reducing header includes (#31644, @ti-mo)
• renovate: temporarily do not update GoBGP (#31123, @rastislavs)
• slices: don't modify missed input slice in test (#31119, @bimmlerd)
• test/verifier: Keep existing environment when running make (#31632, @gentoo-root)
• test/verifier: Sort BPF program names for stable output (#31617, @gentoo-root)
• test: Update KPR value in ipsec upgrade jobs (#31649, @sayboras)
• update azure k8s versions (#31220, @brlbil)
• workflows: Cover IPsec encrypted overlay mode in end-to-end tests (#31637, @pchaigno)
• workflows: Debug info for key rotations (#31627, @pchaigno)
• workflows: ipsec-e2e: add missing key types for some configs (#31636, @julianwiedmann)

Misc Changes:

• Add monitor aggregation for all events related to packets ingressing to the network-facing device. (#31015, @learnitall)
• Add the documentation for using serviceAdvertisements (#31331, @chaunceyjiang)
• agent: Remove redundant pod spec checks (#31105, @aditighag)
• agent: Wrap propagating errors from proxy wait group (#31398, @aditighag)
• all: remove repetitive words (#31566, @deterclosed)
• api: Upgrade go-swagger version to v0.30.5 (#31647, @sayboras)
• Avoid depending on sysctl in the kind.sh script for IPv6 determination (#31180, @giorio94)
• bgpv1: Adjust ConnectionRetryTimeSeconds to 1 in component tests (#31218, @YutaroHayakawa)
• bgpv1: Disable PodCIDR Reconciler for unsupported IPAM modes (#31181, @YutaroHayakawa)
• bgpv2: fix operator flaky test cases (#31255, @harsimran-pabla)
• bgpv2: Introducing pod cidr reconciler for bgpv2. (#30815, @harsimran-pabla)
• bgpv2: introducing PodIPPool reconciler (#31546, @harsimran-pabla)
• bgpv2: remove automatic bgp peering policy translation to new BGP CRDs. (#31252, @harsimran-pabla)
• bpf,config: Add ENABLE_LOCAL_REDIRECT_POLICY macro (#31098, @aditighag)
• bpf: add node_key to alignchecker (#31393, @julianwiedmann)
• bpf: Don't skip local delivery for plain-text packets when IPsec is enabled (#31193, @pchaigno)
• bpf: host: optimize from-host's ICMPv6 path (#31127, @julianwiedmann)
• bpf: lxc: also set from_tunnel for IPv6 CT entries (#30877, @julianwiedmann)
• bpf: nodeport: add nodeport_rev_dnat_ingress_ipv4_hook infra (#31244, @jibi)
• bpf: nodeport: clean up ct_state usage in nodeport_lb*() (#31427, @julianwiedmann)
• bpf: nodeport: don't forward host id in nodeport_lb4 (#31120, @jibi)
• bpf: nodeport: simplify CT entry validation in nodeport_lb*() (#31165, @julianwiedmann)
• bpf: update unreachable-tailcall.o after updating CILIUM_BUILDER_IMAGE (#31412, @mhofstetter)
• bpf: xdp: remove unused set_encrypt_dip() (#31367, @julianwiedmann)
• bugtool: Capture memory fragmentation info from /proc (#30966, @pchaigno)
• cec: move config property 'envoy-config-timeout' into hive config (#31086, @mhofstetter)
• chore(deps): update all github action dependencies (main) (#31282, @renovate[bot])
• chore(deps): update all github action dependencies (main) (#31443, @renovate[bot])
• chore(deps): update all github action dependencies (main) (#31573, @renovate[bot])
• chore(deps): update all github action dependencies (main) (#31697, @renovate[bot])
• chore(deps): update all github action dependencies (main) (patch) (#31130, @renovate[bot])
• chore(deps): update all lvh-images main (main) (patch) (#31131, @renovate[bot])
• chore(deps): update all lvh-images main (main) (patch) (#31230, @renovate[bot])
• chore(deps): update all lvh-images main to bpf-next-20240309.012251 (main) (patch) (#31276, @renovate[bot])
• chore(deps): update all lvh-images main to bpf-next-20240315.012542 (main) (patch) (#31440, @renovate[bot])
• chore(deps): update all-dependencies (main) (#31275, @renovate[bot])
• chore(deps): update cilium/cilium-cli action to v0.16.0 (main) (#31281, @renovate[bot])
• chore(deps): update cilium/little-vm-helper action to v0.0.17 (main) (#31695, @renovate[bot])
• chore(deps): update dependency cilium/cilium-cli to v0.16.0 (main) (#31171, @renovate[bot])
• chore(deps): update dependency cilium/cilium-cli to v0.16.3 (main) (#31386, @renovate[bot])
• chore(deps): update dependency cilium/cilium-cli to v0.16.4 (main) (#31673, @renovate[bot])
• chore(deps): update docker.io/library/golang:1.22.1 docker digest to 0b55ab8 (main) (#31438, @renovate[bot])
• chore(deps): update gcr.io/distroless/static-debian11:nonroot docker digest to 55c6361 (main) (#31439, @renovate[bot])
• chore(deps): update github/codeql-action action to v3.24.8 (main) (#31479, @renovate[bot])
• chore(deps): update go to v1.22.1 (main) (#31277, @renovate[bot])
• chore(deps): update golangci/golangci-lint docker tag to v1.57.1 (main) (#31576, @renovate[bot])
• chore(deps): update golangci/golangci-lint docker tag to v1.57.2 (main) (#31696, @renovate[bot])
• chore(deps): update hubble cli to v0.13.2 (main) (#31320, @renovate[bot])
• chore(deps): update module github.com/go-jose/go-jose/v3 to v3.0.3 [security] (main) (#31241, @renovate[bot])
• chore: update json-mock image source in examples (#31373, @loomkoom)
• cilium, bpf: pkts/byte count conversion for ct (#31087, @borkmann)
• cilium-dbg: listing load-balancing configurations displays L7LB proxy port (#31503, @mhofstetter)
• cilium: Enable plain IPIP/IP6IP6 termination (#31213, @borkmann)
• config: Remove unused ENCRYPT_IFACE macro (#31323, @pchaigno)
• container/bitlpm: Add Lookup Boolean Return Value (#31037, @nathanjsweet)
• contrib: Add installation script for tools in devcontainer (#31534, @fujitatomoya)
• controller: Add and use lookup function for controllers (#31236, @christarazi)
• datapath, bpf: Remove unnecessary IPsec code (#31344, @pchaigno)
• dev: Enable IPv6 system setting for devcontainer environment. (#31268, @fujitatomoya)
• doc,bgpv1: Add some failure scenarios (#31249, @YutaroHayakawa)
• doc,bgpv1: Bootstrapping BGP CPlane failure scenario doc (#31153, @YutaroHayakawa)
• doc,bgpv1: More failure scenario and wording improvement (#31470, @YutaroHayakawa)
• doc: Clarified GwAPI KPR prerequisites (#31366, @PhilipSchmid)
• doc: Document APAC community meeting (#31461, @YutaroHayakawa)
• docs: aks: avoid overlapping service and pod CIDRs (#31543, @bimmlerd)
• docs: Correct dynamic hubble exporter sample configs example (#31445, @littlesheng19)
• docs: Document No node ID found drops in case of remote node deletion (#31635, @pchaigno)
• docs: Fix 'kubectl exec' invocations (quotes, double dash separator) in example script kafka-sw-gen-traffic.sh (#30462, @saintdle)
• docs: Fix profiling related debugging instructions (#31044, @aditighag)
• docs: Fix various typos in README.rst (#31072, @payneInTheBrian)
• docs: ipsec: document native-routing + Egress proxy case (#31478, @julianwiedmann)
• docs: Suggest using operator logs for troubleshooting (#31500, @simonfelding)
• docs: Update link to cilium/ebpf's list of eBPF program types (#31699, @haiyuewa)
• docs: Update link to USERS.md in README from RAW Github to standard Github UI (#30589, @ondrejsika)
• docs: Warn on key rotations during upgrades (#31437, @pchaigno)
• Document the process for disabling workflows (#31603, @michi-covalent)
• Downgrade L2 Neighbor Discovery failure log to Debug (#31179, @YutaroHayakawa)
• endpointmanager: Improve health reporter messages when stopped (#31231, @christarazi)
• envoy: Bump golang version to 1.21.8 (#31224, @sayboras)
• envoy: cleanup istio specifics (#31448, @mhofstetter)
• envoy: move config values from global config into hive cell (#31351, @mhofstetter)
• envoy: Remove deprecated runtime key logs (#31108, @sayboras)
• envoy: support configurable Envoy base id in embedded mode (#31449, @mhofstetter)
• fix 'mismatch' typos in error messages (#31660, @julianwiedmann)
• Fix helm template for hubble-relay prometheus annotations (#31253, @glrf)
• Fix running tests locally in kind. (#31234, @gentoo-root)
• fix(deps): update all go dependencies main (main) (#31112, @renovate[bot])
• fix(deps): update all go dependencies main (main) (#31278, @renovate[bot])
• fix(deps): update all go dependencies main (main) (#31441, @renovate[bot])
• fix(deps): update all go dependencies main (main) (#31462, @renovate[bot])
• fix(deps): update google.golang.org/genproto/googleapis/rpc digest to a219d84 (main) (#31305, @renovate[bot])
• fix(deps): update google.golang.org/genproto/googleapis/rpc digest to c811ad7 (main) (#31322, @renovate[bot])
• fix(deps): update module github.com/docker/docker to v25.0.5+incompatible [security] (main) (#31531, @renovate[bot])
• gateway-api: Replace deprecated status (#31111, @sayboras)
• helm: Remove pipe in value comments to avoid breaking Helm reference (#31588, @qmonnet)
• helm: update nodeinit image using renovate (#31641, @tklauser)
• hive/cell/health: don't warn when reporting on stopped reporter. (#31262, @tommyp1ckles)
• hubble/relay/server: remove unused Server.stop chan (#31560, @tklauser)
• Ignore kvstore node events for the local node, to avoid unnecessarily increasing the ipcache_errors_total (cannot_overwrite_by_source) metric. (#31399, @giorio94)
• images/builder: get rid of annoying git ownership warnings (#31538, @ti-mo)
• images: bump cni plugins to v1.4.1 (#31347, @aanm)
• Improve compatibility with LLVM 17. (#31403, @gentoo-root)
• Improve compatibility with LLVM 17. (#31459, @gentoo-root)
• Improve insertNodeNeighbor behavior to report health (#29415, @derailed)
• Improve LocalNodeStore.Get() performance and fix possible deadlock (#31013, @giorio94)
• ingress/gateway-api: stable address order for Ingress hostnetwork listener addresses (#31477, @mhofstetter)
• ingress: sort all shared ingresses during model generation (#31494, @mhofstetter)
• ingress: Update docs with network policy example (#31060, @sayboras)
• IPAM: Refactors Node API Types to Support Separate IP Families (#30684, @danehans)
• ipam: Remove unused variable (#31401, @christarazi)
• ipcache: Remove synchronous CIDR identity allocation (#31311, @gandro)
• iptables: Manage IP sets independently with the stateDB reconciler (#31099, @pippolo84)
• iptables: Simplify proxy rules removing ingress/egress flag (#31068, @pippolo84)
• iptables: Unit tests cleanup (#31368, @pippolo84)
• kind: reset sysctl net.ipv4.ip_unprivileged_port_start to 1024 (#31370, @mhofstetter)
• lint: Remove temp variable in the 'for' loop (#31523, @sayboras)
• loader: add message if error is ENOTSUP (#31413, @kkourt)
• lxcmap: Fix comment about byte-order (#31362, @joestringer)
• Make it clear USERS.md should be production use cases (#31316, @xmulligan)
• Makefiles: Allow external input for go build/test/clean flags. (#29646, @wanlin31)
• Miscellaneous cleanups around node discovery (#31397, @giorio94)
• modularize node discovery (#31589, @dylandreimerink)
• multicast: modify list operations from iterator to batch lookup. (#31562, @harsimran-pabla)
• node: add support for injection of optional ipset filter (#31550, @giorio94)
• node: Replace ipv[46]MasqAddrs with Table[NodeAddress] (#30457, @joamaki)
• pkg/ip: Updates PrefixToIps() to Limit the Number of Returned IPs (#30921, @danehans)
• policy/k8s: Refactor and move ToServices translation to policy package (#31062, @gandro)
• policy: Fix missing labels from SelectorCache selectors (#31358, @christarazi)
• Prepare for release v1.16.0-pre.0 (#31121, @aanm)
• proxy: configurable portrange (#31556, @mhofstetter)
• proxy: remove unused ifaces and code for proxy <-> endpoint interaction (#31547, @mhofstetter)
• README: Update releases (#31665, @thorn3r)
• Remove HAVE_LARGE_INSN_LIMIT (#31094, @dylandreimerink)
• Remove Istio ambient compatibility blurb (#31525, @bleggett)
• Remove old bpf feature probes (#31096, @dylandreimerink)
• Remove tcx links created by Cilium 1.16 onwards (#31553, @ti-mo)
• renovate: Drop references to Cilium 1.12 (#31148, @joestringer)
• renovate: separate major.minor.patch for lvh images (#31126, @aanm)
• secret-sync: improve logging (#31415, @mhofstetter)
• signal: remove spare debug logs (#31723, @tklauser)
• stream: Relocate to cilium/stream (#30846, @joamaki)
• update readme with 1.16.0-pre.0 (#31128, @aanm)

Docker Manifests

cilium

quay.io/cilium/cilium:v1.16.0-pre.1@sha256:f822fed7e9ab9ef9251e3e21eaf6d4d5179a6b5831e147c3ab1caaa3f9b17b79

clustermesh-apiserver

quay.io/cilium/clustermesh-apiserver:v1.16.0-pre.1@sha256:6489a11ebdf28be5238842afaea4e5e2a9628e8c4fb66d712b3998fb1bfa034b

docker-plugin

quay.io/cilium/docker-plugin:v1.16.0-pre.1@sha256:0540dce44dc09dd54cbb1a665736664913dc242b9bca261fb138b8ac6de3aa8e

hubble-relay

quay.io/cilium/hubble-relay:v1.16.0-pre.1@sha256:80a213c50bc9915b73950c2efbbc04a32ab2df5058e0d5afe86c64d83a59cc2d

operator-alibabacloud

quay.io/cilium/operator-alibabacloud:v1.16.0-pre.1@sha256:9237c6dfc208e5f76c01922932d3c568f269356f485076a62c9a503d1af76710

operator-aws

quay.io/cilium/operator-aws:v1.16.0-pre.1@sha256:bf75d57fcfd1fb0b6ad8c6257e0758872278609847640fc4245cd04be139d7fd

operator-azure

quay.io/cilium/operator-azure:v1.16.0-pre.1@sha256:099fb5537d294bdf41755f93acbf8c6e2ecbca162b139028b4897f2904e04e4b

operator-generic

quay.io/cilium/operator-generic:v1.16.0-pre.1@sha256:73e8c7a415dfd3c6bb166848248c719ced5db53123c0f29c77e08771d1ec8400

operator

quay.io/cilium/operator:v1.16.0-pre.1@sha256:eb3303b6290ee9b06da28c383a65c680d03bc2028f6bdc046d5f1494eb5a485c