Implement the controller for API BGPPolicy

This commit implements the controller of API `BGPPolicy`, designed to advertise Service IPs, Egress IPs, and Pod IPs to BGP peers from selected Nodes. According to the spec of `BGPPolicy`, the Node selector is used to select Nodes to which a `BGPPolicy` is applied. Multiple `BGPPolicies` can be applied to the same Node. However, only one `BGPPolicy` can be effective on a Node, with others serving as alternatives. The first `BGPPolicy` applied to a Node will be the effective one, and the latter ones will serve as alternatives. The effective one may be changed in the following cases: - The current effective BGPPolicy is updated and not applied to the Node. - The current effective BGPPolicy is deleted. - The antrea-agent is rebooted, and an original alternative BGPPolicy is synced first and becomes effective. The BGP server instance is only started for the effective BGPPolicy on a Node. If the effective BGPPolicy is changed, the corresponding BGP server instance will be terminated by calling the `Stop` method, and a new BGP server instance will be created and started by calling the `Start` method for the new effective BGPPolicy. To start a BGP server instance, ASN, routerID, and listen port must be specified. ASN and listen port are specified in the spec of the effective BGPPolicy. For routerID, if the cluster is IPv4 only or dual stack, the IPv4 NodeIP is used as the routerID; if the cluster is IPv6 only, the routerID must be specified in the Node annotation `antrea.io/bgp-route-id`. Additionally, a new BGP server instance should be created and started when any of ASN, routerID, or listen port changes. The information of the BGP peers is specified in the effective BGPPolicy. The unique identification of a BGP peer is the peer IP address. To reconcile the latest BGP peers: - Get the BGP peers to be added and add them by calling the `AddPeer` method of the BGP server instance. - Get the BGP peers to be deleted and delete them by calling the `RemovePeer` method of the BGP server instance. - Get the remaining BGP peers and calculate the updated BGP peers, then update them by calling the `UpdatePeer` method of the BGP server instance. The information of the IPs to be advertised can be calculated from the spec of the effective BGPPolicy. Currently, we advertise the IPs and CIDRs to all the BGP peers. To reconcile the latest IPs to all BGP peers: - If the BGP server instance is newly created and started, advertise all the IPs by calling the `AdvertiseRoutes` method. - If the BGP server instance is not newly created and started: - Get the IPs/CIDRs to be added and advertise them by calling the `AdvertiseRoutes` method. - Get the IPs/CIDRs to be removed and withdraw them by calling the `WithdrawRoutes` method. The feature is gated by the alpha `BGPPolicy` feature gate. Signed-off-by: Hongliang Liu <lhongliang@vmware.com>
antrea-io · Jun 28, 2024 · 2e762c8 · 2e762c8
1 parent 6004f42
commit 2e762c8
Show file tree

Hide file tree

Showing 18 changed files with 2,796 additions and 11 deletions.
diff --git a/build/charts/antrea/README.md b/build/charts/antrea/README.md
@@ -64,6 +64,7 @@ Kubernetes: `>= 1.19.0-0`
 | auditLogging.maxAge | int | `28` | MaxAge is the maximum number of days to retain old log files based on the timestamp encoded in their filename. If set to 0, old log files are not removed based on age. |
 | auditLogging.maxBackups | int | `3` | MaxBackups is the maximum number of old log files to retain. If set to 0, all log files will be retained (unless MaxAge causes them to be deleted). |
 | auditLogging.maxSize | int | `500` | MaxSize is the maximum size in MB of a log file before it gets rotated. |
+| bgpPolicy.secretName | string | `"antrea-bgp-passwords"` | The name of the Secret storing the passwords of BGP peers. |
 | clientCAFile | string | `""` | File path of the certificate bundle for all the signers that is recognized for incoming client certificates. |
 | cni.hostBinPath | string | `"/opt/cni/bin"` | Installation path of CNI binaries on the host. |
 | cni.plugins | object | `{"bandwidth":true,"portmap":true}` | Chained plugins to use alongside antrea-cni. |

diff --git a/build/charts/antrea/conf/antrea-agent.conf b/build/charts/antrea/conf/antrea-agent.conf
@@ -91,6 +91,9 @@ featureGates:
 # Enable NodeLatencyMonitor to monitor the latency between Nodes.
 {{- include "featureGate" (dict "featureGates" .Values.featureGates "name" "NodeLatencyMonitor" "default" false) }}
 
+# Allow users to advertise Service IPs, Pod IPs, and Egress IPs to BGP peers.
+{{- include "featureGate" (dict "featureGates" .Values.featureGates "name" "BGPPolicy" "default" false) }}
+
 # Name of the OpenVSwitch bridge antrea-agent will create and use.
 # Make sure it doesn't conflict with your existing OpenVSwitch bridges.
 ovsBridge: {{ .Values.ovs.bridgeName | quote }}
@@ -443,3 +446,9 @@ secondaryNetwork:
 {{- end }}
 
 {{- end }}
+
+bgpPolicy:
+  # The name of the Secret storing passwords of the BGP peers. For each BGP peer, the Secret key is generated by
+  # concatenating its IP address and AS number, e.g., `192.168.1.1-65521`, `fec0::1-65521. If you decide to use a
+  # different Secret for storing the passwords, ensure that you update the ClusterRole of antrea-agent accordingly.
+  secretName: {{ .Values.bgpPolicy.secretName | quote }}
diff --git a/build/charts/antrea/templates/agent/bgp-secret.yaml b/build/charts/antrea/templates/agent/bgp-secret.yaml
@@ -0,0 +1,6 @@
+apiVersion: v1
+kind: Secret
+metadata:
+  name: {{ .Values.bgpPolicy.secretName }}
+  namespace: {{ .Release.Namespace }}
+type: Opaque
diff --git a/build/charts/antrea/templates/agent/clusterrole.yaml b/build/charts/antrea/templates/agent/clusterrole.yaml
@@ -177,6 +177,7 @@ rules:
   - apiGroups:
       - crd.antrea.io
     resources:
+      - bgppolicies
       - externalippools
       - ippools
       - trafficcontrols
@@ -234,3 +235,12 @@ rules:
       - create
       - patch
       - update
+  - apiGroups:
+      - ""
+    resources:
+      - secrets
+    resourceNames:
+      - {{ .Values.bgpPolicy.secretName }}
+    verbs:
+      - get
+      - watch
diff --git a/build/charts/antrea/values.yaml b/build/charts/antrea/values.yaml
@@ -207,6 +207,10 @@ secondaryNetwork:
   # [{bridgeName: "br1", physicalInterfaces: ["eth1"]}]
   ovsBridges: []
 
+bgpPolicy:
+  # -- The name of the Secret storing the passwords of BGP peers.
+  secretName: "antrea-bgp-passwords"
+
 agent:
   # -- Port for the antrea-agent APIServer to serve on.
   apiPort: 10350

diff --git a/build/yamls/antrea-aks.yml b/build/yamls/antrea-aks.yml
@@ -3684,6 +3684,14 @@ metadata:
   labels:
     app: antrea
 ---
+# Source: antrea/templates/agent/bgp-secret.yaml
+apiVersion: v1
+kind: Secret
+metadata:
+  name: antrea-bgp-passwords
+  namespace: kube-system
+type: Opaque
+---
 # Source: antrea/templates/agent/secret.yaml
 apiVersion: v1
 kind: Secret
@@ -3807,6 +3815,9 @@ data:
     # Enable NodeLatencyMonitor to monitor the latency between Nodes.
     #  NodeLatencyMonitor: false
 
+    # Allow users to advertise Service IPs, Pod IPs, and Egress IPs to BGP peers.
+    #  BGPPolicy: false
+
     # Name of the OpenVSwitch bridge antrea-agent will create and use.
     # Make sure it doesn't conflict with your existing OpenVSwitch bridges.
     ovsBridge: "br-int"
@@ -4115,6 +4126,12 @@ data:
       maxAge: 28
       # Compress enables gzip compression on rotated files.
       compress: true
+
+    bgpPolicy:
+      # The name of the Secret storing passwords of the BGP peers. For each BGP peer, the Secret key is generated by
+      # concatenating its IP address and AS number, e.g., `192.168.1.1-65521`, `fec0::1-65521. If you decide to use a
+      # different Secret for storing the passwords, ensure that you update the ClusterRole of antrea-agent accordingly.
+      secretName: "antrea-bgp-passwords"
   antrea-cni.conflist: |
     {
         "cniVersion":"0.3.0",
@@ -4445,6 +4462,7 @@ rules:
   - apiGroups:
       - crd.antrea.io
     resources:
+      - bgppolicies
       - externalippools
       - ippools
       - trafficcontrols
@@ -4502,6 +4520,15 @@ rules:
       - create
       - patch
       - update
+  - apiGroups:
+      - ""
+    resources:
+      - secrets
+    resourceNames:
+      - antrea-bgp-passwords
+    verbs:
+      - get
+      - watch
 ---
 # Source: antrea/templates/antctl/clusterrole.yaml
 kind: ClusterRole
@@ -5110,7 +5137,7 @@ spec:
         kubectl.kubernetes.io/default-container: antrea-agent
         # Automatically restart Pods with a RollingUpdate if the ConfigMap changes
         # See https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
-        checksum/config: f976029accf54258d01ad907fe19b50ac671eee014cd8aea968c6a0bc7e8f95a
+        checksum/config: 399630bc3a4615b700d529e9f841889fc647cea8392b80e6d82a3ad34a5d1a98
       labels:
         app: antrea
         component: antrea-agent
@@ -5348,7 +5375,7 @@ spec:
       annotations:
         # Automatically restart Pod if the ConfigMap changes
         # See https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
-        checksum/config: f976029accf54258d01ad907fe19b50ac671eee014cd8aea968c6a0bc7e8f95a
+        checksum/config: 399630bc3a4615b700d529e9f841889fc647cea8392b80e6d82a3ad34a5d1a98
       labels:
         app: antrea
         component: antrea-controller

diff --git a/build/yamls/antrea-eks.yml b/build/yamls/antrea-eks.yml
@@ -3684,6 +3684,14 @@ metadata:
   labels:
     app: antrea
 ---
+# Source: antrea/templates/agent/bgp-secret.yaml
+apiVersion: v1
+kind: Secret
+metadata:
+  name: antrea-bgp-passwords
+  namespace: kube-system
+type: Opaque
+---
 # Source: antrea/templates/agent/secret.yaml
 apiVersion: v1
 kind: Secret
@@ -3807,6 +3815,9 @@ data:
     # Enable NodeLatencyMonitor to monitor the latency between Nodes.
     #  NodeLatencyMonitor: false
 
+    # Allow users to advertise Service IPs, Pod IPs, and Egress IPs to BGP peers.
+    #  BGPPolicy: false
+
     # Name of the OpenVSwitch bridge antrea-agent will create and use.
     # Make sure it doesn't conflict with your existing OpenVSwitch bridges.
     ovsBridge: "br-int"
@@ -4115,6 +4126,12 @@ data:
       maxAge: 28
       # Compress enables gzip compression on rotated files.
       compress: true
+
+    bgpPolicy:
+      # The name of the Secret storing passwords of the BGP peers. For each BGP peer, the Secret key is generated by
+      # concatenating its IP address and AS number, e.g., `192.168.1.1-65521`, `fec0::1-65521. If you decide to use a
+      # different Secret for storing the passwords, ensure that you update the ClusterRole of antrea-agent accordingly.
+      secretName: "antrea-bgp-passwords"
   antrea-cni.conflist: |
     {
         "cniVersion":"0.3.0",
@@ -4445,6 +4462,7 @@ rules:
   - apiGroups:
       - crd.antrea.io
     resources:
+      - bgppolicies
       - externalippools
       - ippools
       - trafficcontrols
@@ -4502,6 +4520,15 @@ rules:
       - create
       - patch
       - update
+  - apiGroups:
+      - ""
+    resources:
+      - secrets
+    resourceNames:
+      - antrea-bgp-passwords
+    verbs:
+      - get
+      - watch
 ---
 # Source: antrea/templates/antctl/clusterrole.yaml
 kind: ClusterRole
@@ -5110,7 +5137,7 @@ spec:
         kubectl.kubernetes.io/default-container: antrea-agent
         # Automatically restart Pods with a RollingUpdate if the ConfigMap changes
         # See https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
-        checksum/config: f976029accf54258d01ad907fe19b50ac671eee014cd8aea968c6a0bc7e8f95a
+        checksum/config: 399630bc3a4615b700d529e9f841889fc647cea8392b80e6d82a3ad34a5d1a98
       labels:
         app: antrea
         component: antrea-agent
@@ -5349,7 +5376,7 @@ spec:
       annotations:
         # Automatically restart Pod if the ConfigMap changes
         # See https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
-        checksum/config: f976029accf54258d01ad907fe19b50ac671eee014cd8aea968c6a0bc7e8f95a
+        checksum/config: 399630bc3a4615b700d529e9f841889fc647cea8392b80e6d82a3ad34a5d1a98
       labels:
         app: antrea
         component: antrea-controller

diff --git a/build/yamls/antrea-gke.yml b/build/yamls/antrea-gke.yml
@@ -3684,6 +3684,14 @@ metadata:
   labels:
     app: antrea
 ---
+# Source: antrea/templates/agent/bgp-secret.yaml
+apiVersion: v1
+kind: Secret
+metadata:
+  name: antrea-bgp-passwords
+  namespace: kube-system
+type: Opaque
+---
 # Source: antrea/templates/agent/secret.yaml
 apiVersion: v1
 kind: Secret
@@ -3807,6 +3815,9 @@ data:
     # Enable NodeLatencyMonitor to monitor the latency between Nodes.
     #  NodeLatencyMonitor: false
 
+    # Allow users to advertise Service IPs, Pod IPs, and Egress IPs to BGP peers.
+    #  BGPPolicy: false
+
     # Name of the OpenVSwitch bridge antrea-agent will create and use.
     # Make sure it doesn't conflict with your existing OpenVSwitch bridges.
     ovsBridge: "br-int"
@@ -4115,6 +4126,12 @@ data:
       maxAge: 28
       # Compress enables gzip compression on rotated files.
       compress: true
+
+    bgpPolicy:
+      # The name of the Secret storing passwords of the BGP peers. For each BGP peer, the Secret key is generated by
+      # concatenating its IP address and AS number, e.g., `192.168.1.1-65521`, `fec0::1-65521. If you decide to use a
+      # different Secret for storing the passwords, ensure that you update the ClusterRole of antrea-agent accordingly.
+      secretName: "antrea-bgp-passwords"
   antrea-cni.conflist: |
     {
         "cniVersion":"0.3.0",
@@ -4445,6 +4462,7 @@ rules:
   - apiGroups:
       - crd.antrea.io
     resources:
+      - bgppolicies
       - externalippools
       - ippools
       - trafficcontrols
@@ -4502,6 +4520,15 @@ rules:
       - create
       - patch
       - update
+  - apiGroups:
+      - ""
+    resources:
+      - secrets
+    resourceNames:
+      - antrea-bgp-passwords
+    verbs:
+      - get
+      - watch
 ---
 # Source: antrea/templates/antctl/clusterrole.yaml
 kind: ClusterRole
@@ -5110,7 +5137,7 @@ spec:
         kubectl.kubernetes.io/default-container: antrea-agent
         # Automatically restart Pods with a RollingUpdate if the ConfigMap changes
         # See https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
-        checksum/config: 5299e6235e262daf606758cf900766470fcb8dd21a0d707a3ae284548bd8c2b2
+        checksum/config: 372806a584106e61cc1d23ff74753a822c7d4c7e65be453768faa06cc2d9b6ee
       labels:
         app: antrea
         component: antrea-agent
@@ -5346,7 +5373,7 @@ spec:
       annotations:
         # Automatically restart Pod if the ConfigMap changes
         # See https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
-        checksum/config: 5299e6235e262daf606758cf900766470fcb8dd21a0d707a3ae284548bd8c2b2
+        checksum/config: 372806a584106e61cc1d23ff74753a822c7d4c7e65be453768faa06cc2d9b6ee
       labels:
         app: antrea
         component: antrea-controller

diff --git a/build/yamls/antrea-ipsec.yml b/build/yamls/antrea-ipsec.yml
@@ -3684,6 +3684,14 @@ metadata:
   labels:
     app: antrea
 ---
+# Source: antrea/templates/agent/bgp-secret.yaml
+apiVersion: v1
+kind: Secret
+metadata:
+  name: antrea-bgp-passwords
+  namespace: kube-system
+type: Opaque
+---
 # Source: antrea/templates/agent/ipsec-secret.yaml
 apiVersion: v1
 kind: Secret
@@ -3820,6 +3828,9 @@ data:
     # Enable NodeLatencyMonitor to monitor the latency between Nodes.
     #  NodeLatencyMonitor: false
 
+    # Allow users to advertise Service IPs, Pod IPs, and Egress IPs to BGP peers.
+    #  BGPPolicy: false
+
     # Name of the OpenVSwitch bridge antrea-agent will create and use.
     # Make sure it doesn't conflict with your existing OpenVSwitch bridges.
     ovsBridge: "br-int"
@@ -4128,6 +4139,12 @@ data:
       maxAge: 28
       # Compress enables gzip compression on rotated files.
       compress: true
+
+    bgpPolicy:
+      # The name of the Secret storing passwords of the BGP peers. For each BGP peer, the Secret key is generated by
+      # concatenating its IP address and AS number, e.g., `192.168.1.1-65521`, `fec0::1-65521. If you decide to use a
+      # different Secret for storing the passwords, ensure that you update the ClusterRole of antrea-agent accordingly.
+      secretName: "antrea-bgp-passwords"
   antrea-cni.conflist: |
     {
         "cniVersion":"0.3.0",
@@ -4458,6 +4475,7 @@ rules:
   - apiGroups:
       - crd.antrea.io
     resources:
+      - bgppolicies
       - externalippools
       - ippools
       - trafficcontrols
@@ -4515,6 +4533,15 @@ rules:
       - create
       - patch
       - update
+  - apiGroups:
+      - ""
+    resources:
+      - secrets
+    resourceNames:
+      - antrea-bgp-passwords
+    verbs:
+      - get
+      - watch
 ---
 # Source: antrea/templates/antctl/clusterrole.yaml
 kind: ClusterRole
@@ -5123,7 +5150,7 @@ spec:
         kubectl.kubernetes.io/default-container: antrea-agent
         # Automatically restart Pods with a RollingUpdate if the ConfigMap changes
         # See https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
-        checksum/config: ba93df141f512a1f8483114b5994444c7231b298e7e9133483ddc1f4210ec395
+        checksum/config: 34609f1840d2af82d870efbc9b6f634474fbb5a11fda9a055552c55ff67fa3e4
         checksum/ipsec-secret: d0eb9c52d0cd4311b6d252a951126bf9bea27ec05590bed8a394f0f792dcb2a4
       labels:
         app: antrea
@@ -5405,7 +5432,7 @@ spec:
       annotations:
         # Automatically restart Pod if the ConfigMap changes
         # See https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
-        checksum/config: ba93df141f512a1f8483114b5994444c7231b298e7e9133483ddc1f4210ec395
+        checksum/config: 34609f1840d2af82d870efbc9b6f634474fbb5a11fda9a055552c55ff67fa3e4
       labels:
         app: antrea
         component: antrea-controller