[IPv6] Merge IPv6 branch to master (#1518)

* Enable verbose logging through generate-manifest.sh (#1142)

This patch enables verbose logging for antrea-agent and
antrea-controller when generate manifest. This will help in
troubleshooting with increased log level.

* Bug in PR#1142 (#1248)

manifest generation failing in release mode.
Fixed the bug.

* [IPv6] Consume Node.Spec.CIDRs to support dual-stack configuration (#971)

1. Consume Node.Spec.CIDRs to support IPv4/IPv6 dual-stack Pod Subnets
2. Change NodeConfig.PodCIDR as a slice
3. Change GatewayConfig.IP as a slice to support multiple addresses for antrea-gw0
4. Change InterfaceConfig.IP as a slice to support multiple address for a Pod

* [IPv6] Change openflow pipeline for L2 Pod networking (#1040)

1. Add a new table named IPv6 to handle IPv6 ND Solicitation,
ND advertisement and IPv6 Multicast traffic.

2. Add flows in openflow tables (spoofGuardTable, IPv6,
conntrackTable, conntrackStateTable, conntrackCommitTable,
L2ForwardingOutTable) for handling IPv6 L2 Pod networking.

* [IPv6] Change host-local IPAM configuration for IPv6 (#1039)

1. Add new field Ranges in IPAMConfig for allocating
   both IPv4 and IPv6 addresses.

2. Populate subnet and gateway for both IPv4 range and IPv6 range.

* [IPv6] Use separate fields for IPv4 and IPv6 in GatewayConfig (#1111)

Replace IP slice in GatewayConfig with separate IPv4
and IPv6 fields.

* [IPv6] Implement L3 connectivity for IPv6 traffic (#1011)

1. Use IPv6 in iptables and ipset configuration.
2. Identifiy IPv6 address and configure in OpenFlow.
3. Use Node Internal address for tunnel.

* [IPv6] Handle Spec.PodCIDR with IPv6 CIDR (#1151)

For IPv6 single stack case, node.Spec.PodCIDR is
configured with IPv6 CIDR. This patch handles the case
and sets nodeConfig.PodIPv6CIDR with parsed CIDR.

* [IPv6] Add support for IPv6 address in antctl and agent's apiserver (#1118)

* [IPv6] Add support for IPv6 address in antctl and agent's apiserver

1. Support using IPv6 address in OVS tracing.
2. Support displaying Node's and Pod's IPv6 address in agent apiserver.

Co-authored-by: Zhecheng Li <zhechel1@uci.edu>

* [IPv6] Support IPv6 in e2e (#1129)

* [IPv6] Display dual stack NodeSubnet in antrea-octant-plugin (#1156)

NodeSubnet can have two values in dual stack case and
this patch enhances octant-plugin to show both subnets.

* [IPv6] Handle dual stack NodeSubnet for monitoring CRD (#1182)

1. Rename NodeSubnet to NodeSubnets for AntreaAgentInfo.

2. Make a new string slice for dual stack node subnet instead of
appending agentInfo.NodeSubnets directly to avoid duplicate CIDRs.

* [IPv6][e2e] Fix testDeletePod (#1193)

On a dual-stack cluster, podInterfaces[0].IP returns "[ipv4-address], [ipv6-address]".
Current implementation doesn't distingush two.

* [IPv6] Collect service CIDR in e2e

* [IPv6] Add support for dual-stack when using kube-proxy for Service (#1200)

1. Add a config item for IPv6 Serivce CIDR if using kube-proxy to
   provide Service functions.
2. Output IPv6 traffic from host gateway if its destination is a
   Service address.
3. Use ct_mark to identify Service traffic and output the reply
   packet to the host gateway to ensure the DNAT processing in iptables.

* [IPv6] Extend e2e tests for dual-stack (#1192)

1. Extend generic functions "podWaitForIP" to return all assigned
   IPs of a given Pod.
2. Validate each IP address in the cluster's network CIDR
3. Use each valid IP to check connectivity.
4. Use each valid IP to execute tests.

* [IPv6] E2e bug fixes (#1311)

1. No -6 option in busybox nc
So, no need to distinguish if it is an IPv6 environment for runNetcatCommandFromTestPod()
nc
BusyBox v1.31.1 (2019-10-28 18:40:01 UTC) multi-call binary.

Usage: nc [OPTIONS] HOST PORT  - connect
nc [OPTIONS] -l -p PORT [HOST] [PORT]  - listen

	-e PROG	Run PROG after connect (must be last)
	-l	Listen mode, for inbound connects
	-lk	With -e, provides persistent server
	-p PORT	Local port
	-s ADDR	Local address
	-w SEC	Timeout for connects and final net reads
	-i SEC	Delay interval for lines sent
	-n	Don't do DNS resolution
	-u	UDP mode
	-v	Verbose
	-o FILE	Hex dump traffic
	-z	Zero-I/O mode (scanning)
2. testCert
* IPv6 address should be in "[]"

* [IPv6] Fix TestReconcileGatewayRoutesOnStartup failure (#1313)

Use "ip -6 route" for IPv6 network.

* [IPv6] adjust MTU for IPv6 overhead (#1305)

If Antrea MTU is too large in IPv6 environment, large packet with overhead exceeds node MTU cannot be transmitted successfully across nodes.
IPv6ExtraOverhead, 20 is from observation of IPv4 and IPv6 packets under same situation.

* [IPv6] Fix MTU config (#1317)

Use Node's internal address to decide if extra IPv6 overhead is needed.

* [IPv6] Skip IPsec e2e test (#1373)

* With OVS v2.14.0, IPsec in IPv6 envinronment is not supported.
* More user-friendly output for PodIPs.
From:
Retrieved all Pod IPs: map[test-pod-0-upgp1ung:0xc000708960 test-pod-1-pbva9007:0xc0006ec8a0]
To:
Retrieved all Pod IPs: map[test-pod-0-mudzj847:IPv6: fd74:ca9b:172:16::4, IP strings: fd74:ca9b:172:16::4 test-pod-1-apcmyd30:IPv6: fd74:ca9b:172:16:1::3c, IP strings: fd74:ca9b:172:16:1::3c]

* [IPv6] Add 2 Network Policy tests (#1399)

2 upstream Network Policy tests didn't consider netmask for IPv6, this patch
is to add correct tests. When bug is fixed in latest release, these 2 tests can be deleted.
Kubernetes PR:
kubernetes/kubernetes#93583
2 testcases:
https://github.com/kubernetes/kubernetes/blob/v1.20.0-alpha.0/test/e2e/network/network_policy.go#L1365
https://github.com/kubernetes/kubernetes/blob/v1.20.0-alpha.0/test/e2e/network/network_policy.go#L1444

* Skip 2 Network Policy testcases before Network Policy IPv6 is supported (#1460)

* [IPv6] Fix after rebasing
* format code
* fix TestPodTrafficShaping
* fix TestIPv6RoutesAndNeighbors

* [IPv6] Fix issues (#1496)

* unit test
* manifest

* [IPv6] Skip TestAntctlProxy for IPv6 (#1498)

* [IPv6] Add IPv6 support for NetworkPolicy

1. Add enhancement in Antrea Controller and Agent to support NetworkPolicy
   in IPv6.
2. Optimize test cases to support IPv6
3. Use regex in CRD to validate IPv4 or IPv6 string
4. Add TestEgressToServerInCIDRBlock and TestEgressToServerInCIDRBlockWithException
5. networkpolicy_controller.go: PodIPs includes PodIP

* [IPv6] Fix issues
* remove Github Actions integration test, Jenkins: jenkins-integration
-> Integration tests
* go fmt
* add FlowProtocl() to interface Flow
* remove extra lines when rebasing for an octant commit
* TestIPv6RoutesAndNeighbors: routeClient.Initialize

Co-authored-by: srikartati <stati@vmware.com>
Co-authored-by: Wenying Dong <wenyingd@vmware.com>
Co-authored-by: Mengdie Song <songm@vmware.com>
Co-authored-by: Zhecheng Li <zhechel1@uci.edu>

Makefile

@@ -132,6 +132,7 @@ docker-test-integration: .coverage
 		-v $(DOCKER_CACHE)/gocache:/tmp/gocache \
 		-v $(CURDIR)/.coverage:/usr/src/github.com/vmware-tanzu/antrea/.coverage \
 		-v $(CURDIR):/usr/src/github.com/vmware-tanzu/antrea:ro \
+		-v /lib/modules:/lib/modules \
 		antrea/test test-integration $(USERID) $(GRPID)
 
 .PHONY: docker-tidy

build/yamls/antrea-aks.yml

@@ -212,7 +212,7 @@ spec:
                 items:
                   properties:
                     ip:
-                      format: ipv4
+                      pattern: ^(((([1]?\d)?\d|2[0-4]\d|25[0-5])\.){3}(([1]?\d)?\d|2[0-4]\d|25[0-5]))|([\da-fA-F]{1,4}(\:[\da-fA-F]{1,4}){7})|(([\da-fA-F]{1,4}:){0,5}::([\da-fA-F]{1,4}:){0,5}[\da-fA-F]{1,4})$
                       type: string
                     name:
                       type: string
@@ -485,7 +485,7 @@ spec:
                   - ip
                 properties:
                   ip:
-                    format: ipv4
+                    pattern: ^(((([1]?\d)?\d|2[0-4]\d|25[0-5])\.){3}(([1]?\d)?\d|2[0-4]\d|25[0-5]))|([\da-fA-F]{1,4}(\:[\da-fA-F]{1,4}){7})|(([\da-fA-F]{1,4}:){0,5}::([\da-fA-F]{1,4}:){0,5}[\da-fA-F]{1,4})$
                     type: string
                   namespace:
                     type: string
@@ -503,7 +503,7 @@ spec:
                       protocol:
                         type: integer
                       srcIP:
-                        format: ipv4
+                        pattern: ^(((([1]?\d)?\d|2[0-4]\d|25[0-5])\.){3}(([1]?\d)?\d|2[0-4]\d|25[0-5]))|([\da-fA-F]{1,4}(\:[\da-fA-F]{1,4}){7})|(([\da-fA-F]{1,4}:){0,5}::([\da-fA-F]{1,4}:){0,5}[\da-fA-F]{1,4})$
                         type: string
                       ttl:
                         type: integer
@@ -1056,7 +1056,8 @@ data:
     featureGates:
     # Enable AntreaProxy which provides ServiceLB for in-cluster Services in antrea-agent.
     # It should be enabled on Windows, otherwise NetworkPolicy will not take effect on
-    # Service traffic.
+    # Service traffic. Antrea proxy doesn't support an IPv6 only cluster or a Dual-Stack cluster
+    # before PR #1102[https://github.com/vmware-tanzu/antrea/pull/1102] is merged.
       AntreaProxy: true
 
     # Enable traceflow which provides packet tracing feature to diagnose network issue.
@@ -1124,6 +1125,12 @@ data:
     # for the GRE tunnel type.
     #enableIPSecTunnel: false
 
+    # ClusterIP CIDR range for IPv6 Services. It's required when using kube-proxy to provide IPv6 Service in a Dual-Stack
+    # cluster or an IPv6 only cluster. The value should be the same as the configuration for kube-apiserver specified by
+    # --service-cluster-ip-range. When AntreaProxy is enabled, this parameter is not needed.
+    # No default value for this field.
+    #serviceCIDRv6:
+
     # The port for the antrea-agent APIServer to serve on.
     # Note that if it's set to another value, the `containerPort` of the `api` port of the
     # `antrea-agent` container must be set to the same value.
@@ -1202,7 +1209,7 @@ metadata:
   annotations: {}
   labels:
     app: antrea
-  name: antrea-config-2fm49f65tg
+  name: antrea-config-hmttgfbf78
   namespace: kube-system
 ---
 apiVersion: v1
@@ -1309,7 +1316,7 @@ spec:
         key: node-role.kubernetes.io/master
       volumes:
       - configMap:
-          name: antrea-config-2fm49f65tg
+          name: antrea-config-hmttgfbf78
         name: antrea-config
       - name: antrea-controller-tls
         secret:
@@ -1573,7 +1580,7 @@ spec:
         operator: Exists
       volumes:
       - configMap:
-          name: antrea-config-2fm49f65tg
+          name: antrea-config-hmttgfbf78
         name: antrea-config
       - hostPath:
           path: /etc/cni/net.d

build/yamls/antrea-eks.yml

@@ -212,7 +212,7 @@ spec:
                 items:
                   properties:
                     ip:
-                      format: ipv4
+                      pattern: ^(((([1]?\d)?\d|2[0-4]\d|25[0-5])\.){3}(([1]?\d)?\d|2[0-4]\d|25[0-5]))|([\da-fA-F]{1,4}(\:[\da-fA-F]{1,4}){7})|(([\da-fA-F]{1,4}:){0,5}::([\da-fA-F]{1,4}:){0,5}[\da-fA-F]{1,4})$
                       type: string
                     name:
                       type: string
@@ -485,7 +485,7 @@ spec:
                   - ip
                 properties:
                   ip:
-                    format: ipv4
+                    pattern: ^(((([1]?\d)?\d|2[0-4]\d|25[0-5])\.){3}(([1]?\d)?\d|2[0-4]\d|25[0-5]))|([\da-fA-F]{1,4}(\:[\da-fA-F]{1,4}){7})|(([\da-fA-F]{1,4}:){0,5}::([\da-fA-F]{1,4}:){0,5}[\da-fA-F]{1,4})$
                     type: string
                   namespace:
                     type: string
@@ -503,7 +503,7 @@ spec:
                       protocol:
                         type: integer
                       srcIP:
-                        format: ipv4
+                        pattern: ^(((([1]?\d)?\d|2[0-4]\d|25[0-5])\.){3}(([1]?\d)?\d|2[0-4]\d|25[0-5]))|([\da-fA-F]{1,4}(\:[\da-fA-F]{1,4}){7})|(([\da-fA-F]{1,4}:){0,5}::([\da-fA-F]{1,4}:){0,5}[\da-fA-F]{1,4})$
                         type: string
                       ttl:
                         type: integer
@@ -1056,7 +1056,8 @@ data:
     featureGates:
     # Enable AntreaProxy which provides ServiceLB for in-cluster Services in antrea-agent.
     # It should be enabled on Windows, otherwise NetworkPolicy will not take effect on
-    # Service traffic.
+    # Service traffic. Antrea proxy doesn't support an IPv6 only cluster or a Dual-Stack cluster
+    # before PR #1102[https://github.com/vmware-tanzu/antrea/pull/1102] is merged.
       AntreaProxy: true
 
     # Enable traceflow which provides packet tracing feature to diagnose network issue.
@@ -1124,6 +1125,12 @@ data:
     # for the GRE tunnel type.
     #enableIPSecTunnel: false
 
+    # ClusterIP CIDR range for IPv6 Services. It's required when using kube-proxy to provide IPv6 Service in a Dual-Stack
+    # cluster or an IPv6 only cluster. The value should be the same as the configuration for kube-apiserver specified by
+    # --service-cluster-ip-range. When AntreaProxy is enabled, this parameter is not needed.
+    # No default value for this field.
+    #serviceCIDRv6:
+
     # The port for the antrea-agent APIServer to serve on.
     # Note that if it's set to another value, the `containerPort` of the `api` port of the
     # `antrea-agent` container must be set to the same value.
@@ -1202,7 +1209,7 @@ metadata:
   annotations: {}
   labels:
     app: antrea
-  name: antrea-config-2fm49f65tg
+  name: antrea-config-hmttgfbf78
   namespace: kube-system
 ---
 apiVersion: v1
@@ -1309,7 +1316,7 @@ spec:
         key: node-role.kubernetes.io/master
       volumes:
       - configMap:
-          name: antrea-config-2fm49f65tg
+          name: antrea-config-hmttgfbf78
         name: antrea-config
       - name: antrea-controller-tls
         secret:
@@ -1575,7 +1582,7 @@ spec:
         operator: Exists
       volumes:
       - configMap:
-          name: antrea-config-2fm49f65tg
+          name: antrea-config-hmttgfbf78
         name: antrea-config
       - hostPath:
           path: /etc/cni/net.d

build/yamls/antrea-gke.yml

@@ -212,7 +212,7 @@ spec:
                 items:
                   properties:
                     ip:
-                      format: ipv4
+                      pattern: ^(((([1]?\d)?\d|2[0-4]\d|25[0-5])\.){3}(([1]?\d)?\d|2[0-4]\d|25[0-5]))|([\da-fA-F]{1,4}(\:[\da-fA-F]{1,4}){7})|(([\da-fA-F]{1,4}:){0,5}::([\da-fA-F]{1,4}:){0,5}[\da-fA-F]{1,4})$
                       type: string
                     name:
                       type: string
@@ -485,7 +485,7 @@ spec:
                   - ip
                 properties:
                   ip:
-                    format: ipv4
+                    pattern: ^(((([1]?\d)?\d|2[0-4]\d|25[0-5])\.){3}(([1]?\d)?\d|2[0-4]\d|25[0-5]))|([\da-fA-F]{1,4}(\:[\da-fA-F]{1,4}){7})|(([\da-fA-F]{1,4}:){0,5}::([\da-fA-F]{1,4}:){0,5}[\da-fA-F]{1,4})$
                     type: string
                   namespace:
                     type: string
@@ -503,7 +503,7 @@ spec:
                       protocol:
                         type: integer
                       srcIP:
-                        format: ipv4
+                        pattern: ^(((([1]?\d)?\d|2[0-4]\d|25[0-5])\.){3}(([1]?\d)?\d|2[0-4]\d|25[0-5]))|([\da-fA-F]{1,4}(\:[\da-fA-F]{1,4}){7})|(([\da-fA-F]{1,4}:){0,5}::([\da-fA-F]{1,4}:){0,5}[\da-fA-F]{1,4})$
                         type: string
                       ttl:
                         type: integer
@@ -1056,7 +1056,8 @@ data:
     featureGates:
     # Enable AntreaProxy which provides ServiceLB for in-cluster Services in antrea-agent.
     # It should be enabled on Windows, otherwise NetworkPolicy will not take effect on
-    # Service traffic.
+    # Service traffic. Antrea proxy doesn't support an IPv6 only cluster or a Dual-Stack cluster
+    # before PR #1102[https://github.com/vmware-tanzu/antrea/pull/1102] is merged.
       AntreaProxy: true
 
     # Enable traceflow which provides packet tracing feature to diagnose network issue.
@@ -1124,6 +1125,12 @@ data:
     # for the GRE tunnel type.
     #enableIPSecTunnel: false
 
+    # ClusterIP CIDR range for IPv6 Services. It's required when using kube-proxy to provide IPv6 Service in a Dual-Stack
+    # cluster or an IPv6 only cluster. The value should be the same as the configuration for kube-apiserver specified by
+    # --service-cluster-ip-range. When AntreaProxy is enabled, this parameter is not needed.
+    # No default value for this field.
+    #serviceCIDRv6:
+
     # The port for the antrea-agent APIServer to serve on.
     # Note that if it's set to another value, the `containerPort` of the `api` port of the
     # `antrea-agent` container must be set to the same value.
@@ -1202,7 +1209,7 @@ metadata:
   annotations: {}
   labels:
     app: antrea
-  name: antrea-config-m742b78ct4
+  name: antrea-config-8bc4m9g22g
   namespace: kube-system
 ---
 apiVersion: v1
@@ -1309,7 +1316,7 @@ spec:
         key: node-role.kubernetes.io/master
       volumes:
       - configMap:
-          name: antrea-config-m742b78ct4
+          name: antrea-config-8bc4m9g22g
         name: antrea-config
       - name: antrea-controller-tls
         secret:
@@ -1573,7 +1580,7 @@ spec:
         operator: Exists
       volumes:
       - configMap:
-          name: antrea-config-m742b78ct4
+          name: antrea-config-8bc4m9g22g
         name: antrea-config
       - hostPath:
           path: /etc/cni/net.d

build/yamls/antrea-ipsec.yml

@@ -212,7 +212,7 @@ spec:
                 items:
                   properties:
                     ip:
-                      format: ipv4
+                      pattern: ^(((([1]?\d)?\d|2[0-4]\d|25[0-5])\.){3}(([1]?\d)?\d|2[0-4]\d|25[0-5]))|([\da-fA-F]{1,4}(\:[\da-fA-F]{1,4}){7})|(([\da-fA-F]{1,4}:){0,5}::([\da-fA-F]{1,4}:){0,5}[\da-fA-F]{1,4})$
                       type: string
                     name:
                       type: string
@@ -485,7 +485,7 @@ spec:
                   - ip
                 properties:
                   ip:
-                    format: ipv4
+                    pattern: ^(((([1]?\d)?\d|2[0-4]\d|25[0-5])\.){3}(([1]?\d)?\d|2[0-4]\d|25[0-5]))|([\da-fA-F]{1,4}(\:[\da-fA-F]{1,4}){7})|(([\da-fA-F]{1,4}:){0,5}::([\da-fA-F]{1,4}:){0,5}[\da-fA-F]{1,4})$
                     type: string
                   namespace:
                     type: string
@@ -503,7 +503,7 @@ spec:
                       protocol:
                         type: integer
                       srcIP:
-                        format: ipv4
+                        pattern: ^(((([1]?\d)?\d|2[0-4]\d|25[0-5])\.){3}(([1]?\d)?\d|2[0-4]\d|25[0-5]))|([\da-fA-F]{1,4}(\:[\da-fA-F]{1,4}){7})|(([\da-fA-F]{1,4}:){0,5}::([\da-fA-F]{1,4}:){0,5}[\da-fA-F]{1,4})$
                         type: string
                       ttl:
                         type: integer
@@ -1056,7 +1056,8 @@ data:
     featureGates:
     # Enable AntreaProxy which provides ServiceLB for in-cluster Services in antrea-agent.
     # It should be enabled on Windows, otherwise NetworkPolicy will not take effect on
-    # Service traffic.
+    # Service traffic. Antrea proxy doesn't support an IPv6 only cluster or a Dual-Stack cluster
+    # before PR #1102[https://github.com/vmware-tanzu/antrea/pull/1102] is merged.
     #  AntreaProxy: false
 
     # Enable traceflow which provides packet tracing feature to diagnose network issue.
@@ -1129,6 +1130,12 @@ data:
     # AntreaProxy is enabled, this parameter is not needed and will be ignored if provided.
     #serviceCIDR: 10.96.0.0/12
 
+    # ClusterIP CIDR range for IPv6 Services. It's required when using kube-proxy to provide IPv6 Service in a Dual-Stack
+    # cluster or an IPv6 only cluster. The value should be the same as the configuration for kube-apiserver specified by
+    # --service-cluster-ip-range. When AntreaProxy is enabled, this parameter is not needed.
+    # No default value for this field.
+    #serviceCIDRv6:
+
     # The port for the antrea-agent APIServer to serve on.
     # Note that if it's set to another value, the `containerPort` of the `api` port of the
     # `antrea-agent` container must be set to the same value.
@@ -1207,7 +1214,7 @@ metadata:
   annotations: {}
   labels:
     app: antrea
-  name: antrea-config-2gdh849dhk
+  name: antrea-config-kgd27dftgd
   namespace: kube-system
 ---
 apiVersion: v1
@@ -1323,7 +1330,7 @@ spec:
         key: node-role.kubernetes.io/master
       volumes:
       - configMap:
-          name: antrea-config-2gdh849dhk
+          name: antrea-config-kgd27dftgd
         name: antrea-config
       - name: antrea-controller-tls
         secret:
@@ -1622,7 +1629,7 @@ spec:
         operator: Exists
       volumes:
       - configMap:
-          name: antrea-config-2gdh849dhk
+          name: antrea-config-kgd27dftgd
         name: antrea-config
       - hostPath:
           path: /etc/cni/net.d