Add support for NodeNetworkPolicy datapath #5658

hongliangl · 2023-11-02T12:15:43Z

This PR introduces support for the NodeNetworkPolicy datapath, extending Antrea
ClusterNetworkPolicy (ACNP). The implementation leverages iptables and ipset for
enforcing rules, safeguarding Kubernetes Nodes.

There are four key components to implement the data path:

Core iptables rule
- Integrated into static chains ANTREA-POL-INGRESS-RULES (ingress) or
  ANTREA-POL-EGRESS-RULES (egress).
- Matches an ipset that includes NodeNetworkPolicy rule source or
  destination IPs, or directly matches a single IP.
- Targets an action or a service chain created for NodeNetworkPolicy
  rule with multiple services.
Service iptables chain
- Created for NodeNetworkPolicy rule with multiple services.
Service iptables rules:
- Added to the service chain for NodeNetworkPolicy rule, constructed from
  rule services.
From/To ipset:
- Created for a NodeNetworkPolicy rule, containing source (ingress) or
  destination (egress) IPs.

Example ingress or egress core iptables rules organized by priorities:

:ANTREA-POL-INGRESS-RULES
-A ANTREA-POL-INGRESS-RULES -m set --match-set ANTREA-POL-RULE1-4 src -j ANTREA-POL-RULE1 -m comment --comment "Antrea: for rule RULE1, policy AntreaClusterNetworkPolicy:name1"
-A ANTREA-POL-INGRESS-RULES -m set --match-set ANTREA-POL-RULE2-4 src -p tcp --dport 8080 -j ACCEPT -m comment --comment "Antrea: for rule RULE2, policy AntreaClusterNetworkPolicy:name2"
-A ANTREA-POL-INGRESS-RULES -s 3.3.3.3/32 src -j ANTREA-POL-RULE3 -m comment --comment "Antrea: for rule RULE3, policy AntreaClusterNetworkPolicy:name3"
-A ANTREA-POL-INGRESS-RULES -s 4.4.4.4/32 -p tcp --dport 80 -j ACCEPT -m comment --comment "Antrea: for rule RULE4, policy AntreaClusterNetworkPolicy:name4"

Example service chain (for rule with multiple services)::

:ANTREA-POL-RULE1
-A ANTREA-POL-RULE1 -j ACCEPT -p tcp --dport 80
-A ANTREA-POL-RULE1 -j ACCEPT -p tcp --dport 443

Example ipset (for rule with multiple source or destination IPs)

Name: ANTREA-POL-RULE1-4
Type: hash:net
Revision: 6
Header: family inet hashsize 1024 maxelem 65536
Size in memory: 472
References: 1
Number of entries: 2
Members:
1.1.1.1
1.1.1.2

Signed-off-by: Hongliang Liu lhongliang@vmware.com

hack/update-checksum-windows.sh

docs/antrea-node-network-policy.md

Dyanngg · 2024-01-04T22:37:36Z

docs/antrea-node-network-policy.md

+          port: 22
+```
+
+## Limitations


General question: should we add more clarifications in the limitations section in terms of what it means to have a ipBlock specified in the ingress block of a node networkpolicy? This could cause a great deal of confusion for users. For example, consider a multi-cluster setup with non-overlapping CIDRs, pod-to-pod connectivity enabled with gateways. In case the node where the policy is applied to is not the gateway node, wouldn't the ingress ipBlock feature not work properly due to the SNAT at the gateway (if the user specify CIDRs from the other clusters)? In addition, out of cluster traffic tend to have similar issues due SNATs at cloud load balancers before they enter Nodes? @tnqn

This is not specific to Node networkpolicy. Even for Pod networkpolicy, SNAT can happen in many cases (lke NodePort traffic), I feel it's not too hard for users to get that if your traffic gets SNATed before reaching the endpoint, the policy will be enforced based on the translated IP. But adding a note to explain the SNAT and IPBlock stuff in https://github.com/antrea-io/antrea/blob/main/docs/antrea-network-policy.md#notes-and-constraints sounds good to me.

docs/antrea-node-network-policy.md

docs/feature-gates.md

pkg/agent/controller/networkpolicy/networkpolicy_controller.go

pkg/agent/controller/networkpolicy/node_reconciler_linux.go

pkg/agent/route/route_linux.go

luolanzone

LGTM overall, a few nits.

docs/antrea-node-network-policy.md

pkg/agent/controller/networkpolicy/networkpolicy_controller.go

pkg/agent/controller/networkpolicy/node_reconciler_linux.go

luolanzone · 2024-01-09T07:02:45Z

@hongliangl please help to update the PR summary to explain this PR.

hongliangl · 2024-01-09T07:51:20Z

@hongliangl please help to update the PR summary to explain this PR.

Updated

docs/feature-gates.md

pkg/agent/controller/networkpolicy/node_reconciler_linux.go

pkg/agent/route/route_linux.go

pkg/features/antrea_features.go

hongliangl · 2024-01-10T02:35:15Z

/test-all

This PR introduces support for the NodeNetworkPolicy datapath, extending Antrea ClusterNetworkPolicy (ACNP). The implementation leverages iptables and ipset for enforcing rules, safeguarding Kubernetes Nodes. There are four key components to implement the data path: - Core iptables rule - Integrated into static chains ANTREA-POL-INGRESS-RULES (ingress) or ANTREA-POL-EGRESS-RULES (egress). - Matches an ipset that includes NodeNetworkPolicy rule source or destination IPs, or directly matches a single IP. - Targets an action or a service chain created for NodeNetworkPolicy rule with multiple services. - Service iptables chain - Created for NodeNetworkPolicy rule with multiple services. - Service iptables rules: - Added to the service chain for NodeNetworkPolicy rule, constructed from rule services. - From/To ipset: - Created for a NodeNetworkPolicy rule, containing source (ingress) or destination (egress) IPs. Example ingress or egress core iptables rules organized by priorities: ``` :ANTREA-POL-INGRESS-RULES -A ANTREA-POL-INGRESS-RULES -m set --match-set ANTREA-POL-RULE1-4 src -j ANTREA-POL-RULE1 -m comment --comment "Antrea: for rule RULE1, policy AntreaClusterNetworkPolicy:name1" -A ANTREA-POL-INGRESS-RULES -m set --match-set ANTREA-POL-RULE2-4 src -p tcp --dport 8080 -j ACCEPT -m comment --comment "Antrea: for rule RULE2, policy AntreaClusterNetworkPolicy:name2" -A ANTREA-POL-INGRESS-RULES -s 3.3.3.3/32 src -j ANTREA-POL-RULE3 -m comment --comment "Antrea: for rule RULE3, policy AntreaClusterNetworkPolicy:name3" -A ANTREA-POL-INGRESS-RULES -s 4.4.4.4/32 -p tcp --dport 80 -j ACCEPT -m comment --comment "Antrea: for rule RULE4, policy AntreaClusterNetworkPolicy:name4" ``` Example service chain (for rule with multiple services):: ``` :ANTREA-POL-RULE1 -A ANTREA-POL-RULE1 -j ACCEPT -p tcp --dport 80 -A ANTREA-POL-RULE1 -j ACCEPT -p tcp --dport 443 ``` Example ipset (for rule with multiple source or destination IPs) ``` Name: ANTREA-POL-RULE1-4 Type: hash:net Revision: 6 Header: family inet hashsize 1024 maxelem 65536 Size in memory: 472 References: 1 Number of entries: 2 Members: 1.1.1.1 1.1.1.2 ``` Signed-off-by: Hongliang Liu <lhongliang@vmware.com>

tnqn

LGTM

@jianjuns @luolanzone do you have other comments?

hongliangl · 2024-01-10T08:26:38Z

/test-all

jianjuns · 2024-01-10T18:56:26Z

I have no extra comment.

hongliangl · 2024-01-12T02:39:34Z

/test-ipv6-all

hongliangl · 2024-01-12T04:43:52Z

/test-conformance

hongliangl force-pushed the 20230828-bm-dfw-reboot branch from 4dd92cb to ed61f70 Compare November 2, 2023 16:12

hongliangl changed the title ~~[WIP] Add support for HostNetworkPolicy~~ [WIP] Add support for HostNetworkPolicy datapath Nov 3, 2023

hongliangl force-pushed the 20230828-bm-dfw-reboot branch 3 times, most recently from 621e5a0 to fa67a44 Compare November 9, 2023 11:42

hongliangl force-pushed the 20230828-bm-dfw-reboot branch 13 times, most recently from 3a18e52 to 2b2307f Compare November 15, 2023 08:04

hongliangl marked this pull request as ready for review November 15, 2023 08:04

hongliangl force-pushed the 20230828-bm-dfw-reboot branch from 2b2307f to fafbd00 Compare November 15, 2023 09:06

hongliangl changed the title ~~[WIP] Add support for HostNetworkPolicy datapath~~ Add support for HostNetworkPolicy datapath Nov 15, 2023

hongliangl commented Nov 15, 2023

View reviewed changes

hack/update-checksum-windows.sh Outdated Show resolved Hide resolved

hongliangl force-pushed the 20230828-bm-dfw-reboot branch from fafbd00 to a69ef99 Compare November 15, 2023 09:45

hongliangl changed the title ~~Add support for HostNetworkPolicy datapath~~ Add support for NodeNetworkPolicy datapath Nov 15, 2023

hongliangl requested review from tnqn, luolanzone, Atish-iaf and rajnkamr November 15, 2023 12:30

hongliangl mentioned this pull request Nov 15, 2023

Support applying ClusterNetworkPolicy to Nodes in Antrea #5671

Closed

hongliangl force-pushed the 20230828-bm-dfw-reboot branch from a69ef99 to 5c3b084 Compare November 16, 2023 07:06

jianjuns reviewed Jan 4, 2024

View reviewed changes

docs/antrea-node-network-policy.md Outdated Show resolved Hide resolved

docs/antrea-node-network-policy.md Outdated Show resolved Hide resolved

docs/antrea-node-network-policy.md Outdated Show resolved Hide resolved

docs/antrea-node-network-policy.md Outdated Show resolved Hide resolved

Dyanngg reviewed Jan 4, 2024

View reviewed changes

hongliangl force-pushed the 20230828-bm-dfw-reboot branch 2 times, most recently from c5a8f07 to d0f3bd5 Compare January 5, 2024 10:31

jianjuns reviewed Jan 5, 2024

View reviewed changes

docs/antrea-node-network-policy.md Outdated Show resolved Hide resolved

docs/antrea-node-network-policy.md Outdated Show resolved Hide resolved

docs/antrea-node-network-policy.md Outdated Show resolved Hide resolved

docs/antrea-node-network-policy.md Outdated Show resolved Hide resolved

hongliangl force-pushed the 20230828-bm-dfw-reboot branch 4 times, most recently from 912aa68 to 795f130 Compare January 8, 2024 07:26

hongliangl requested review from tnqn and jianjuns January 8, 2024 07:44

tnqn reviewed Jan 9, 2024

View reviewed changes

hongliangl force-pushed the 20230828-bm-dfw-reboot branch from 795f130 to 783ef52 Compare January 9, 2024 05:58

luolanzone reviewed Jan 9, 2024

View reviewed changes

tnqn reviewed Jan 9, 2024

View reviewed changes

docs/feature-gates.md Outdated Show resolved Hide resolved

pkg/agent/controller/networkpolicy/node_reconciler_linux.go Outdated Show resolved Hide resolved

pkg/agent/route/route_linux.go Outdated Show resolved Hide resolved

pkg/features/antrea_features.go Outdated Show resolved Hide resolved

hongliangl force-pushed the 20230828-bm-dfw-reboot branch from 45d69c4 to 471948c Compare January 9, 2024 12:09

hongliangl force-pushed the 20230828-bm-dfw-reboot branch from 471948c to 32e091b Compare January 10, 2024 03:51

tnqn approved these changes Jan 10, 2024

View reviewed changes

luolanzone approved these changes Jan 12, 2024

View reviewed changes

tnqn merged commit 4eb1de4 into antrea-io:main Jan 12, 2024
51 of 54 checks passed

hongliangl deleted the 20230828-bm-dfw-reboot branch January 12, 2024 09:42

ColonelBundy mentioned this pull request Feb 23, 2024

Control node traffic through network policies #4213

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for NodeNetworkPolicy datapath #5658

Add support for NodeNetworkPolicy datapath #5658

hongliangl commented Nov 2, 2023 •

edited

Dyanngg Jan 4, 2024 •

edited

tnqn Jan 5, 2024

luolanzone left a comment

luolanzone commented Jan 9, 2024

hongliangl commented Jan 9, 2024

hongliangl commented Jan 10, 2024

tnqn left a comment

hongliangl commented Jan 10, 2024

jianjuns commented Jan 10, 2024

hongliangl commented Jan 12, 2024

hongliangl commented Jan 12, 2024

Add support for NodeNetworkPolicy datapath #5658

Add support for NodeNetworkPolicy datapath #5658

Conversation

hongliangl commented Nov 2, 2023 • edited

Dyanngg Jan 4, 2024 • edited

Choose a reason for hiding this comment

tnqn Jan 5, 2024

Choose a reason for hiding this comment

luolanzone left a comment

Choose a reason for hiding this comment

luolanzone commented Jan 9, 2024

hongliangl commented Jan 9, 2024

hongliangl commented Jan 10, 2024

tnqn left a comment

Choose a reason for hiding this comment

hongliangl commented Jan 10, 2024

jianjuns commented Jan 10, 2024

hongliangl commented Jan 12, 2024

hongliangl commented Jan 12, 2024

hongliangl commented Nov 2, 2023 •

edited

Dyanngg Jan 4, 2024 •

edited