Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Node Selector in ACNP/ANP ingress/egress rules #3038

Merged
merged 2 commits into from Mar 24, 2022

Conversation

wenqiq
Copy link
Contributor

@wenqiq wenqiq commented Nov 19, 2021

Support nodeSelector in ACNP/ANP ingress/egress rules.

  1. Add nodeSelector field in ACNP/ANP ingress/egress rules to restrict traffic from/to particular Nodes.

  2. Add Node info in the output of antctl get addressgroup.
    For example:

antctl get addressgroup       
NAME                                 POD-IPS                                NODE-IPS                  
5e1bbf8e-67a3-5384-b2bf-e42e93bd68aa <NONE>                                 192.168.0.1, 10.176.27.105
f8c56571-d6db-51ec-9352-a9a47476a9a0 192.168.0.70,192.168.1.38,192.168.1.39 <NONE>  

antctl get addressgroup -oyaml
- name: 5e1bbf8e-67a3-5384-b2bf-e42e93bd68aa
  nodes:
  - ip: 192.168.0.1, 10.176.27.105
    node:
      name: wenqiq01-1
- name: f8c56571-d6db-51ec-9352-a9a47476a9a0
  pods:
  - ip: 192.168.1.38
    pod:
      name: iperf3-55bcff667d-v495l
      namespace: demo
  - ip: 192.168.1.39
    pod:
      name: iperf3-55bcff667d-c74ll
      namespace: demo
  - ip: 192.168.0.70
    pod:
      name: iperf3-55bcff667d-pz2d2
      namespace: demo

Fixes: #3023

Signed-off-by: Wenqi Qiu wenqiq@vmware.com

@codecov-commenter
Copy link

codecov-commenter commented Nov 19, 2021

Codecov Report

Merging #3038 (4b06963) into main (c8a3544) will decrease coverage by 3.75%.
The diff coverage is 70.09%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #3038      +/-   ##
==========================================
- Coverage   65.75%   62.00%   -3.76%     
==========================================
  Files         277      277              
  Lines       27264    37635   +10371     
==========================================
+ Hits        17927    23334    +5407     
- Misses       7439    12363    +4924     
- Partials     1898     1938      +40     
Flag Coverage Δ
e2e-tests 54.26% <55.60%> (?)
kind-e2e-tests 40.11% <8.49%> (-15.99%) ⬇️
unit-tests 43.19% <66.43%> (+0.10%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...ntroller/networkpolicy/networkpolicy_controller.go 68.20% <ø> (-3.45%) ⬇️
...gent/controller/noderoute/node_route_controller.go 57.69% <0.00%> (+1.95%) ⬆️
pkg/antctl/transform/common/transform.go 0.00% <0.00%> (ø)
pkg/apis/controlplane/types.go 100.00% <ø> (ø)
pkg/controller/networkpolicy/validate.go 73.88% <25.00%> (-6.74%) ⬇️
...g/controller/networkpolicy/clusternetworkpolicy.go 76.47% <35.00%> (-4.83%) ⬇️
pkg/util/k8s/node.go 76.28% <81.15%> (+11.28%) ⬆️
pkg/controller/types/group.go 85.00% <92.30%> (+4.23%) ⬆️
...ntroller/networkpolicy/networkpolicy_controller.go 80.63% <97.67%> (-3.41%) ⬇️
pkg/controller/egress/controller.go 62.19% <100.00%> (-26.26%) ⬇️
... and 250 more

@@ -526,6 +526,9 @@ since Pod IPs are ephemeral and unpredictable.
select Fully Qualified Domain Names (FQDNs), specified either by exact name or wildcard
expressions, when defining `egress` rules.

**nodeSelector**: This selects certain Node IPs as ingress from address or egress to address.
It is applicable only to the `from` section in an `ingress` block or the `to` section in an `egress` block.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please refer to the existing doc for podSelector and namespaceSelector fields

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated.

@@ -1062,6 +1065,37 @@ spec:
- fqdn: "svcA.default.svc.cluster.local"
```

## Node Selector

NodeSelector selects certain Nodes which match the label selector. Add Node IPs to address group memberSet.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
NodeSelector selects certain Nodes which match the label selector. Add Node IPs to address group memberSet.
NodeSelector selects certain Nodes which match the label selector. It adds Node IPs to address group memberSet.

And this is user facing doc, I'm not sure if users can understand "address group memberSet". At least there is no other fields mentioning it. Could you aligh with the description of other similar fields?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated.

## Node Selector

NodeSelector selects certain Nodes which match the label selector. Add Node IPs to address group memberSet.
The following rule applied to Pods with label `app=antrea-test-app` and will `Drop` egress traffic to
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The following rule applied to Pods with label `app=antrea-test-app` and will `Drop` egress traffic to
The following rule applies to Pods with label `app=antrea-test-app` and will `Drop` egress traffic to

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.


NodeSelector selects certain Nodes which match the label selector. Add Node IPs to address group memberSet.
The following rule applied to Pods with label `app=antrea-test-app` and will `Drop` egress traffic to
Nodes which has labels `kubernetes.io/hostname=kind-control-plane`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Nodes which has labels `kubernetes.io/hostname=kind-control-plane`.
Nodes which have the labels `kubernetes.io/hostname=kind-control-plane`.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

docs/antrea-network-policy.md Show resolved Hide resolved
- key: kubernetes.io/hostname
operator: In
values:
- kind-control-plane
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe a more proper example for your use case is:

matchExpressions:
- key: node-role.kubernetes.io/control-plane
  operator: Exists

as hostname can only match a single host.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@@ -15,6 +15,7 @@
package v1alpha1

import (
_ "github.com/golang/mock/mockgen/model"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this change?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Forget to delete this. It is used in make codegen. Done.

@@ -434,6 +435,11 @@ type NetworkPolicyPeer struct {
// Exact FQDNs, i.e. "google.com", "db-svc.default.svc.cluster.local"
// Wildcard expressions, i.e. "*wayfair.com".
FQDN string `json:"fqdn,omitempty"`
// Select certain Nodes which match the label selector,
// if no nodeSelector is specified, then all Nodes will be selected
// in the cluster.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this behavior is expected. It basically breaks compaibility of existing rules.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated.

@@ -538,6 +539,10 @@ with `namespaces` field.
specifies both namespaceSelector and podSelector selects particular Pods within
particular Namespaces.

**nodeSelector**: This selects particular Nodes in cluster.
The selected Node's IPs will set as "sources" if `nodeSelector` set in `ingress` section, or as "destinations" if set in
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you align this lines?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

pkg/apis/crd/v1alpha1/types.go Outdated Show resolved Hide resolved
for _, addressGroupObj := range addressGroupObjs {
addressGroup := addressGroupObj.(*antreatypes.AddressGroup)
nS := addressGroup.Selector.NodeSelector
if nS == nil && !addressGroup.SpanMeta.Has(node.Name) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why the addressGroup's span must contain this Node?
For example, if an addressGroup which contains Node A is applied to a Pod running on Node B, the addressGroup's span should contain only Node B.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated.

pkg/controller/networkpolicy/clusternetworkpolicy.go Outdated Show resolved Hide resolved
pkg/controller/networkpolicy/clusternetworkpolicy.go Outdated Show resolved Hide resolved
pkg/controller/networkpolicy/networkpolicy_controller.go Outdated Show resolved Hide resolved
pkg/controller/types/group.go Show resolved Hide resolved
groupSelector.NamespaceSelector, _ = metav1.LabelSelectorAsSelector(nsSelector)
}

if len(nodeSelectors) > 0 && nodeSelectors[0] != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a proof why variadic function doesn't apply here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated.

}

if len(nodeSelectors) > 0 && nodeSelectors[0] != nil {
groupSelector.NodeSelector, _ = metav1.LabelSelectorAsSelector(nodeSelectors[0])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think when nodeSelector is present, no other fields including "Namespace" should be set to avoid ambiguity

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated.

// the following format: "namespace=NamespaceName And podSelector=normalizedPodSelector".
// Note: Namespace and nsSelector may or may not be set depending on the
// selector. However, they cannot be set simultaneously.
func generateNormalizedName(namespace string, podSelector, nsSelector, eeSelector labels.Selector) string {
func GenerateNormalizedName(namespace string, podSelector, nsSelector, eeSelector labels.Selector, nodeSelectors ...labels.Selector) string {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@wenqiq wenqiq force-pushed the anp-nodeselector branch 3 times, most recently from 01279b8 to 67cc539 Compare February 21, 2022 06:34
@wenqiq wenqiq force-pushed the anp-nodeselector branch 3 times, most recently from 6f3842f to 275530d Compare March 3, 2022 15:46
NodeSelector selects certain Nodes which match the label selector. It adds Node IPs to egress rules in `to` field
or ingress rules in `from` filed.
The following rule applies to Pods with label `app=antrea-test-app` and will `Drop` egress traffic to
Nodes which have the labels `node-role.kubernetes.io/control-plane`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: on TCP port 6443

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Comment on lines 307 to 311
oldIPs, err := k8s.GetNodeAddrs(oldNode)
if err != nil {
return false, err
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
oldIPs, err := k8s.GetNodeAddrs(oldNode)
if err != nil {
return false, err
}
if oldIPs, err := k8s.GetNodeAddrs(oldNode); err != nil {
return false, err
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated.

@@ -115,10 +115,14 @@ func (n *NetworkPolicyController) toAntreaPeerForCRD(peers []v1alpha1.NetworkPol
} else if peer.FQDN != "" {
fqdns = append(fqdns, peer.FQDN)
} else if peer.ServiceAccount != nil {
normalizedUID := n.createAddressGroup(peer.ServiceAccount.Namespace, serviceAccountNameToPodSelector(peer.ServiceAccount.Name), nil, nil)
normalizedUID := n.createAddressGroup(peer.ServiceAccount.Namespace, serviceAccountNameToPodSelector(peer.ServiceAccount.Name), nil, nil, nil)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not entirely related to your PR but comment on L98 needs update

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated.

Name: name,
AppliedTo: appliedTos,
}
b.Spec.Egress = append(b.Spec.Egress, newRule)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is nodeSelector only supported for egress?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for reviewing. Both egress and ingress rules, and will add more e2e test cases.

@wenqiq wenqiq force-pushed the anp-nodeselector branch 2 times, most recently from bbf96b4 to 90e87b5 Compare March 9, 2022 10:51
@tnqn tnqn added this to the Antrea v1.6 release milestone Mar 9, 2022
@tnqn tnqn added the action/release-note Indicates a PR that should be included in release notes. label Mar 9, 2022
docs/antrea-network-policy.md Outdated Show resolved Hide resolved
@@ -1102,6 +1107,35 @@ spec:
- fqdn: "svcA.default.svc.cluster.local"
```

## Node Selector

NodeSelector selects certain Nodes which match the label selector. It adds Node IPs to egress rules in `to` field
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about:

When used in the to field of an egress rule, it adds the Node IPs to the rule's destination address group; when used in the from field of an ingress rule, it adds the Node IPs to the rule's source address group.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated.

@@ -545,6 +546,10 @@ with `namespaces` field.
specifies both namespaceSelector and podSelector selects particular Pods within
particular Namespaces.

**nodeSelector**: This selects particular Nodes in cluster. The selected Node's
IPs will set as "sources" if `nodeSelector` set in `ingress` section, or as
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will set as -> will be used as

set in -> is set in the

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@@ -439,6 +439,12 @@ type NetworkPolicyPeer struct {
// Cannot be set with any other selector.
// +optional
ServiceAccount *NamespacedName `json:"serviceAccount,omitempty"`
// Select certain Nodes which match the label selector,
// if no nodeSelector is specified, then no additional Nodes IPs will be added
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel we can remove this sentence (line 443 - 444), as it does add useful information.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, done.

@@ -439,6 +439,12 @@ type NetworkPolicyPeer struct {
// Cannot be set with any other selector.
// +optional
ServiceAccount *NamespacedName `json:"serviceAccount,omitempty"`
// Select certain Nodes which match the label selector,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

, -> .

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@@ -49,27 +49,36 @@ type GroupSelector struct {
// If Namespace and NamespaceSelector both are unset, it selects the ExternalEntities in all the Namespaces.
// TODO: Add validation in API to not allow externalEntitySelector and podSelector in the same group.
ExternalEntitySelector labels.Selector

// This is a label selector which selects certain Node IPs. Within a group NodeSelector cannot be set concurrently with
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

concurrently -> together

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@@ -57,3 +58,26 @@ func GetNodeAddrs(node *v1.Node) (*ip.DualStackIPs, error) {
}
return nodeAddrs, nil
}

// GetNodeAddressFromAnnotations gets available IPs from the Node Annotation, the annotations are set by Antrea, includes
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Node Annotation, the -> Node Annotation. The

includes -> including

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@@ -641,6 +641,20 @@ func (i *Initializer) setupGatewayInterface() error {
return err
}

// Update the Node Antrea gateway IP address in Node's annotation.
gwIPv4Addr, gwIPv6Addr := i.nodeConfig.GatewayConfig.IPv4, i.nodeConfig.GatewayConfig.IPv6
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I remember we reach a consensus in the community meeting that we could just "predict" the gateway IPs as some implementation already does it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated.

pkg/controller/networkpolicy/clusternetworkpolicy.go Outdated Show resolved Hide resolved
pkg/controller/networkpolicy/clusternetworkpolicy.go Outdated Show resolved Hide resolved
pkg/controller/networkpolicy/clusternetworkpolicy.go Outdated Show resolved Hide resolved
}
if newIPs, err = k8s.GetNodeAddrs(newNode); err != nil {
return
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should return early if change is already identified.
And I think if there is an error, we should return changed as true. Imagine this scenario:
A Node doesn't have a valid Node IP because of a bug or it's not reported yet, after it gets an valid IP, why we don't reconcile the policy with the new IP?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Updated.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Second thought, which might make more sense:
We don't need to care whether there is an error when getting oldIPs and newIPs. We should just focus on whether there is a change. If both get error, oldIPs and newIPs are both nil, there is no point to reconcile the rule as well. So the code could just be:

oldIPs, _ = k8s.GetNodeAddrs(oldNode)
newIPs, _ = k8s.GetNodeAddrs(newNode)
!reflect.DeepEqual(newIPs, oldIPs) {
    return true
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated.

pkg/controller/networkpolicy/crd_utils.go Outdated Show resolved Hide resolved
@@ -1091,18 +1061,36 @@ func (n *NetworkPolicyController) syncAddressGroup(key string) error {
utilsets.MergeString(addrGroupNodeNames, internalNP.SpanMeta.NodeNames)
}
memberSet := n.getAddressGroupMemberSet(addressGroup)
if addressGroup.Selector.NodeSelector != nil {
ms, nodes := n.addNodeSelectorMemberSet(addressGroup.Selector.NodeSelector)
memberSet = memberSet.Union(ms)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, I think the API is already designed in this way. You should just update getAddressGroupMemberSet to get node IPs when NodeSelector is not nil, instead of still trying to get pod members first.

pkg/controller/networkpolicy/networkpolicy_controller.go Outdated Show resolved Hide resolved
pkg/controller/networkpolicy/networkpolicy_controller.go Outdated Show resolved Hide resolved
@wenqiq
Copy link
Contributor Author

wenqiq commented Mar 15, 2022

/test-all

@wenqiq
Copy link
Contributor Author

wenqiq commented Mar 15, 2022

/test-ipv6-all
/test-ipv6-only-all
/test-windows-all

@wenqiq wenqiq force-pushed the anp-nodeselector branch 2 times, most recently from 275530d to 8a014a0 Compare March 18, 2022 07:34
@tnqn
Copy link
Member

tnqn commented Mar 21, 2022

The PR changes 171 files, please revert unrelated changes

@wenqiq
Copy link
Contributor Author

wenqiq commented Mar 21, 2022

The PR changes 171 files, please revert unrelated changes

I have executed make codegen and it seems that this changed all the Copyright YEAR info in the generated files.

@wenqiq
Copy link
Contributor Author

wenqiq commented Mar 21, 2022

Updated.
/test-all
/test-ipv6-all
/test-ipv6-only-all
/test-windows-all

Copy link
Member

@tnqn tnqn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM overall

pkg/antctl/command_definition_test.go Outdated Show resolved Hide resolved
pkg/antctl/transform/addressgroup/transform.go Outdated Show resolved Hide resolved
pkg/controller/networkpolicy/store/addressgroup.go Outdated Show resolved Hide resolved
pkg/util/k8s/node.go Outdated Show resolved Hide resolved
pkg/util/k8s/node.go Outdated Show resolved Hide resolved
docs/antrea-network-policy.md Outdated Show resolved Hide resolved
docs/antrea-network-policy.md Outdated Show resolved Hide resolved
docs/antrea-network-policy.md Outdated Show resolved Hide resolved
docs/antrea-network-policy.md Outdated Show resolved Hide resolved
docs/antrea-network-policy.md Outdated Show resolved Hide resolved
pkg/controller/networkpolicy/networkpolicy_controller.go Outdated Show resolved Hide resolved
test/e2e/antreapolicy_test.go Outdated Show resolved Hide resolved
docs/antrea-network-policy.md Outdated Show resolved Hide resolved
tnqn
tnqn previously approved these changes Mar 23, 2022
Copy link
Member

@tnqn tnqn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, one nit

@wenqiq wenqiq dismissed stale reviews from tnqn via bb8fd20 March 23, 2022 03:32
@wenqiq
Copy link
Contributor Author

wenqiq commented Mar 23, 2022

/test-all-features-conformance
/test-conformance
/test-e2e
/test-flexible-ipam-e2e
/test-ipv6-conformance
/test-ipv6-e2e
/test-ipv6-networkpolicy
/test-ipv6-only-conformance
/test-ipv6-only-e2e
/test-ipv6-only-networkpolicy
/test-multicluster-e2e
/test-networkpolicy 
/test-windows-conformance
/test-windows-e2e
/test-windows-networkpolicy
/test-windows-proxyall-e2e

docs/antrea-network-policy.md Outdated Show resolved Hide resolved
3. The transport IP (the IP address of the interface used for tunneling or routing the traffic across Nodes) if it's different from Node IP

Traffic to/from other IPs of the Node will be ignored.
Meanwhile, `NodeSelector` doesn’t affect the traffic from Node to Pods running on that Node. Such traffic will always be allowed to make sure that [agents on a Node (e.g. system daemons, kubelet) can communicate with all Pods on that Node](https://kubernetes.io/docs/concepts/services-networking/#the-kubernetes-network-model) to perform liveness and readiness probes. For more information, see [https://github.com/antrea-io/antrea/pull/104](https://github.com/antrea-io/antrea/pull/104).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please break the line into multiple lines, aligning with other lines

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated.

Connected,
},
{
Pod("z/" + clientName1),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With or without NodeSelector, this client can access this server, what does it validate?

Copy link
Contributor Author

@wenqiq wenqiq Mar 24, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This case verified that the other Node to Pod traffic will not be affected by the nodeSelector rule.
However, using the third Node to verify this is more appropriate.
Meanwhile, it seems that there is no test case about local Node traffic to Pod, from this point of view, the test case makes sense?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

testDefaultDenyIngressPolicy tests probe traffic

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated.

1. Add `nodeSelector` field in ACNP/ANP ingress/egress rules to restrict
traffic to/from particular Nodes.

2. Add Node IPs info in the output of `antctl get addressgroup`.
For example:
```
antctl get addressgroup
NAME                                 POD-IPS                                NODE-IPS
5e1bbf8e-67a3-5384-b2bf-e42e93bd68aa <NONE>                                 192.168.0.1, 10.176.27.105
f8c56571-d6db-51ec-9352-a9a47476a9a0 192.168.0.70,192.168.1.38,192.168.1.39 <NONE>

antctl get addressgroup -oyaml
- name: 5e1bbf8e-67a3-5384-b2bf-e42e93bd68aa
  nodes:
  - ip: 192.168.0.1, 10.176.27.105
    node:
      name: wenqiq01-1
- name: f8c56571-d6db-51ec-9352-a9a47476a9a0
  pods:
  - ip: 192.168.1.38
    pod:
      name: iperf3-55bcff667d-v495l
      namespace: demo
  - ip: 192.168.1.39
    pod:
      name: iperf3-55bcff667d-c74ll
      namespace: demo
  - ip: 192.168.0.70
    pod:
      name: iperf3-55bcff667d-pz2d2
      namespace: demo
```

Fixes: antrea-io#3023

Signed-off-by: Wenqi Qiu <wenqiq@vmware.com>

Co-authored-by: Quan Tian <qtian@vmware.com>
@wenqiq
Copy link
Contributor Author

wenqiq commented Mar 24, 2022

Rebase and squash the commits.

Signed-off-by: Wenqi Qiu <wenqiq@vmware.com>
Copy link
Member

@tnqn tnqn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@tnqn
Copy link
Member

tnqn commented Mar 24, 2022

/test-all
/test-ipv6-all
/test-ipv6-only-all
/test-windows-all

@wenqiq
Copy link
Contributor Author

wenqiq commented Mar 24, 2022

unit-test failed:

ok  	antrea.io/antrea/pkg/support	1.388s
I0324 11:35:21.314426    4204 channel.go:83] "Starting SubscribableChannel" name="foo"
--- FAIL: TestSubscribe (0.54s)
    channel_test.go:74: 
        	Error Trace:	channel_test.go:74
        	Error:      	Received unexpected error:
        	            	timed out waiting for the condition
        	Test:       	TestSubscribe
        	Messages:   	Receiver 0 failed to receive all events, expected 1000 events, got 935 events
I0324 11:35:21.857609    4204 channel.go:87] "Stopping SubscribableChannel" name="foo"
E0324 11:35:22.858412    4204 channel.go:77] "Failed to send event to channel, will discard it" name="foo" event="foo"
FAIL
FAIL	antrea.io/antrea/pkg/util/channel	1.587s

@tnqn tnqn merged commit 073f5ab into antrea-io:main Mar 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
action/release-note Indicates a PR that should be included in release notes.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support NodeSelector in ACNP/ANP ingress/egress rules
6 participants