Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Network policy E2E fails #2206

Closed
zhangzujian opened this issue Dec 31, 2022 · 5 comments · Fixed by #2251, #2273 or #2310
Closed

Network policy E2E fails #2206

zhangzujian opened this issue Dec 31, 2022 · 5 comments · Fixed by #2251, #2273 or #2310
Assignees
Labels
bug Something isn't working
Projects

Comments

@zhangzujian
Copy link
Member

Expected Behavior

Actual Behavior

2022-12-30T09:53:48.0127828Z �[38;5;9m�[1mSummarizing 6 Failures:�[0m
2022-12-30T09:53:48.0128460Z   �[38;5;9m[FAIL]�[0m �[0m[sig-network] Netpol �[38;5;243mNetworkPolicy between server and client �[0m�[38;5;9m�[1m[It] should allow ingress access from namespace on one named port [Feature:NetworkPolicy]�[0m�[0m
2022-12-30T09:53:48.0128842Z   �[38;5;243m/home/runner/go/pkg/mod/k8s.io/kubernetes@v1.26.0/test/e2e/network/netpol/test_helper.go:126�[0m
2022-12-30T09:53:48.0129486Z   �[38;5;9m[FAIL]�[0m �[0m[sig-network] Netpol �[38;5;243mNetworkPolicy between server and client �[0m�[38;5;9m�[1m[It] should enforce except clause while egress access to server in CIDR block [Feature:NetworkPolicy]�[0m�[0m
2022-12-30T09:53:48.0129860Z   �[38;5;243m/home/runner/go/pkg/mod/k8s.io/kubernetes@v1.26.0/test/e2e/network/netpol/test_helper.go:126�[0m
2022-12-30T09:53:48.0130500Z   �[38;5;9m[FAIL]�[0m �[0m[sig-network] Netpol �[38;5;243mNetworkPolicy between server and client �[0m�[38;5;9m�[1m[It] should allow ingress access on one named port [Feature:NetworkPolicy]�[0m�[0m
2022-12-30T09:53:48.0130871Z   �[38;5;243m/home/runner/go/pkg/mod/k8s.io/kubernetes@v1.26.0/test/e2e/network/netpol/test_helper.go:126�[0m
2022-12-30T09:53:48.0131600Z   �[38;5;9m[FAIL]�[0m �[0m[sig-network] Netpol �[38;5;243mNetworkPolicy between server and client �[0m�[38;5;9m�[1m[It] should ensure an IP overlapping both IPBlock.CIDR and IPBlock.Except is allowed [Feature:NetworkPolicy]�[0m�[0m
2022-12-30T09:53:48.0131961Z   �[38;5;243m/home/runner/go/pkg/mod/k8s.io/kubernetes@v1.26.0/test/e2e/network/netpol/test_helper.go:126�[0m
2022-12-30T09:53:48.0132532Z   �[38;5;9m[FAIL]�[0m �[0m[sig-network] Netpol �[38;5;243mNetworkPolicy between server and client �[0m�[38;5;9m�[1m[It] should allow egress access to server in CIDR block [Feature:NetworkPolicy]�[0m�[0m
2022-12-30T09:53:48.0132900Z   �[38;5;243m/home/runner/go/pkg/mod/k8s.io/kubernetes@v1.26.0/test/e2e/network/netpol/test_helper.go:126�[0m
2022-12-30T09:53:48.0133714Z   �[38;5;9m[FAIL]�[0m �[0m[sig-network] Netpol �[38;5;243mNetworkPolicy between server and client �[0m�[38;5;9m�[1m[It] should allow egress access on one named port [Feature:NetworkPolicy]�[0m�[0m
2022-12-30T09:53:48.0134091Z   �[38;5;243m/home/runner/go/pkg/mod/k8s.io/kubernetes@v1.26.0/test/e2e/network/netpol/test_helper.go:126�[0m

Steps to Reproduce the Problem

Additional Info

  • Kubernetes version:

    Output of kubectl version:

    (paste your output here)
    
  • kube-ovn version:

    (paste your output here)
    
  • operation-system/kernel version:

    Output of awk -F '=' '/PRETTY_NAME/ { print $2 }' /etc/os-release:
    Output of uname -r:

    (paste your output here)
    
@oilbeater oilbeater added the bug Something isn't working label Jan 3, 2023
@oilbeater oilbeater added this to To do in 2023-1 via automation Jan 3, 2023
@changluyi changluyi moved this from To do to In progress in 2023-1 Jan 6, 2023
@changluyi
Copy link
Collaborator

changluyi commented Jan 12, 2023

=====================================================================

  1. should allow ingress access from namespace on one named port
  2. should allow ingress access on one named port
  3. should allow egress access on one named port

是一类问题,
直接用port的acl规则
ip4.src == $port.policy.default.ingress.allow.IPv4.0 && ip4.src != $port.policy.default.ingress.except.IPv4.0 && tcp.dst == 80 && outport==@port.policy.default && ip

用port name的acl规则
ip4.src == $port.policy.default.ingress.allow.IPv4.0 && ip4.src != $port.policy.default.ingress.except.IPv4.0 && tcp.dst == 0 && outport==@port.policy.default && ip

可以见到tcp.dst == 0没有解析到端口号,需要kube-ovn增加解析

=====================================================================

  1. should enforce except clause while egress access to server in CIDR block
  2. should ensure an IP overlapping both IPBlock.CIDR and IPBlock.Except is allowed
  3. should allow egress access to server in CIDR block
    是一类问题。主要是因为k8s netpo检查可达性的时候,目的ip用的对应pod的svc ip而不是pod ip,导致结果行为不一样

比如case 1
poda 去访问 podb (svcip 10.107.181.41, pod ip 10.16.1.115)

egress:
 - to:
   - ipBlock:
        cidr: 10.16.0.46/4
        except:
        - 10.16.1.115
  podSelector:
      matchLabels:
        pod: a
  policyTypes:
  - Egress

规则如上

脚本期望: poda 访问podb不能通,

实际结果: poda 访问podb通了

对于kube-ovn的行为来说:
如果是poda去访问podb svcip, 那确实会导致实际结果是通的和脚本预期结果不一致。
如果是poda去访问podb podip,结果和脚本预期结果才是一样的。

@oilbeater
Copy link
Collaborator

第一类问题算是功能缺失,需要考虑怎么加上 named port 支持。

第二类问题测试是没问题的,如果不能访问某个 pod 那么通过 service 也不应该能访问。这个可能和 ovn 里的逻辑有关,有可能是先做的 acl 后做的 svc ,不知道后端 ip 导致的。这个可能要考虑下 networkpolicy 的 acl 该怎么加,可以参考下 ovn-kubernetes 这块是怎么实现的

@changluyi
Copy link
Collaborator

是的 先做的acl,然后做dnat,
image

@changluyi
Copy link
Collaborator

ovn acl中加options:apply-after-lb="true" 可以先dnat 后 acl

2023-1 automation moved this from In progress to Done Jan 19, 2023
@oilbeater oilbeater reopened this Jan 28, 2023
2023-1 automation moved this from Done to In progress Jan 28, 2023
@oilbeater
Copy link
Collaborator

Namedport is still WIP

2023-1 automation moved this from In progress to Done Feb 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
No open projects
3 participants