Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reply from pod to outside is dropped when L7 ingress policy is used #21954

Closed
brb opened this issue Nov 1, 2022 · 5 comments · Fixed by #21980 or #24208
Closed

Reply from pod to outside is dropped when L7 ingress policy is used #21954

brb opened this issue Nov 1, 2022 · 5 comments · Fixed by #21980 or #24208
Assignees
Labels
feature/ipv6 Relates to IPv6 protocol support kind/bug This is a bug in the Cilium logic. pinned These issues are not marked stale by our issue bot. sig/datapath Impacts bpf/ or low-level forwarding details, including map management and monitor messages. sig/policy Impacts whether traffic is allowed or denied based on user-defined policies.

Comments

@brb
Copy link
Member

brb commented Nov 1, 2022

This issue is about an L7 ingress policy problem a when a pod is reached directly from outside client / via a NodePort BPF service.

Let's consider the following L7 netpol:

apiVersion: "cilium.io/v2"
kind: CiliumNetworkPolicy
metadata:
  name: foobar
spec:
  description: "Allow to GET on echo from outside"
  endpointSelector:
    matchLabels:
      kind: echo
  ingress:
  - fromEntities:
    - "world"
    toPorts:
    - ports:
      - port: "8080"
        protocol: TCP
      rules:
        http:
        - method: "GET"
          path: "/$"

When the netpol is applied, accessing the echo pod from outside the cluster fails with:

xx drop (Stale or unroutable IP) flow 0xf863e34e to endpoint 0, file bpf_host.c line 665, , identity 28143->unknown: 10.0.1.49:80 -> 192.168.34.1:32884 tcp SYN, ACK

The drop is triggered by https://github.com/cilium/cilium/blob/v1.12.3/bpf/bpf_host.c#L626.

What happens is that the L7 proxy sends the SYN-ACK which gets handled by bpf_host @ cilium_host, and then dropped. See the pwru output (ifindex=9 is cilium_host):

  SKB    CPU          PROCESS                     FUNC
0xffff9a859048e800      7        [<empty>]             ip_local_out netns=4026531992 mark=0xa00 ifindex=0 proto=0 mtu=0 len=60 10.0.1.49:80->192.168.34.1:32884(tcp)
0xffff9a859048e800      7        [<empty>]           __ip_local_out netns=4026531992 mark=0xa00 ifindex=0 proto=0 mtu=0 len=60 10.0.1.49:80->192.168.34.1:32884(tcp)
0xffff9a859048e800      7        [<empty>]             nf_hook_slow netns=4026531992 mark=0xa00 ifindex=0 proto=8 mtu=0 len=60 10.0.1.49:80->192.168.34.1:32884(tcp)
0xffff9a859048e800      7        [<empty>]                ip_output netns=4026531992 mark=0xa00 ifindex=0 proto=8 mtu=0 len=60 10.0.1.49:80->192.168.34.1:32884(tcp)
0xffff9a859048e800      7        [<empty>]             nf_hook_slow netns=4026531992 mark=0xa00 ifindex=9 proto=8 mtu=1500 len=60 10.0.1.49:80->192.168.34.1:32884(tcp)
0xffff9a859048e800      7        [<empty>]  apparmor_ipv4_postroute netns=4026531992 mark=0xa00 ifindex=9 proto=8 mtu=1500 len=60 10.0.1.49:80->192.168.34.1:32884(tcp)
0xffff9a859048e800      7        [<empty>]         ip_finish_output netns=4026531992 mark=0xa00 ifindex=9 proto=8 mtu=1500 len=60 10.0.1.49:80->192.168.34.1:32884(tcp)
0xffff9a859048e800      7        [<empty>] __cgroup_bpf_run_filter_skb netns=4026531992 mark=0xa00 ifindex=9 proto=8 mtu=1500 len=60 10.0.1.49:80->192.168.34.1:32884(tcp)
0xffff9a859048e800      7        [<empty>]       __ip_finish_output netns=4026531992 mark=0xa00 ifindex=9 proto=8 mtu=1500 len=60 10.0.1.49:80->192.168.34.1:32884(tcp)
0xffff9a859048e800      7        [<empty>]        ip_finish_output2 netns=4026531992 mark=0xa00 ifindex=9 proto=8 mtu=1500 len=60 10.0.1.49:80->192.168.34.1:32884(tcp)
0xffff9a859048e800      7        [<empty>]           dev_queue_xmit netns=4026531992 mark=0xa00 ifindex=9 proto=8 mtu=1500 len=74 10.0.1.49:80->192.168.34.1:32884(tcp)
0xffff9a859048e800      7        [<empty>]         __dev_queue_xmit netns=4026531992 mark=0xa00 ifindex=9 proto=8 mtu=1500 len=74 10.0.1.49:80->192.168.34.1:32884(tcp)
0xffff9a859048e800      7        [<empty>]             tcf_classify netns=4026531992 mark=0xa00 ifindex=9 proto=8 mtu=1500 len=74 10.0.1.49:80->192.168.34.1:32884(tcp)
0xffff9a859048e800      7        [<empty>]      skb_ensure_writable netns=4026531992 mark=0x0 ifindex=9 proto=8 mtu=1500 len=74 10.0.1.49:80->192.168.34.1:32884(tcp)
0xffff9a859048e800      7        [<empty>]                kfree_skb netns=4026531992 mark=0x0 ifindex=9 proto=8 mtu=1500 len=74 10.0.1.49:80->192.168.34.1:32884(tcp)

The packet is sent to the cilium_host because of the mark and the following IP rules / routes:

$ ip rule list
...
10:     from all fwmark 0xa00/0xf00 lookup 2005

$ ip route show table 2005
default via 10.0.1.116 dev cilium_host
10.0.1.116 dev cilium_host scope link

One fix is to extend the troublesome check https://github.com/cilium/cilium/blob/v1.12.3/bpf/bpf_host.c#L626 by allowing proxy replies to WORLD_ID.

@brb brb added kind/bug This is a bug in the Cilium logic. sig/datapath Impacts bpf/ or low-level forwarding details, including map management and monitor messages. sig/policy Impacts whether traffic is allowed or denied based on user-defined policies. labels Nov 1, 2022
@github-actions

This comment was marked as off-topic.

@github-actions github-actions bot added the stale The stale bot thinks this issue is old. Add "pinned" label to prevent this from becoming stale. label Jan 1, 2023
@pchaigno pchaigno added pinned These issues are not marked stale by our issue bot. and removed stale The stale bot thinks this issue is old. Add "pinned" label to prevent this from becoming stale. labels Jan 2, 2023
@brb brb self-assigned this Jan 9, 2023
@brb brb reopened this Jan 24, 2023
@brb
Copy link
Member Author

brb commented Jan 25, 2023

Unfortunately, removal of the 2005 rtable is causing the legit CI failures.

After removing the 2005 rtable to fix the L7 issue, the kube-proxy NodePort with L7 netpol started to fail in the CI. After taking closer look, the removal of the rtable is causing the reply from the envoy proxy to be passed to lo instead of cilium_host :

14:54:33.585708 eth0  In  IP6 fc00:f853:ccd:e793::3.52394 > fc00:f853:ccd:e793::4.30239: Flags [S], seq 504540809, win 64800, options [mss 1440,sackOK,TS val 3651151592 ecr 0,nop,wscale 7], length 0
14:54:33.585852 cilium_host Out IP6 fc00:f853:ccd:e793::4.13607 > fd00:10:244:2::c527.80: Flags [S], seq 504540809, win 64800, options [mss 1440,sackOK,TS val 3651151592 ecr 0,nop,wscale 7], length 0
14:54:33.585856 cilium_net P   IP6 fc00:f853:ccd:e793::4.13607 > fd00:10:244:2::c527.80: Flags [S], seq 504540809, win 64800, options [mss 1440,sackOK,TS val 3651151592 ecr 0,nop,wscale 7], length 0
14:54:33.585916 lo    In  IP6 fd00:10:244:2::c527.80 > fc00:f853:ccd:e793::4.13607: Flags [S.], seq 2619962850, ack 504540810, win 65464, options [mss 65476,sackOK,TS val 1096880080 ecr 3651151592,nop,wscale 7], length 0
14:54:33.585960 cilium_host Out IP6 fc00:f853:ccd:e793::4.13607 > fd00:10:244:2::c527.80: Flags [R], seq 504540810, win 0, length 0

The NodePort request gets SNAT-ed by iptables to the cilium_host IP addr. The trace is taken on the fc00:f853:ccd:e793::4 node which runs the selected NodePort endpoint.

@brb
Copy link
Member Author

brb commented Mar 22, 2023

x-posting #23346 (comment)

@brb
Copy link
Member Author

brb commented Mar 23, 2023

Still puzzled why this doesn't happen in the v4 path:

07:04:13.121936 eth0  In  IP 172.12.1.6.44452 > 172.12.1.5.30452: Flags [S], seq 2923705218, win 64240, options [mss 1460,sackOK,TS val 2856811446 ecr 0,nop,wscale 7], length 0
07:04:13.122035 cilium_host Out IP 10.244.1.195.43576 > 10.244.1.237.8080: Flags [S], seq 2923705218, win 64240, options [mss 1460,sackOK,TS val 2856811446 ecr 0,nop,wscale 7], length 0
07:04:13.122038 cilium_net P   IP 10.244.1.195.43576 > 10.244.1.237.8080: Flags [S], seq 2923705218, win 64240, options [mss 1460,sackOK,TS val 2856811446 ecr 0,nop,wscale 7], length 0
07:04:13.122100 eth0  Out IP 172.12.1.5.30452 > 172.12.1.6.44452: Flags [S.], seq 1251754280, ack 2923705219, win 65483, options [mss 65495,sackOK,TS val 3449593346 ecr 2856811446,nop,wscale 7], length 0

The request got SNAT-ed to the cilium_host's v4 addr (same as in the v6 case).

@jschwinger233
Copy link
Member

jschwinger233 commented Mar 31, 2023

Ok I believe I have thoroughly figured out this matter.

TL;DR

ip6tables miss some rules to allow the responding SYN+ACK from proxy to do reverse NAT:

this is current ip6tables rules:

-A CILIUM_OUTPUT_raw -m mark --mark 0xa00/0xfffffeff -m comment --comment "cilium: NOTRACK for proxy return traffic" -j CT --notrack

this is what we expect:

-A CILIUM_OUTPUT_raw -o lxc+ -m mark --mark 0xa00/0xfffffeff -m comment --comment "cilium: NOTRACK for proxy return traffic" -j CT --notrack
-A CILIUM_OUTPUT_raw -o cilium_host -m mark --mark 0xa00/0xfffffeff -m comment --comment "cilium: NOTRACK for proxy return traffic" -j CT --notrack
-A CILIUM_OUTPUT_raw -o lxc+ -m mark --mark 0x800/0xe00 -m comment --comment "cilium: NOTRACK for L7 proxy upstream traffic" -j CT --notrack
-A CILIUM_OUTPUT_raw -o cilium_host -m mark --mark 0x800/0xe00 -m comment --comment "cilium: NOTRACK for L7 proxy upstream traffic" -j CT --notrack

Long version:

  1. Why this issue is relevant to route table 2005?

Route table 2005 is simply routing skb with mark 0xa00 to cilium_host, and skb returned by proxy is marked by 0xa00.

Thanks to this routing, the responding skb is going to cilium_host, then cilium_net, and finally back to kernel stack, but the mark 0xa00 is made clear after cilium_net. After that, the current NOTRACK for proxy return traffic ip6tables rule will be hit no more, and conntrack can rev-NAT the skb as we wish.

Once we delete route table 2005, responding skb will be routed by the following rule in local table:

local fd00:10:244:3::bba5 dev cilium_host proto kernel metric 0 pref medium

According to the kenel source code, route rules with type local will use lo if the dev is not an L3 master:

  1. if route type is local, mark flags with RTF_LOCAL: https://github.com/torvalds/linux/blob/62bad54b26db8bc98e28749cd76b2d890edb4258/net/ipv6/route.c#L5031-L5032
  2. if flags is marked with RTF_LOCAL, set dst dev lo: https://github.com/torvalds/linux/blob/62bad54b26db8bc98e28749cd76b2d890edb4258/net/ipv6/route.c#L1011-L1020

Therefore, responding skb will go to lo instead of cilium_host, and gets no chance to be rev-NAT-ed.

This explains why my PR #24208 fails to fix this issue once deleting route table 2005.

  1. Why deleting route table 2005 doesn't affect ipv4?

ip4tables has rules of "NOTRACK for proxy return traffic" which are only applied for -o cilium_host or -o lxc+, so after route table 2005 disappears, the return skb which is prepared to be routed to lo (because of the same local type routing in local route table) can be processed by conntrack properly, and at last will be rerouted to eth0 and complete the tcp handshake.

This will be fixed together in #24208

jschwinger233 added a commit to jschwinger233/cilium-cli that referenced this issue May 18, 2023
…-policy when running on < 1.14.0 Cilium

cilium/cilium#21954 for the IPv6 path was
resolved only for v1.14, but not for v1.13. In order to be able to run
the latest connectivity tests on v1.13, we need to skip curl requests to
the IPv6 addresses in that particular test.

Fixes: cilium#1627

Signed-off-by: Zhichuan Liang <gray.liang@isovalent.com>
brb pushed a commit to cilium/cilium-cli that referenced this issue May 19, 2023
…-policy when running on < 1.14.0 Cilium

cilium/cilium#21954 for the IPv6 path was
resolved only for v1.14, but not for v1.13. In order to be able to run
the latest connectivity tests on v1.13, we need to skip curl requests to
the IPv6 addresses in that particular test.

Fixes: #1627

Signed-off-by: Zhichuan Liang <gray.liang@isovalent.com>
jrajahalme pushed a commit to jrajahalme/cilium that referenced this issue May 22, 2023
This reverts commit 3ed62d5 partially and only removes ipv4 2005 route
table.

Fixes: cilium#21954

Signed-off-by: Zhichuan Liang <gray.liang@isovalent.com>
jrajahalme pushed a commit to jrajahalme/cilium that referenced this issue May 22, 2023
This commit adds e2e test to cover issue cilium#21954.

Test cases for IPv6 are deleted and PR cilium#24882 will take care of them.

Signed-off-by: Zhichuan Liang <gray.liang@isovalent.com>
jschwinger233 added a commit to jschwinger233/cilium that referenced this issue May 26, 2023
The test case was introduced to cover issue cilium#21954, but it turned out
the test is buggy and caused a number of CI flakes (cilium#25119).
Consequently, PR cilium#25236 put the test case under quarantine.

This commit removes that problematic test, as the target scenario has
been covered by connectivity test in cilium-cli
(cilium/cilium-cli#1547).

Signed-off-by: Zhichuan Liang <gray.liang@isovalent.com>
brb pushed a commit that referenced this issue May 26, 2023
The test case was introduced to cover issue #21954, but it turned out
the test is buggy and caused a number of CI flakes (#25119).
Consequently, PR #25236 put the test case under quarantine.

This commit removes that problematic test, as the target scenario has
been covered by connectivity test in cilium-cli
(cilium/cilium-cli#1547).

Signed-off-by: Zhichuan Liang <gray.liang@isovalent.com>
sayboras pushed a commit to sayboras/cilium that referenced this issue May 29, 2023
[ upstream commit eb5bf06 ]

The test case was introduced to cover issue cilium#21954, but it turned out
the test is buggy and caused a number of CI flakes (cilium#25119).
Consequently, PR cilium#25236 put the test case under quarantine.

This commit removes that problematic test, as the target scenario has
been covered by connectivity test in cilium-cli
(cilium/cilium-cli#1547).

Signed-off-by: Zhichuan Liang <gray.liang@isovalent.com>
Signed-off-by: Tam Mach <tam.mach@cilium.io>
michi-covalent pushed a commit to michi-covalent/cilium that referenced this issue May 30, 2023
This test case covers cilium#21954.

A new policy `echo-ingress-l7-policy-from-anywhere` is added to allow
HTTP GET / on echo pods from outside.

Use `cilium connectivity test --test north-south-loadbalancing --datapath`
to run this test.

Signed-off-by: Zhichuan Liang <gray.liang@isovalent.com>
michi-covalent pushed a commit to michi-covalent/cilium that referenced this issue May 30, 2023
…-policy when running on < 1.14.0 Cilium

cilium#21954 for the IPv6 path was
resolved only for v1.14, but not for v1.13. In order to be able to run
the latest connectivity tests on v1.13, we need to skip curl requests to
the IPv6 addresses in that particular test.

Fixes: cilium#1627

Signed-off-by: Zhichuan Liang <gray.liang@isovalent.com>
julianwiedmann pushed a commit that referenced this issue Jun 2, 2023
[ upstream commit eb5bf06 ]

The test case was introduced to cover issue #21954, but it turned out
the test is buggy and caused a number of CI flakes (#25119).
Consequently, PR #25236 put the test case under quarantine.

This commit removes that problematic test, as the target scenario has
been covered by connectivity test in cilium-cli
(cilium/cilium-cli#1547).

Signed-off-by: Zhichuan Liang <gray.liang@isovalent.com>
Signed-off-by: Tam Mach <tam.mach@cilium.io>
@jschwinger233 jschwinger233 added feature/ipv6 Relates to IPv6 protocol support and removed feature/ipv6 Relates to IPv6 protocol support labels Jul 3, 2023
jschwinger233 added a commit to jschwinger233/cilium that referenced this issue Oct 12, 2023
This reverts commit 9dd6cfc.

2005 route table is meant to push packets with mark 0xa00 to
cilium_host:

 ```
$ ip ru
10:	from all fwmark 0xa00/0xf00 lookup 2005

$ ip r s t 2005
default via 10.244.1.237 dev cilium_host
10.244.1.237 dev cilium_host scope link
 ```

2005 route table was deleted to fix cilium#21954
(Reply from pod to outside is dropped when L7 ingress policy is used).
We decided to do so because we thought it was no more used, and causing
troubles.

However, we recently realized it's still critical to ensure correct encryption
when IPsec is enabled and L7 policy is applied.

Consider a reply packet from L7 proxy, this packet must have mark 0xa00
to indicate it's from proxy. With 2005 route table, this from-proxy
packet will be routed to cilium_host, where from_host bpf prog will
process the packet for IPsec encryption; without 2005 table, this packet
has no chance to get encrypted, goes out with plain payload.

This commit brings back 2005 route table for IPv6.

For IPv4 2005 route table and resurged issue
cilium#21954, we'll handle later as
separate patches.

Signed-off-by: Zhichuan Liang <gray.liang@isovalent.com>
jschwinger233 added a commit to jschwinger233/cilium that referenced this issue Oct 12, 2023
This commit fixes cilium#21954, as the original patch 9dd6cfc
(datapath: remove 2005 route table for ipv6 only) has been reverted due
to IPsec + L7 policy issues.

This commit simply allows packets to WORLD as long as they are from
proxy. This was one of the solution suggested by Martynas, as recorded
in commit message cilium/cilium@c534bb7:

    One fix was to extend the troublesome check
    https://github.com/cilium/cilium/blob/v1.12.3/bpf/bpf_host.c#L626 by
    allowing proxy replies to `WORLD_ID`.

Fixes: cilium#21954 (Reply from pod to outside is dropped when L7 ingress policy is used)

Signed-off-by: Zhichuan Liang <gray.liang@isovalent.com>
jschwinger233 added a commit to jschwinger233/cilium that referenced this issue Dec 1, 2023
This is an alternative approach to fix cilium#21954, so that we
can re-introduce the 2005 from-proxy routing rule in following patches
to fix L7 proxy issues.

This commit simply allows packets to WORLD as long as they are from
ingress proxy. This was one of the solution suggested by Martynas, as
recorded in commit message cilium/cilium@c534bb7:

    One fix was to extend the troublesome check
    https://github.com/cilium/cilium/blob/v1.12.3/bpf/bpf_host.c#L626 by
    allowing proxy replies to `WORLD_ID`.

To tell if an skb is originated from ingress proxy, the commit extends
the semantic of existing flags `TC_INDEX_F_SKIP_{INGRESS,EGRESS}_PROXY`,
renames flags to clarify the changed meaning.

Fixes: cilium#21954 (Reply from pod to outside is dropped when L7 ingress policy is used)

Signed-off-by: Zhichuan Liang <gray.liang@isovalent.com>
jschwinger233 added a commit to jschwinger233/cilium that referenced this issue Dec 1, 2023
This is an alternative approach to fix cilium#21954, so that we
can re-introduce the 2005 from-proxy routing rule in following patches
to fix L7 proxy issues.

This commit simply allows packets to WORLD as long as they are from
ingress proxy. This was one of the solution suggested by Martynas, as
recorded in commit message cilium/cilium@c534bb7:

    One fix was to extend the troublesome check
    https://github.com/cilium/cilium/blob/v1.12.3/bpf/bpf_host.c#L626 by
    allowing proxy replies to `WORLD_ID`.

To tell if an skb is originated from ingress proxy, the commit extends
the semantic of existing flags `TC_INDEX_F_SKIP_{INGRESS,EGRESS}_PROXY`,
renames flags to clarify the changed meaning.

Fixes: cilium#21954 (Reply from pod to outside is dropped when L7 ingress policy is used)

Signed-off-by: Zhichuan Liang <gray.liang@isovalent.com>
jschwinger233 added a commit to jschwinger233/cilium that referenced this issue Dec 1, 2023
This is an alternative approach to fix cilium#21954, so that we
can re-introduce the 2005 from-proxy routing rule in following patches
to fix L7 proxy issues.

This commit simply allows packets to WORLD as long as they are from
ingress proxy. This was one of the solution suggested by Martynas, as
recorded in commit message cilium/cilium@c534bb7:

    One fix was to extend the troublesome check
    https://github.com/cilium/cilium/blob/v1.12.3/bpf/bpf_host.c#L626 by
    allowing proxy replies to `WORLD_ID`.

To tell if an skb is originated from ingress proxy, the commit extends
the semantic of existing flags `TC_INDEX_F_SKIP_{INGRESS,EGRESS}_PROXY`,
renames flags to clarify the changed meaning.

Fixes: cilium#21954 (Reply from pod to outside is dropped when L7 ingress policy is used)

Signed-off-by: Zhichuan Liang <gray.liang@isovalent.com>
github-merge-queue bot pushed a commit that referenced this issue Dec 1, 2023
This is an alternative approach to fix #21954, so that we
can re-introduce the 2005 from-proxy routing rule in following patches
to fix L7 proxy issues.

This commit simply allows packets to WORLD as long as they are from
ingress proxy. This was one of the solution suggested by Martynas, as
recorded in commit message c534bb7:

    One fix was to extend the troublesome check
    https://github.com/cilium/cilium/blob/v1.12.3/bpf/bpf_host.c#L626 by
    allowing proxy replies to `WORLD_ID`.

To tell if an skb is originated from ingress proxy, the commit extends
the semantic of existing flags `TC_INDEX_F_SKIP_{INGRESS,EGRESS}_PROXY`,
renames flags to clarify the changed meaning.

Fixes: #21954 (Reply from pod to outside is dropped when L7 ingress policy is used)

Signed-off-by: Zhichuan Liang <gray.liang@isovalent.com>
pjablonski123 pushed a commit to pjablonski123/cilium that referenced this issue Dec 15, 2023
This is an alternative approach to fix cilium#21954, so that we
can re-introduce the 2005 from-proxy routing rule in following patches
to fix L7 proxy issues.

This commit simply allows packets to WORLD as long as they are from
ingress proxy. This was one of the solution suggested by Martynas, as
recorded in commit message cilium/cilium@c534bb7:

    One fix was to extend the troublesome check
    https://github.com/cilium/cilium/blob/v1.12.3/bpf/bpf_host.c#L626 by
    allowing proxy replies to `WORLD_ID`.

To tell if an skb is originated from ingress proxy, the commit extends
the semantic of existing flags `TC_INDEX_F_SKIP_{INGRESS,EGRESS}_PROXY`,
renames flags to clarify the changed meaning.

Fixes: cilium#21954 (Reply from pod to outside is dropped when L7 ingress policy is used)

Signed-off-by: Zhichuan Liang <gray.liang@isovalent.com>
pchaigno pushed a commit that referenced this issue Jan 8, 2024
[ upstream commit eb5bf06 ]

The test case was introduced to cover issue #21954, but it turned out
the test is buggy and caused a number of CI flakes (#25119).
Consequently, PR #25236 put the test case under quarantine.

This commit removes that problematic test, as the target scenario has
been covered by connectivity test in cilium-cli
(cilium/cilium-cli#1547).

Signed-off-by: Zhichuan Liang <gray.liang@isovalent.com>
Signed-off-by: Tam Mach <tam.mach@cilium.io>
Signed-off-by: Michi Mutsuzaki <michi@isovalent.com>
julianwiedmann pushed a commit to julianwiedmann/cilium that referenced this issue Mar 5, 2024
[ upstream commit ac63856 ]

This is an alternative approach to fix cilium#21954, so that we
can re-introduce the 2005 from-proxy routing rule in following patches
to fix L7 proxy issues.

This commit simply allows packets to WORLD as long as they are from
ingress proxy. This was one of the solution suggested by Martynas, as
recorded in commit message cilium/cilium@c534bb7:

    One fix was to extend the troublesome check
    https://github.com/cilium/cilium/blob/v1.12.3/bpf/bpf_host.c#L626 by
    allowing proxy replies to `WORLD_ID`.

To tell if an skb is originated from ingress proxy, the commit extends
the semantic of existing flags `TC_INDEX_F_SKIP_{INGRESS,EGRESS}_PROXY`,
renames flags to clarify the changed meaning.

Fixes: cilium#21954 (Reply from pod to outside is dropped when L7 ingress policy is used)

Signed-off-by: Zhichuan Liang <gray.liang@isovalent.com>
Signed-off-by: Julian Wiedmann <jwi@isovalent.com>
julianwiedmann pushed a commit to julianwiedmann/cilium that referenced this issue Mar 5, 2024
[ upstream commit ac63856 ]

This is an alternative approach to fix cilium#21954, so that we
can re-introduce the 2005 from-proxy routing rule in following patches
to fix L7 proxy issues.

This commit simply allows packets to WORLD as long as they are from
ingress proxy. This was one of the solution suggested by Martynas, as
recorded in commit message cilium/cilium@c534bb7:

    One fix was to extend the troublesome check
    https://github.com/cilium/cilium/blob/v1.12.3/bpf/bpf_host.c#L626 by
    allowing proxy replies to `WORLD_ID`.

To tell if an skb is originated from ingress proxy, the commit extends
the semantic of existing flags `TC_INDEX_F_SKIP_{INGRESS,EGRESS}_PROXY`,
renames flags to clarify the changed meaning.

Fixes: cilium#21954 (Reply from pod to outside is dropped when L7 ingress policy is used)

Signed-off-by: Zhichuan Liang <gray.liang@isovalent.com>
Signed-off-by: Julian Wiedmann <jwi@isovalent.com>
julianwiedmann pushed a commit to julianwiedmann/cilium that referenced this issue Mar 5, 2024
[ upstream commit ac63856 ]

This is an alternative approach to fix cilium#21954, so that we
can re-introduce the 2005 from-proxy routing rule in following patches
to fix L7 proxy issues.

This commit simply allows packets to WORLD as long as they are from
ingress proxy. This was one of the solution suggested by Martynas, as
recorded in commit message cilium/cilium@c534bb7:

    One fix was to extend the troublesome check
    https://github.com/cilium/cilium/blob/v1.12.3/bpf/bpf_host.c#L626 by
    allowing proxy replies to `WORLD_ID`.

To tell if an skb is originated from ingress proxy, the commit extends
the semantic of existing flags `TC_INDEX_F_SKIP_{INGRESS,EGRESS}_PROXY`,
renames flags to clarify the changed meaning.

Fixes: cilium#21954 (Reply from pod to outside is dropped when L7 ingress policy is used)

Signed-off-by: Zhichuan Liang <gray.liang@isovalent.com>
Signed-off-by: Julian Wiedmann <jwi@isovalent.com>
julianwiedmann pushed a commit that referenced this issue Mar 7, 2024
[ upstream commit ac63856 ]

This is an alternative approach to fix #21954, so that we
can re-introduce the 2005 from-proxy routing rule in following patches
to fix L7 proxy issues.

This commit simply allows packets to WORLD as long as they are from
ingress proxy. This was one of the solution suggested by Martynas, as
recorded in commit message c534bb7:

    One fix was to extend the troublesome check
    https://github.com/cilium/cilium/blob/v1.12.3/bpf/bpf_host.c#L626 by
    allowing proxy replies to `WORLD_ID`.

To tell if an skb is originated from ingress proxy, the commit extends
the semantic of existing flags `TC_INDEX_F_SKIP_{INGRESS,EGRESS}_PROXY`,
renames flags to clarify the changed meaning.

Fixes: #21954 (Reply from pod to outside is dropped when L7 ingress policy is used)

Signed-off-by: Zhichuan Liang <gray.liang@isovalent.com>
Signed-off-by: Julian Wiedmann <jwi@isovalent.com>
julianwiedmann pushed a commit to julianwiedmann/cilium that referenced this issue Mar 7, 2024
[ upstream commit ac63856 ]

This is an alternative approach to fix cilium#21954, so that we
can re-introduce the 2005 from-proxy routing rule in following patches
to fix L7 proxy issues.

This commit simply allows packets to WORLD as long as they are from
ingress proxy. This was one of the solution suggested by Martynas, as
recorded in commit message cilium/cilium@c534bb7:

    One fix was to extend the troublesome check
    https://github.com/cilium/cilium/blob/v1.12.3/bpf/bpf_host.c#L626 by
    allowing proxy replies to `WORLD_ID`.

To tell if an skb is originated from ingress proxy, the commit extends
the semantic of existing flags `TC_INDEX_F_SKIP_{INGRESS,EGRESS}_PROXY`,
renames flags to clarify the changed meaning.

Fixes: cilium#21954 (Reply from pod to outside is dropped when L7 ingress policy is used)

Signed-off-by: Zhichuan Liang <gray.liang@isovalent.com>
Signed-off-by: Julian Wiedmann <jwi@isovalent.com>
julianwiedmann pushed a commit to julianwiedmann/cilium that referenced this issue Mar 7, 2024
[ upstream commit ac63856 ]

This is an alternative approach to fix cilium#21954, so that we
can re-introduce the 2005 from-proxy routing rule in following patches
to fix L7 proxy issues.

This commit simply allows packets to WORLD as long as they are from
ingress proxy. This was one of the solution suggested by Martynas, as
recorded in commit message cilium/cilium@c534bb7:

    One fix was to extend the troublesome check
    https://github.com/cilium/cilium/blob/v1.12.3/bpf/bpf_host.c#L626 by
    allowing proxy replies to `WORLD_ID`.

To tell if an skb is originated from ingress proxy, the commit extends
the semantic of existing flags `TC_INDEX_F_SKIP_{INGRESS,EGRESS}_PROXY`,
renames flags to clarify the changed meaning.

Fixes: cilium#21954 (Reply from pod to outside is dropped when L7 ingress policy is used)

Signed-off-by: Zhichuan Liang <gray.liang@isovalent.com>
Signed-off-by: Julian Wiedmann <jwi@isovalent.com>
julianwiedmann pushed a commit that referenced this issue Mar 7, 2024
[ upstream commit ac63856 ]

This is an alternative approach to fix #21954, so that we
can re-introduce the 2005 from-proxy routing rule in following patches
to fix L7 proxy issues.

This commit simply allows packets to WORLD as long as they are from
ingress proxy. This was one of the solution suggested by Martynas, as
recorded in commit message c534bb7:

    One fix was to extend the troublesome check
    https://github.com/cilium/cilium/blob/v1.12.3/bpf/bpf_host.c#L626 by
    allowing proxy replies to `WORLD_ID`.

To tell if an skb is originated from ingress proxy, the commit extends
the semantic of existing flags `TC_INDEX_F_SKIP_{INGRESS,EGRESS}_PROXY`,
renames flags to clarify the changed meaning.

Fixes: #21954 (Reply from pod to outside is dropped when L7 ingress policy is used)

Signed-off-by: Zhichuan Liang <gray.liang@isovalent.com>
Signed-off-by: Julian Wiedmann <jwi@isovalent.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature/ipv6 Relates to IPv6 protocol support kind/bug This is a bug in the Cilium logic. pinned These issues are not marked stale by our issue bot. sig/datapath Impacts bpf/ or low-level forwarding details, including map management and monitor messages. sig/policy Impacts whether traffic is allowed or denied based on user-defined policies.
Projects
None yet
3 participants