Fix loopback issue injecting istio proxy #494
Conversation
|
Jenkins job manager/presubmit passed |
|
Jenkins job manager/e2e-smoketest passed |
Codecov Report
@@ Coverage Diff @@
## master #494 +/- ##
==========================================
+ Coverage 75.7% 75.83% +0.12%
==========================================
Files 25 25
Lines 3219 3219
==========================================
+ Hits 2437 2441 +4
+ Misses 596 594 -2
+ Partials 186 184 -2
Continue to review full report at Codecov.
|
docker/prepare_proxy.sh
Outdated
| @@ -41,6 +41,12 @@ if [[ -z "${ISTIO_PROXY_PORT-}" ]] || [[ -z "${ISTIO_PROXY_UID-}" ]]; then | |||
| fi | |||
|
|
|||
| iptables -t nat -A PREROUTING -p tcp -j REDIRECT --to-port ${ISTIO_PROXY_PORT} | |||
| # To make sure when pod A wants to talk to service A, which is backed by pod A, | |||
| # the traffic is going through proxy twice, client-side and server-side. | |||
| # lo traffic doesn't go through PREROUTING, so needed to be processed in OUTPUT. | |||
There was a problem hiding this comment.
can you rephrase the comment to say pod belonging to service "a" rather than pod "a"?
There was a problem hiding this comment.
can you expand on the meaning on the rule? e.g. concretely in terms of TCP quadruples and POD_IPs
docker/prepare_proxy.sh
Outdated
| # the traffic is going through proxy twice, client-side and server-side. | ||
| # lo traffic doesn't go through PREROUTING, so needed to be processed in OUTPUT. | ||
| iptables -t nat -A OUTPUT -p tcp -j REDIRECT -o lo \ | ||
| ! -d 127.0.0.1/32 --to-port ${ISTIO_PROXY_PORT} |
There was a problem hiding this comment.
how is this not capturing traffic from proxy?
There was a problem hiding this comment.
this is intended to capture traffic from (client-side) proxy to (server-side) proxy, and exclude (server-side) proxy to app by excluding -d 127.0.0.1/32.
There was a problem hiding this comment.
I think I've convinced myself this is okay. I double checked with "iptables -t nat -S -v" in a standalone docker container that the appropriate rules are triggered for the 3 cases (external inbound traffic, implicit loopback, explicit loopback/proxy traffic).
Rule #1 - external traffic is redirected to proxy port
Rule #2 - locally generated traffic sent on loopback interface (because traffic was routed locally) and destination IP is not explicitly the loopback IP (e.g. 172.17.0.2 and not 127.0.0.1) is redirected back to proxy port. This should include any proxy generated traffic which is necessary to make (app => proxy => proxy => app) work as Lizan described.
Rule #3 - locally generated traffic not sent explicitly to loopback IP (i.e. proxy aware) or not from the proxy itself is redirected proxy port.
|
Thanks @ayj! added comments based on yours to the shell script for each iptables rule. |
|
Jenkins job manager/presubmit passed |
|
Jenkins job manager/e2e-smoketest passed |
|
Jenkins job manager/presubmit passed |
|
Jenkins job manager/e2e-smoketest passed |
Previously for loopback traffic the iptables rule routed it as:
app -> proxy -> app
This will route it as:
app -> proxy (client-side) -> proxy (server-side) -> app
So proxy can upgrade the protocol between proxies (e.g. HTTP/2, TLS)
cc @myidpt @kyessenov
Closes #103 .