-
Notifications
You must be signed in to change notification settings - Fork 1.6k
Add blog on inbound forwarding changes #9150
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
e4d2c87
6fd17d6
efc4d36
e2a6056
c3173da
45d4aca
81d8a39
d7a7d14
f670bf6
ac78903
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,91 @@ | ||
| --- | ||
| title: "Upcoming networking changes in Istio 1.10" | ||
| description: Understanding the upcoming changes to Istio networking, how they may impact your cluster, and what action to take. | ||
| publishdate: 2021-04-15 | ||
| attribution: "John Howard (Google)" | ||
| --- | ||
|
|
||
| ## Background | ||
|
|
||
| While Kubernetes networking is customizable, a typical pod's network will look like this: | ||
|
|
||
| {{< image width="75%" link="./pod.svg" caption="A pod's network" >}} | ||
|
|
||
| An application may choose to bind to either the loopback interface `lo` (typically binding to `127.0.0.1`), or the pods network interface `eth0` (typically to the pod's IP), or both (typically binding to `0.0.0.0`). | ||
|
|
||
| Binding to `lo` allows calls such as `curl localhost` to work from within the pod. | ||
| Binding to `eth0` allows calls to the pod from other pods. | ||
|
|
||
| Typically, an application will bind to both. | ||
| However, applications which have internal logic, such as an admin interface may choose to bind to only `lo` to avoid access from other pods. | ||
| Additionally, some applications, typically stateful applications, choose to bind only to `eth0`. | ||
|
|
||
| ## Current behavior | ||
|
|
||
| In Istio prior to release 1.10, the Envoy proxy, running in the same pod as the application, binds to the `eth0` interface and redirects all inbound traffic to the `lo` interface. | ||
|
|
||
| {{< image width="75%" link="./current.svg" caption="A pod's network with Istio today" >}} | ||
|
|
||
| This has two important side effects that cause the behavior to differ from standard Kubernetes: | ||
|
|
||
| * Applications binding only to `lo` will receive traffic from other pods, when otherwise this is not allowed. | ||
| * Applications binding only to `eth0` will not receive traffic. | ||
|
|
||
| Applications that bind to both interfaces (which is typical) will not be impacted. | ||
|
|
||
| ## Future behavior | ||
|
|
||
| Starting with Istio 1.10, the networking behavior is changed to align with the standard behavior present in Kubernetes. | ||
|
|
||
| {{< image width="75%" link="./planned.svg" caption="A pod's network with Istio in the future" >}} | ||
|
|
||
| Here we can see that the proxy no longer redirects the traffic to the `lo` interface, but instead forwards it to the application on `eth0`. | ||
| As a result, the standard behavior of Kubernetes is retained, but we still get all the benefits of Istio. | ||
| This change allows Istio to get closer to its goal of being a drop-in transparent proxy that works with existing workloads with [zero configuration](/blog/2021/zero-config-istio/). | ||
| Additionally, it avoids unintended exposure of applications binding only to `lo`. | ||
|
|
||
| ## Am I impacted? | ||
|
|
||
| For new users, this change should only be an improvement. | ||
| However, if you are an existing user, you may have come to depend on the old behavior, intentionally or accidentally. | ||
|
|
||
| To help detect these situations, we have added a check to find pods that will be impacted. | ||
| You can run the `istioctl experimental precheck` command to get a report of any pods binding to `lo` on a port exposed in a `Service`. | ||
| This command is available in Istio 1.10+. | ||
| **Without action, these ports will no longer be accessible upon upgrade.** | ||
|
|
||
| {{< text bash >}} | ||
| $ istioctl experimental precheck | ||
| Error [IST0143] (Pod echo-local-849647c5bd-g9wxf.default) Port 443 is exposed in a Service but listens on localhost. It will not be exposed to other pods. | ||
| Error [IST0143] (Pod echo-local-849647c5bd-g9wxf.default) Port 7070 is exposed in a Service but listens on localhost. It will not be exposed to other pods. | ||
| Error: Issues found when checking the cluster. Istio may not be safe to install or upgrade. | ||
| See https://istio.io/latest/docs/reference/config/analysis for more information about causes and resolutions. | ||
| {{< /text >}} | ||
|
|
||
| ### Migration | ||
|
|
||
| If you are currently binding to `lo`, you have a few options: | ||
|
|
||
| * Switch your application to bind to all interfaces (`0.0.0.0` or `::`). | ||
| * Explicitly configure the port using the [`Sidecar` ingress configuration](/docs/reference/config/networking/sidecar/#IstioIngressListener) to send to `lo`, preserving the old behavior. | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is the configuration for the Walking through a migration scenario, if I create the sidecar resource for my istio 1.10 cp, and also created my app with 1.10's data plane, would that sidecar resource break traffic for my app with 1.9's data plane? (Correct me if I am wrong, I think our sidecar resource is global for a ns regardless of which control plane revision?) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Both are required fields The steps would be:
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yea was trying to see if we could isolate the sidecar only for 1.10. I don't think we could. In that case, user should validate sidecar resource has no impact to the app (1.9 based) first before upgrading to 1.10. Note: some of the stateful apps, we ask users to change certain configs to listen on localhost, probably a good time to revert that change after testing. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You can with istio.io/rev, but there is no real reason to. I already changed that in https://preliminary.istio.io/latest/about/faq#applications. |
||
|
|
||
| For example, to configure request to be sent to `localhost` for the `ratings` application: | ||
|
|
||
| {{< text yaml >}} | ||
| apiVersion: networking.istio.io/v1beta1 | ||
| kind: Sidecar | ||
| metadata: | ||
| name: ratings | ||
| spec: | ||
| workloadSelector: | ||
| labels: | ||
| app: ratings | ||
| ingress: | ||
| - port: | ||
| number: 8080 | ||
| protocol: HTTP | ||
| name: http | ||
| defaultEndpoint: 127.0.0.1:8080 | ||
| {{< /text >}} | ||
|
|
||
| * Disable the change entirely with the `PILOT_ENABLE_INBOUND_PASSTHROUGH=false` environment variable in Istiod, to enable the same behavior as prior to Istio 1.10. This option will be removed in the future. | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What version of
istioctlhas these checks? Is it already in 1.9? If we post the blog before Istio 1.10 is released, you should mention what version they need to use to do the checking.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
1.10 only unfortunately, and not easy to backport. Ill add a note