Skip to content

Commit

Permalink
Egress blog part 2 (#4232)
Browse files Browse the repository at this point in the history
* add the second part of the series about secure egress traffic control in Istio (#4196)

* requirements for your system -> requirements for a system for egress traffic control

* add links from part 1 to part 2

* add istio-identity to .spelling

* add gateway and tls as keywords

Co-Authored-By: Rigs Caballero <grca@google.com>

* This is -> Welcome to, a new series -> our new series

Co-Authored-By: Rigs Caballero <grca@google.com>

* an egress traffic control system -> a secure control system for egress traffic

Co-Authored-By: Rigs Caballero <grca@google.com>

* for controlling egress traffic securely ->to securely control the egress traffic,  prevents the -> can help you prevent such

Co-Authored-By: Rigs Caballero <grca@google.com>

* Egress traffic control by Istio -> Secure control of egress traffic in Istio

Co-Authored-By: Rigs Caballero <grca@google.com>

* add bullets regarding security measures for Istio control plane

Co-Authored-By: Rigs Caballero <grca@google.com>

* you can securely monitor the traffic and define security policies on it -> you can securely monitor and define security policies for the traffic

Co-Authored-By: Rigs Caballero <grca@google.com>

* Possible attacks and their prevention -> Preventing possible attacks

Co-Authored-By: Rigs Caballero <grca@google.com>

* e.g. -> like, add a comma, split a sentence

Co-Authored-By: Rigs Caballero <grca@google.com>

* the -> said

Co-Authored-By: Rigs Caballero <grca@google.com>

* remove "for TLS traffic"

it is clear that it is TLS Traffic from TLS origination

Co-Authored-By: Rigs Caballero <grca@google.com>

* monitor SNI and the service account of the source pod -> monitor SNI and the service account of the source pod's TLS traffic

Co-Authored-By: Rigs Caballero <grca@google.com>

* L3 firewall -> an L3 firewall, remove parentheses, provided -> should be provided

* The L3 firewall can have -> you can configure the L3 firewall

Co-Authored-By: Rigs Caballero <grca@google.com>

* from pods only -> only allow. Remove "Note that"

Co-Authored-By: Rigs Caballero <grca@google.com>

* move the diagram right after its introduction

* remove parentheses

Co-Authored-By: Rigs Caballero <grca@google.com>

* emphasize the label (A, B)

Co-Authored-By: Rigs Caballero <grca@google.com>

* policy with regard -> policies as they regard

Co-Authored-By: Rigs Caballero <grca@google.com>

* rewrite the sentence about a compromised pod

Co-Authored-By: Rigs Caballero <grca@google.com>

* traffic must be monitored -> traffic is monitored

Co-Authored-By: Rigs Caballero <grca@google.com>

* Note that application A is allowed -> since application A is allowed

Co-Authored-By: Rigs Caballero <grca@google.com>

* rewrite the sentence about monitoring access of the compromised version of the application

Co-Authored-By: Rigs Caballero <grca@google.com>

* split the sentence about detecting suspicious traffic

Co-Authored-By: Rigs Caballero <grca@google.com>

* rewrite the sentence about thwarting the second goal of the attackers

Co-Authored-By: Rigs Caballero <grca@google.com>

* Istio must enforce -> enforces, forbids access of application A -> forbids application A from accessing

Co-Authored-By: Rigs Caballero <grca@google.com>

* Rewrite the sentence "let's see which attacks"

Co-Authored-By: Rigs Caballero <grca@google.com>

* rewrite the sentence "I hope that"

Co-Authored-By: Rigs Caballero <grca@google.com>

* in the next blog post -> in the next part

Co-Authored-By: Rigs Caballero <grca@google.com>

* remove mentioning wildcard domains

* rewrite the "Secure control of egress traffic in Istio" section

* remove a leftover from suggested changes

* as they regard to egress traffic -> for egress traffic

* convert security policies into bullets

* make the labels (A,B) bold

* remove the sentences about thwarting the second goal

* rewrite the paragraph about which goals of the attackers can be thwarted

* remove a leftover from the previous changes

* such attacks -> the attacks

* rewrite the section about preventing the attacks

* secure egress traffic control -> secure control of egress traffic

* sending HTTP traffic -> sending unencrypted HTTP traffic

* define security policies -> enforce security policies

* change the publish date to July 9

* formatting

Co-Authored-By: Rigs Caballero <grca@google.com>

* Kubernetes Network Policies -> Kubernetes network policies

Co-Authored-By: Rigs Caballero <grca@google.com>

* [an example for Kubernetes Network Policies configuration] -> an example of the [Kubernetes Network Policies configuration]

Co-Authored-By: Rigs Caballero <grca@google.com>

* use proper capitalization and punctuation for bullet 1

Co-Authored-By: Rigs Caballero <grca@google.com>

* use proper capitalization and punctuation for bullet 2

Co-Authored-By: Rigs Caballero <grca@google.com>

* use proper capitalization and punctuation for bullet 3

Co-Authored-By: Rigs Caballero <grca@google.com>

* use proper capitalization and punctuation for bullet 4

Co-Authored-By: Rigs Caballero <grca@google.com>

* check -> verify,  access the destination, mongo1, access mongo1

Co-Authored-By: Rigs Caballero <grca@google.com>

* You can thwart the third goal -> to stop attackers from

Co-Authored-By: Rigs Caballero <grca@google.com>

* remove mentioning anomaly detection

Co-Authored-By: Rigs Caballero <grca@google.com>

* Provide context instead of "after all"

Co-Authored-By: Rigs Caballero <grca@google.com>

* split a long line

Co-Authored-By: Rigs Caballero <grca@google.com>

* connect two sentences

Co-Authored-By: Rigs Caballero <grca@google.com>

* First -> Next

Co-Authored-By: Rigs Caballero <grca@google.com>

* use - instead of * for bulleted lists

* make the first attacker's goal a bullet

Co-Authored-By: Rigs Caballero <grca@google.com>

* make the first attacker's goal a bullet

the previous commit was related to the third goal

Co-Authored-By: Rigs Caballero <grca@google.com>

* make the second attacker's goal a bullet

Co-Authored-By: Rigs Caballero <grca@google.com>

* fix indentation

Co-Authored-By: Rigs Caballero <grca@google.com>

* make the reference to prevention of the first goal a bullet

Co-Authored-By: Rigs Caballero <grca@google.com>

* make the reference to prevention of the second goal a bullet

Co-Authored-By: Rigs Caballero <grca@google.com>

* rephrase the sentence about applying additional security measures

Co-Authored-By: Rigs Caballero <grca@google.com>

* remove leftover from a previous change

Co-Authored-By: Rigs Caballero <grca@google.com>

* that will enforce -> to enforce

Co-Authored-By: Rigs Caballero <grca@google.com>

* split long lines

* rewrite the part about increasing security of the control plane pods

* fix indentation

* fix indentation and remove a leftover from a previous change

* extend the bold font from a single word to a phrase

* rewrite the prevention of the straightforward access and the attacks

* add conclusion after the attacks part

* control planes pods -> control plane pods

* control plane -> Istio control plane

* is able to access it indistinguishable -> is indistinguishable

Co-Authored-By: Rigs Caballero <grca@google.com>

* rewrite the sentence "The choice would mainly depend on"

Co-Authored-By: Rigs Caballero <grca@google.com>

* insure -> ensure

Co-Authored-By: Rigs Caballero <grca@google.com>

* update the publish date to 10-th of July
  • Loading branch information
vadimeisenbergibm authored and mergify[bot] committed Jul 10, 2019
1 parent 2625f05 commit 24f9ca7
Show file tree
Hide file tree
Showing 4 changed files with 140 additions and 7 deletions.
1 change: 1 addition & 0 deletions .spelling
Expand Up @@ -266,6 +266,7 @@ istio.io
istio.io.
Istiofied
IstioMesh
istio-identity
istio-mixer
istio-system
iter8
Expand Down
17 changes: 10 additions & 7 deletions content/blog/2019/egress-traffic-control-in-istio-part-1/index.md
Expand Up @@ -9,12 +9,14 @@ keywords: [traffic-management,egress,security]

This is part 1 in a new series about secure control of egress traffic in Istio that I am going to publish.
In this installment, I explain why you should apply egress traffic control to your cluster, the attacks
involving egress traffic you want to prevent, and the requirements for your system to do so.
involving egress traffic you want to prevent, and the requirements for a system for egress traffic control
to do so.
Once you agree that you should control the egress traffic coming from your cluster, the following questions arise:
What requirements does a system have for secure control of egress traffic? Which is the best solution to fulfill
What is required from a system for secure control of egress traffic? Which is the best solution to fulfill
these requirements? (spoiler: Istio in my opinion)
Future installments will describe the implementation of the secure control of egress traffic in Istio and
compare it with other solutions.
Future installments will describe
[the implementation of the secure control of egress traffic in Istio](/blog/2019/egress-traffic-control-in-istio-part-2/)
and compare it with other solutions.

The most important security aspect for a service mesh is probably ingress traffic. You definitely must prevent attackers
from penetrating the cluster through ingress APIs. Having said that, securing
Expand Down Expand Up @@ -167,8 +169,9 @@ all of these requirements, in particular it is transparent, DNS-aware, and Kuber

## Summary

I hope that you are convinced that controlling egress traffic is important for the security of your cluster. In the
next blogs in this series I will describe the Istio way to perform secure control of egress traffic and compare it with
alternative solutions such as
I hope that you are convinced that controlling egress traffic is important for the security of your cluster. In [the
part 2 of this series](/blog/2019/egress-traffic-control-in-istio-part-2/) I describe the Istio way to perform secure
control of egress traffic.
Next, I will compare it with alternative solutions such as
[Kubernetes Network Policies](https://kubernetes.io/docs/concepts/services-networking/network-policies/) and legacy
egress proxies/firewalls.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
128 changes: 128 additions & 0 deletions content/blog/2019/egress-traffic-control-in-istio-part-2/index.md
@@ -0,0 +1,128 @@
---
title: Secure Control of Egress Traffic in Istio, part 2
subtitle: Use Istio Egress Traffic Control to prevent attacks involving egress traffic
description: Use Istio Egress Traffic Control to prevent attacks involving egress traffic.
publishdate: 2019-07-10
attribution: Vadim Eisenberg (IBM)
keywords: [traffic-management,egress,security,gateway,tls]
---

Welcome to part 2 in our new series about secure control of egress traffic in Istio.
In [the first part in the series](/blog/2019/egress-traffic-control-in-istio-part-1/), I presented the attacks involving
egress traffic and the requirements we collected for a secure control system for egress traffic.
In this installment, I describe the Istio way to securely control the egress traffic, and show how Istio can help you
prevent the attacks.

## Secure control of egress traffic in Istio

To implement secure control of egress traffic in Istio, you must
[direct TLS traffic to external services through an egress gateway](/docs/tasks/traffic-management/egress/egress-gateway/#egress-gateway-for-https-traffic).
Alternatively, you
can [direct HTTP traffic through an egress gateway](/docs/tasks/traffic-management/egress/egress-gateway/#egress-gateway-for-http-traffic)
and [let the egress gateway perform TLS origination](/docs/tasks/traffic-management/egress/egress-gateway-tls-origination/#perform-tls-origination-with-an-egress-gateway).

Both alternatives have their pros and cons, you should choose between them according to your circumstances.
The choice mainly depends on whether your application can send unencrypted HTTP requests and whether your
organization's security policies allow sending unencrypted HTTP requests.
For example, if your application uses some client library that encrypts the traffic without a possibility to cancel the
encryption, you cannot use the option of sending unencrypted HTTP traffic.
The same in the case your organization's security policies do not allow sending unencrypted HTTP requests
**inside the pod** (outside the pod the traffic is encrypted by Istio).

If the application sends HTTP requests and the egress gateway performs TLS origination, you can monitor HTTP
information like HTTP methods, headers, and URL paths. You can also
[define policies](/blog/2018/egress-monitoring-access-control) based on said HTTP information. If the application
performs TLS origination, you can
[monitor SNI and the service account](/docs/tasks/traffic-management/egress/egress_sni_monitoring_and_policies/) of the
source pod's TLS traffic, and define policies based on SNI and service accounts.

You must ensure that traffic from your cluster to the outside cannot bypass the egress gateway. Istio cannot enforce it
for you, so you must apply some
[additional security mechanisms](/docs/tasks/traffic-management/egress/egress-gateway/#additional-security-considerations),
for example,
the [Kubernetes network policies](https://kubernetes.io/docs/concepts/services-networking/network-policies/) or an L3
firewall. See an example of the
[Kubernetes network policies configuration](/docs/tasks/traffic-management/egress/egress-gateway/#apply-kubernetes-network-policies).
According to the [Defense in depth](https://en.wikipedia.org/wiki/Defense_in_depth_(computing)) concept, the more
security mechanisms you apply for the same goal, the better.

You must also ensure that Istio control plane and the egress gateway cannot be compromised. While you may have hundreds
or thousands of application pods in your cluster, there are only a dozen of Istio control plane pods and the gateways.
You can and should focus on protecting the control plane pods and the gateways, since it is easy (there is a small
number of pods to protect) and it is most crucial for the security of your cluster.
If attackers compromise the control plane or the egress gateway, they could violate any policy.

You might have multiple tools to protect the control plane pods, depending on your environment.
The reasonable security measures are:

- Run the control plane pods on nodes separate from the application nodes.
- Run the control plane pods in their own separate namespace.
- Apply the Kubernetes RBAC and network policies to protect the control plane pods.
- Monitor the control plane pods more closely than you do the application pods.

Once you direct egress traffic through an egress gateway and apply the additional security mechanisms,
you can securely monitor and enforce security policies for the traffic.

The following diagram shows Istio's security architecture, augmented with an L3 firewall which is part of the
[additional security mechanisms](/docs/tasks/traffic-management/egress/egress-gateway/#additional-security-considerations)
that should be provided outside of Istio.

{{< image width="80%" link="./SecurityArchitectureWithL3Firewalls.svg" caption="Istio Security Architecture with Egress Gateway and L3 Firewall" >}}

You can configure the L3 firewall trivially to only allow incoming traffic through the Istio ingress gateway and
only allow outgoing traffic through the Istio egress gateway. The Istio proxies of the gateways enforce
policies and report telemetry just as all other proxies in the mesh do.

Now let's examine possible attacks and let me show you how the secure control of egress traffic in Istio prevents them.

## Preventing possible attacks

Consider the following security policies for egress traffic:

- Application **A** is allowed to access `*.ibm.com`, which includes all the external services with URLs matching
`*.ibm.com`.
- Application **B** is allowed to access `mongo1.composedb.com`.
- All egress traffic is monitored.

Suppose the attackers have the following goals:

- Access `*.ibm.com` from your cluster.
- Access `*.ibm.com` from your cluster, unmonitored. The attackers want their traffic to be unmonitored to prevent a
possibility that you will detect the forbidden access.
- Access `mongo1.composedb.com` from your cluster.

Now suppose that the attackers manage to break into one of the pods of application **A**, and try to use the compromised
pod to perform the forbidden access. The attackers may try their luck and access the external services in a
straightforward way. You will react to the straightforward attempts as follows:

- Initially, there is no way to prevent a compromised application **A** to access `*.ibm.com`, because the compromised
pod is indistinguishable from the original pod.
- Fortunately, you can monitor all access to external services, detect suspicious traffic, and thwart attackers from
gaining unmonitored access to `*.ibm.com`. For example, you could apply anomaly detection tools on the
egress traffic logs.
- To stop attackers from accessing `mongo1.composedb.com` from your cluster, Istio will correctly detect the source of
the traffic, application **A** in this case, and verify that it is not allowed to access `mongo1.composedb.com`
according to the security policies mentioned above.

Having failed to achieve their goals in a straightforward way, the malicious actors may resort to advanced attacks:

- **Bypass the container's sidecar proxy** to be able to access any external service directly, without the sidecar's
policy enforcement and reporting. This attack is prevented by a Kubernetes Network Policy or by an L3 firewall that
allow egress traffic to exit the mesh only from the egress gateway.
- **Compromise the egress gateway** to be able to force it to send fake information to the monitoring system or to
disable enforcement of the security policies. This attack is prevented by applying the special security measures to
the egress gateway pods.
- **Impersonate as application B** since application **B** is allowed to access `mongo1.composedb.com`. This attack,
fortunately, is prevented by Istio's [strong identity support](/docs/concepts/security/#istio-identity).

As far as we can see, all the forbidden access is prevented, or at least is monitored and can be prevented later.
If you see other attacks that involve egress traffic or security holes in the current design, we would be happy
[to hear about it](https://discuss.istio.io).

## Summary

Hopefully, I managed to convince you that Istio is an effective tool to prevent attacks involving egress
traffic. In the next part of this series, I compare secure control of egress traffic in Istio with alternative
solutions such as
[Kubernetes Network Policies](https://kubernetes.io/docs/concepts/services-networking/network-policies/) and legacy
egress proxies/firewalls.

0 comments on commit 24f9ca7

Please sign in to comment.