Skip to content

Commit

Permalink
tetragon: create Install section and move baremetal install out
Browse files Browse the repository at this point in the history
Create an installation section and move systemd install there. This further
simplifies 'getting started' section.

Signed-off-by: John Fastabend <john.fastabend@gmail.com>
  • Loading branch information
jrfastab committed Oct 12, 2023
1 parent 33c3f33 commit d12ba5b
Show file tree
Hide file tree
Showing 14 changed files with 534 additions and 59 deletions.
205 changes: 180 additions & 25 deletions docs/content/en/docs/getting-started/enforcement.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,41 +5,196 @@ description: "Policy Enforcement"
---

This adds a network and file policy enforcement on top of execution, file tracing
and networking policy already deployed in the quick start. In this use case we
use a namespace and pod labels to limit the scope of where the network, file
and some security policies will be applied. This highlights two important concepts
of Tetragon. First in kernel filter provides performance advantages, but also allows for
enforcing policies inline with the action. Second, by including kubernetes
filters, such as namespace and labels we can segment a policy to apply to
targeted pods. For implementation details see Enforcement section and for
modifying and creating additional policies see Tracing Policies.

# Enforcement

To apply the policy

{{< tabpane >}}
{{< tab header="K8s" >}}
kubectl apply -f tbd.base-enforce.yaml
{{< /tab >}}
{{< tab header="Docker" >}}
{{< /tab >}}
{{< tab header="Systemd" >}}
and networking policy already deployed in the quick start. In this use case we use
a namespace filter to limit the scope of the enforcement policy to just the 'darkstar'
cluster we installed the demo application in from the
[Quick Kubernetes Install]({{< ref "docs/getting-started/install-k8s" >}}).

This highlights two important concepts of Tetragon. First in kernel filtering
provides a key performance improvement by limiting events from kernel to user
space. But, also allows for enforcing policies in the kernel. By issueing a
SIGKILL to the binary at this point the application will be stopped from
continuing to run. If the operation is triggered through a syscall this means
the application will not return from the syscall and will be terminated.

Second, by including kubernetes filters, such as namespace and labels we can
segment a policy to apply to targeted namespaces and pods. This is critical
for effective policy segmentation.

For implementation details see the [Enforcement]({{< ref "/docs/concepts/enforcement" >}})
section.

## Kubernetes Enforcement

The following section is layed out with the following: A guide to promote the
network observation policy that observer all network traffic egressing the
cluster to enforce this policy. A guide to promote the file access monitoring
policy to block write and read operations to sensitive files.

### Kubernetes Block TCP Connect outside Cluster

First we will deploy the [Network Monitoring]({{< ref "docs/getting-started/network" >}})
policy with enforcement on. For this case the policy is written to only apply
against the 'empire' namespace. This limits the scope of the policy for the
getting started guide.

Ensure we have the proper Pod CIDRs

```shell-session
export PODCIDR=`kubectl get nodes -o jsonpath='{.items[*].spec.podCIDR}'`
```

and Service CIDRs configured.

{{< tabpane text=true >}}
{{% tab GKE %}}

```shell-session
export SERVICECIDR=$(gcloud container clusters describe ${NAME} --zone ${ZONE} --project ${PROJECT} | awk '/servicesIpv4CidrBlock/ { print $2; }')
```
{{% /tab %}}

{{% tab Kind %}}
```shell-session
export SERVICECIDR=$(kubectl describe pod -n kube-system kube-apiserver-kind-control-plane | awk -F= '/--service-cluster-ip-range/ {print $2; }')
```
{{< /tab >}}
{{< /tabpane >}}

Then we can apply the egress cluster enforcement policy

```shell-session
wget http://github.com/cilium/tetragon/quickstart/network_egress_cluster_enforce.yaml
envsubst < network_egress_cluster_enforce.yaml | kubectl apply -n default -f -
```

With the enforcement policy applied we can attach tetra to observe events again,

```shell-session
kubectl exec -ti -n kube-system ds/tetragon -c tetragon -- tetra getevents -o compact --pods xwing
```

And once again execute a curl command in the xwing,

```shell-session
kubectl exec -ti xwing -- bash -c 'curl https://ebpf.io/applications/#tetragon'
```

The command returns an error code because the egress TCP connects are blocked shown here
```shell-session
$ kubectl exec -ti xwing -- bash -c 'curl https://ebpf.io/applications/#tetragon'
command terminated with exit code 137
```

connect inside the cluster will work as expected,

```shell-session
kubectl exec -ti xwing -- bash -c 'curl -s -XPOST deathstar.default.svc.cluster.local/v1/request-landing'
```

The Tetra CLI will print the curl and annotate that the process that was issued a Sigkill. The successful internal connect is filtered and will not be shown.

``` shell
🚀 process default/xwing /bin/bash -c "curl https://ebpf.io/applications/#tetragon"
🚀 process default/xwing /usr/bin/curl https://ebpf.io/applications/#tetragon
🔌 connect default/xwing /usr/bin/curl tcp 10.32.0.28:45200 -> 104.198.14.52:443
💥 exit default/xwing /usr/bin/curl https://ebpf.io/applications/#tetragon SIGKILL
🚀 process default/xwing /bin/bash -c "curl -s -XPOST deathstar.default.svc.cluster.local/v1/request-landing"
🚀 process default/xwing /usr/bin/curl -s -XPOST deathstar.default.svc.cluster.local/v1/request-landing
```

The enforces TCP connects see [Enforce Sandbox]({{< ref "#enforce-common-security-policy" >}}) below to further restrict possible
workaround such as writing through /dev devices and raw sockets application may
attempt.

### Enforce File Access Monitoring

The following extends the example from [File Access Monitoring]({{< ref "docs/getting-started/file-events" >}})
with enforcement to ensure sensitive files are not read. The policy used is the
[`file-monitoring-enforce.yaml`](https://github.com/cilium/tetragon/blob/main/quickstart/file-monitoring-enforce.yaml)
it can be reviewed and extended as needed. The only difference between the
observation policy and the enforce policy is the addition of an action block
to sigkill the application and return an error on the op.

To apply the policy.

{{< tabpane lang=shell-session >}}

{{< tab Kubernetes >}}
kubectl delete -f http://github.com/cilium/tetragon/quickstart/file_monitoring.yaml
kubectl apply -f http://github.com/cilium/tetragon/quickstart/file_monitoring_enforce.yaml
{{< /tab >}}
{{< tab Docker >}}
wget http://github.com/cilium/tetragon/quickstart/file-monitoring.yaml
docker stop tetragon-container
docker run --name tetragon-container --rm --pull always \
--pid=host --cgroupns=host --privileged \
-v ${PWD}/file_monitoring.yaml:/etc/tetragon/tetragon.tp.d/file_monitoring_enforce.yaml \
-v /sys/kernel/btf/vmlinux:/var/lib/tetragon/btf \
quay.io/cilium/tetragon-ci:latest
{{< /tab >}}
{{< /tabpane >}}

With the file applied we can attach tetra to observe events again,

{{< tabpane lang=shell-session >}}
{{< tab Kubernetes >}}
kubectl exec -ti -n kube-system ds/tetragon -c tetragon -- tetra getevents -o compact --pods xwing
{{< /tab >}}
{{< tab Docker >}}
docker exec tetragon-container tetra getevents -o compact
{{< /tab >}}
{{< /tabpane >}}

Then reading a sensitive file,

{{< tabpane lang=shell-session >}}
{{< tab Kubernetes >}}
kubectl exec -ti xwing -- bash -c 'cat /etc/shadow'
{{< /tab >}}
{{< tab Docker >}}
cat /etc/shadow
{{< /tab >}}
{{< /tabpane >}}

The command will fail with an error code because this is one of our sensitive files,
```
kubectl exec -ti xwing -- bash -c 'curl https://ebpf.io/applications/#tetragon
$ kubectl exec -ti xwing -- bash -c 'cat /etc/shadow'
command terminated with exit code 137
```

And once again execute a curl command in the xwing,
This will generate a read event (Docker events will omit Kubernetes metadata),

```shell-session
🚀 process default/xwing /bin/bash -c "cat /etc/shadow"
🚀 process default/xwing /bin/cat /etc/shadow
📚 read default/xwing /bin/cat /etc/shadow
📚 read default/xwing /bin/cat /etc/shadow
📚 read default/xwing /bin/cat /etc/shadow
💥 exit default/xwing /bin/cat /etc/shadow SIGKILL
```
kubectl exec -ti xwing -- bash -c 'curl https://ebpf.io/applications/#tetragon

Writes and reads to files not part of the enforced file policy will not be
impacted.

```shell-session
🚀 process default/xwing /bin/bash -c "echo foo >> bar; cat bar"
🚀 process default/xwing /bin/cat bar
💥 exit default/xwing /bin/cat bar 0
💥 exit default/xwing /bin/bash -c "echo foo >> bar; cat bar" 0
```

The CLI will print the exec tracing and file access as before, but will additional show the network connection outside the K8s cluster.
# What's next

The completes the quick start guides. At this point we should be able to observe
execution traces in a Kubernetes cluster and extend the base deployment of
Tetragon with policies to observe and enforce different aspects of a Kubernetes
system.

The rest of the docs provide further documentation about installation and
using policies. Some useful links:

#
To explore details of writing and implementing policies the [Concepts]({{< ref "/docs/concepts" >}}) is a good jumping off point.
For installation into production environments we recommend reviewing [Advanced Installations]({{< ref "docs/installation" >}}).
For a more in depth discussion on Tetragon overhead and how we measure system load try [Benchmarks]({{< ref "docs/benchmarks" >}}).
Finally [Use Cases]({{< ref "docs/use-cases" >}}) and [Tutorials]({{< ref "docs/tutorials" >}}) cover different uses and deployment concerns related to Tetragon.
20 changes: 13 additions & 7 deletions docs/content/en/docs/getting-started/execution.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,16 +23,21 @@ kubectl exec -ti -n kube-system ds/tetragon -c tetragon -- tetra getevents -o co
{{< tab Docker >}}
docker exec tetragon-container tetra getevents -o compact
{{< /tab >}}
{{< tab Systemd >}}
{{< /tab >}}
{{< /tabpane >}}

This will print a compact form of the exec logs. For an example we do the following
with the demo application.

```shell-session

{{< tabpane lang=shell-session >}}
{{< tab Kubernetes >}}
kubectl exec -ti xwing -- bash -c 'curl https://ebpf.io/applications/#tetragon
```
{{< /tab >}}
{{< tab Docker >}}
curl https://ebpf.io/applications/#tetragon
{{< /tab >}}
{{< /tabpane >}}

The CLI will print a compact form of the event to the terminal

```shell-session
Expand All @@ -53,11 +58,11 @@ kubectl exec -ti -n kube-system ds/tetragon -c tetragon -- tetra getevents
{{< tab Docker >}}
docker exec tetragon-container tetra getevents
{{< /tab >}}
{{< tab Systemd >}}
{{< /tab >}}
{{< /tabpane >}}

This will include a lot more details related the binary and event. A full example of the above curl is hown here,
In a Kubernetes environment this will include the Kubernetes metadata include the Pod, Container, Namespaces, and
Labels among other useful metadata.

<details><summary>Process execution event</summary>
<p>
Expand Down Expand Up @@ -139,8 +144,9 @@ This will include a lot more details related the binary and event. A full exampl
}

```

</p>
</details>

## What's next

Execution events are the most basic event Tetragon can produce. To see how to use tracing policies to enable file monitoring see the [File Access Monitoring]({{< ref "/docs/getting-started/file-events" >}})quickstart. To see a network policy check the [Networking Monitoring]({{< ref "/docs/getting-started/network" >}}) quickstart.
41 changes: 31 additions & 10 deletions docs/content/en/docs/getting-started/file-events.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,33 +14,51 @@ remains low even on busy systems.

The following extends the example from Execution Tracing with a policy to
monitor sensitive files in Linux. The policy used is the [`file-monitoring.yaml`](https://github.com/cilium/tetragon/blob/main/quickstart/file-monitoring.yaml) it can be reviewed and extended
as needed. However, files monitored here serve as a good base set of files.
as needed. Files monitored here serve as a good base set of files.

To apply the policy
To apply the policy Kubernetes uses a CRD that can be applied with kubectl.
Uses the same YAML configuration as Kuberenetes, but loaded through a file
on disk.

{{< tabpane lang=shell-session >}}

{{< tab Kubernetes >}}
kubectl apply -f http://github.com/cilium/tetragon/quickstart/file-monitoring.yaml
{{< /tab >}}
{{< tab Docker >}}
{{< /tab >}}
{{< tab Systemd >}}
{{< /tab >}}
wget http://github.com/cilium/tetragon/quickstart/file-monitoring.yaml
docker stop tetragon-container
docker run --name tetragon-container --rm --pull always \
--pid=host --cgroupns=host --privileged \
-v ${PWD}/file_monitoring.yaml:/etc/tetragon/tetragon.tp.d/file_monitoring.yaml \
-v /sys/kernel/btf/vmlinux:/var/lib/tetragon/btf \
quay.io/cilium/tetragon-ci:latest
{{< /tab >}}
{{< /tabpane >}}

With the file applied we can attach tetra to observe events again,

```shell-session
{{< tabpane lang=shell-session >}}
{{< tab Kubernetes >}}
kubectl exec -ti -n kube-system ds/tetragon -c tetragon -- tetra getevents -o compact --pods xwing
```
{{< /tab >}}
{{< tab Docker >}}
docker exec tetragon-container tetra getevents -o compact
{{< /tab >}}
{{< /tabpane >}}

Then reading a sensitive file,

```shell-session
{{< tabpane lang=shell-session >}}
{{< tab Kubernetes >}}
kubectl exec -ti xwing -- bash -c 'cat /etc/shadow'
```
{{< /tab >}}
{{< tab Docker >}}
cat /etc/shadow
{{< /tab >}}
{{< /tabpane >}}

This will generate a read event,
This will generate a read event (Docker events will omit Kubernetes metadata),

```shell-session
🚀 process default/xwing /bin/bash -c "cat /etc/shadow"
Expand All @@ -59,3 +77,6 @@ Attempts to write in sensitive directories will similar create an event. For exa
```

# What's next

To explore tracing policies for networking try the [Networking Monitoring]({{< ref "/docs/getting-started/network" >}}) quickstart.
To dive into the details of policies and events please see [Concepts]({{< ref "docs/concepts" >}}) section.
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: "Try Tetragon locally "
title: "Quick Local Docker Install"
weight: 1
description: "Discover and experiment with Tetragon on your local Linux host"
---
Expand Down Expand Up @@ -30,3 +30,7 @@ docker run --name tetragon-container --rm --pull always \

This will start Tetragon in a privileged container. Priviliges are required
to load and attach BPF programs. See Installation section for more details.

## Whats Next

[Check for execution events.]({{< ref "/docs/getting-started/execution" >}}).
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: "Try Tetragon on Kubernetes"
title: "Quick Kubernetes Install"
weight: 1
description: "Discover and experiment with Tetragon in a kubernetes environment"
---
Expand Down

0 comments on commit d12ba5b

Please sign in to comment.