Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

use-cases: add monitor process credentials at syscalls #895

Merged
merged 3 commits into from
Jul 6, 2023
Merged

use-cases: add monitor process credentials at syscalls #895

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
f14b05e
docs: add linux process credentials page
tixxdz Jul 5, 2023
92d3a70
use-cases: add monitor setuid() syscalls family tracingpolicy
tixxdz Jul 5, 2023
009d240
use-case: add monitor process credentials at syscalls
tixxdz Jul 5, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
65 changes: 65 additions & 0 deletions docs/content/en/docs/tetragon-events/linux-process-credentials/_index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
---
title: "Linux Process Credentials"
weight: 3
icon: "reference"
description: "Monitor Linux Process Credentials"
---

On Linux each process has various associated user, group IDs, capabilities,
secure management flags, keyring, LSM security that are used part of the
security checks upon acting on other objects. These are called the task
privileges or
[process credentials](https://www.kernel.org/doc/html/latest/security/credentials.html#task-credentials).

Changing the process credentials is a standard operation to perform privileged
actions or to execute commands as another user. The obvious example is
[sudo](https://www.sudo.ws/) that allows to gain high privileges and run commands
as root or another user. An other example is services or containers that can
gain high privileges during execution to perform restricted operations.


## Process Credentials:

Linux process credentials include the following types:

* Traditional UNIX credentials:

Real User ID
Real Group ID
Effective, Saved and FS User ID
Effective, Saved and FS Group ID
Supplementary groups

* Linux Capabilities

Set of permitted capabilities
Set of inheritable capabilities
Set of effective capabilities
Capability bounding set

The effective capabilities are the ones that a task is actually allowed to make use of itself.

The inheritable capabilities are the ones that may get passed across execve().

The bounding set limits the capabilities that may be inherited across execve(), especially when a binary is executed that will execute as UID 0.

* Secure management flags (securebits).

These govern the way the UIDs/GIDs and capabilities are manipulated and inherited over certain operations such as execve().

* [Linux Security Module (LSM)](https://www.kernel.org/doc/html/latest/admin-guide/LSM/index.html)

The LSM framework provides a mechanism for various security checks to be hooked by new kernel extensions.
Tasks can have extra controls part of LSM on what operations they
are allowed to perform.



Process credentials monitoring is a good practice to identify programs
running with high privileges. Tetragon is able to observe process credentials at
different layers of the operating system. Each layer will generate a number of events,
where the upper layer that is the user space layer may generate a low number
of events, where the lower kernel layers could generate a high number of events.

Depending on the use case, users should chose the right layer at which they want to
monitor process credentials and apply the corresponding [Tracing Policies](/docs/reference/tracing-policy).
242 changes: 242 additions & 0 deletions ...ontent/en/docs/tetragon-events/linux-process-credentials/syscalls-monitoring.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,242 @@
---
title: "Use case: Monitor change Credentials system calls"
weight: 2
icon: "reference"
description: "Monitor change Credentials System calls"
---

Tetragon allows to hook at the system calls that directly manipulate the credentials. This allows to determine which process is trying to change its own credentials and the new credentials that could be applied by the kernel.

This answers the questions:

> Which process or container is trying to change its own UIDs/GIDs in my cluster?

> Which process or container is trying to change its own capabilities in my cluster?


Before going forward, verify that all pods are up and running, ensure you deploy our Demo Application to explore the Security Observability Events:

```bash
kubectl create -f https://raw.githubusercontent.com/cilium/cilium/v1.11/examples/minikube/http-sw-app.yaml
```

It might take several seconds for some pods until they satisfy all the dependencies:

```bash
kubectl get pods -A
NAMESPACE NAME READY STATUS RESTARTS AGE
default deathstar-54bb8475cc-6c6lc 1/1 Running 0 2m54s
default deathstar-54bb8475cc-zmfkr 1/1 Running 0 2m54s
default tiefighter 1/1 Running 0 2m54s
default xwing 1/1 Running 0 2m54s
kube-system tetragon-sdwv6 2/2 Running 0 27m
```

## Monitor UIDs/GIDs credential changes

We use the [process.credentials.changes.at.syscalls](https://raw.githubusercontent.com/cilium/tetragon/main/examples/tracingpolicy/process-credentials/process.credentials.changes.at.syscalls.yaml) Tracing Policy that hooks the [setuid()](https://man7.org/linux/man-pages/man2/setuid.2.html) system calls family:

setuid(), setgid(), setfsuid(), setfsgid(), setreuid(), setregid(), setresuid() and setresgid().


So let's apply the [process.credentials.changes.at.syscalls](https://raw.githubusercontent.com/cilium/tetragon/main/examples/tracingpolicy/process-credentials/process.credentials.changes.at.syscalls.yaml) Tracing Policy.

```bash
kubectl apply -f https://raw.githubusercontent.com/cilium/tetragon/main/examples/tracingpolicy/process-credentials/process.credentials.changes.at.syscalls.yaml

```

Then we start monitoring for events with `tetra` cli:
```bash
kubectl exec -it -n kube-system ds/tetragon -c tetragon -- tetra getevents
```

In another terminal, kubectl exec into the xwing pod:

```bash
kubectl exec -it xwing -- /bin/bash
```

and execute [su](https://en.wikipedia.org/wiki/Su_(Unix)) as this will call the related setuid() system calls:

```bash
su root
```

The `tetra` cli will generate the following [ProcessKprobe](https://tetragon.cilium.io/docs/reference/grpc-api/#processkprobe) events:

```json
{
"process_kprobe": {
"process": {
"exec_id": "a2luZC1jb250cm9sLXBsYW5lOjQwNzk4ODc2MDI2NTk4OjEyNTc5OA==",
"pid": 125798,
"uid": 0,
"cwd": "/",
"binary": "/bin/su",
"arguments": "root",
"flags": "execve rootcwd clone",
"start_time": "2023-07-05T19:14:30.918693157Z",
"auid": 4294967295,
"pod": {
"namespace": "default",
"name": "xwing",
"container": {
"id": "containerd://55936e548de63f77ceb595d64966dd8e267b391ff0ef63b26c17eb8c2f6510be",
"name": "spaceship",
"image": {
"id": "docker.io/tgraf/netperf@sha256:8e86f744bfea165fd4ce68caa05abc96500f40130b857773186401926af7e9e6",
"name": "docker.io/tgraf/netperf:latest"
},
"start_time": "2023-07-05T18:45:16Z",
"pid": 19
},
"pod_labels": {
"app.kubernetes.io/name": "xwing",
"class": "xwing",
"org": "alliance"
}
},
"docker": "55936e548de63f77ceb595d64966dd8",
"parent_exec_id": "a2luZC1jb250cm9sLXBsYW5lOjQwNzk1NjYyMDM3MzMyOjEyNTc5Mg==",
"refcnt": 1,
"tid": 125798
},
"parent": {
"exec_id": "a2luZC1jb250cm9sLXBsYW5lOjQwNzk1NjYyMDM3MzMyOjEyNTc5Mg==",
"pid": 125792,
"uid": 0,
"cwd": "/",
"binary": "/bin/bash",
"flags": "execve rootcwd clone",
"start_time": "2023-07-05T19:14:27.704703805Z",
"auid": 4294967295,
"pod": {
"namespace": "default",
"name": "xwing",
"container": {
"id": "containerd://55936e548de63f77ceb595d64966dd8e267b391ff0ef63b26c17eb8c2f6510be",
"name": "spaceship",
"image": {
"id": "docker.io/tgraf/netperf@sha256:8e86f744bfea165fd4ce68caa05abc96500f40130b857773186401926af7e9e6",
"name": "docker.io/tgraf/netperf:latest"
},
"start_time": "2023-07-05T18:45:16Z",
"pid": 13
},
"pod_labels": {
"app.kubernetes.io/name": "xwing",
"class": "xwing",
"org": "alliance"
}
},
"docker": "55936e548de63f77ceb595d64966dd8",
"parent_exec_id": "a2luZC1jb250cm9sLXBsYW5lOjQwNzk1NjE2MTU0NzA2OjEyNTc4Mw==",
"refcnt": 2,
"tid": 125792
},
"function_name": "__x64_sys_setgid",
"args": [
{
"int_arg": 0
}
],
"action": "KPROBE_ACTION_POST"
},
"node_name": "kind-control-plane",
"time": "2023-07-05T19:14:30.918977160Z"
}
{
"process_kprobe": {
"process": {
"exec_id": "a2luZC1jb250cm9sLXBsYW5lOjQwNzk4ODc2MDI2NTk4OjEyNTc5OA==",
"pid": 125798,
"uid": 0,
"cwd": "/",
"binary": "/bin/su",
"arguments": "root",
"flags": "execve rootcwd clone",
"start_time": "2023-07-05T19:14:30.918693157Z",
"auid": 4294967295,
"pod": {
"namespace": "default",
"name": "xwing",
"container": {
"id": "containerd://55936e548de63f77ceb595d64966dd8e267b391ff0ef63b26c17eb8c2f6510be",
"name": "spaceship",
"image": {
"id": "docker.io/tgraf/netperf@sha256:8e86f744bfea165fd4ce68caa05abc96500f40130b857773186401926af7e9e6",
"name": "docker.io/tgraf/netperf:latest"
},
"start_time": "2023-07-05T18:45:16Z",
"pid": 19
},
"pod_labels": {
"app.kubernetes.io/name": "xwing",
"class": "xwing",
"org": "alliance"
}
},
"docker": "55936e548de63f77ceb595d64966dd8",
"parent_exec_id": "a2luZC1jb250cm9sLXBsYW5lOjQwNzk1NjYyMDM3MzMyOjEyNTc5Mg==",
"refcnt": 1,
"tid": 125798
},
"parent": {
"exec_id": "a2luZC1jb250cm9sLXBsYW5lOjQwNzk1NjYyMDM3MzMyOjEyNTc5Mg==",
"pid": 125792,
"uid": 0,
"cwd": "/",
"binary": "/bin/bash",
"flags": "execve rootcwd clone",
"start_time": "2023-07-05T19:14:27.704703805Z",
"auid": 4294967295,
"pod": {
"namespace": "default",
"name": "xwing",
"container": {
"id": "containerd://55936e548de63f77ceb595d64966dd8e267b391ff0ef63b26c17eb8c2f6510be",
"name": "spaceship",
"image": {
"id": "docker.io/tgraf/netperf@sha256:8e86f744bfea165fd4ce68caa05abc96500f40130b857773186401926af7e9e6",
"name": "docker.io/tgraf/netperf:latest"
},
"start_time": "2023-07-05T18:45:16Z",
"pid": 13
},
"pod_labels": {
"app.kubernetes.io/name": "xwing",
"class": "xwing",
"org": "alliance"
}
},
"docker": "55936e548de63f77ceb595d64966dd8",
"parent_exec_id": "a2luZC1jb250cm9sLXBsYW5lOjQwNzk1NjE2MTU0NzA2OjEyNTc4Mw==",
"refcnt": 2,
"tid": 125792
},
"function_name": "__x64_sys_setuid",
"args": [
{
"int_arg": 0
}
],
"action": "KPROBE_ACTION_POST"
},
"node_name": "kind-control-plane",
"time": "2023-07-05T19:14:30.918990583Z"
}
```

In addition to the Kubernetes Identity and process metadata from exec events, [ProcessKprobe](https://tetragon.cilium.io/docs/reference/grpc-api/#processkprobe) events contain the arguments of the observed system call. In the above case they are:

* `function_name`: that is the system call, `__x64_sys_setuid` or `__x64_sys_setgid`

* `int_arg`: is the uid or gid to use, here it is 0 which corresponds to the root user.


To disable the [process.credentials.changes.at.syscalls Tracing Policy](https://raw.githubusercontent.com/cilium/tetragon/main/examples/tracingpolicy/process-credentials/process.credentials.changes.at.syscalls.yaml) run:

```bash
kubectl delete -f https://raw.githubusercontent.com/cilium/tetragon/main/examples/tracingpolicy/process-credentials/process.credentials.changes.at.syscalls.yaml
```