Skip to content

Improve dump FAQ #44714

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Feb 13, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions docs/core/diagnostics/dumps.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,7 @@ A dump is a file that contains a snapshot of the process at the time the dump wa
Dumps can be collected in a variety of ways depending on which platform your app is running on.

> [!NOTE]
> Dumps may contain sensitive information because they can contain the full memory of the running process. Handle them with any security restrictions and guidances in mind.

> [!TIP]
> For frequently asked questions about dump collection, analysis, and other caveats, see [Dumps: FAQ](faq-dumps.yml).
> Dumps might contain sensitive information because they can contain the full memory of the running process. Handle them with any security restrictions and guidance in mind.

* You can use environment variables to configure your application to [collect a dump on a crash](collect-dumps-crash.md).

Expand All @@ -26,6 +23,9 @@ Dumps can be collected in a variety of ways depending on which platform your app

* If you are running your app in production or you are running it in a distributed manner (several services, replicas), [dotnet-monitor](dotnet-monitor.md) provides support for many common scenarios and ad-hoc diagnostic investigations, including dump collection and egress. It enables dumps to be collected remotely or with triggering conditions.

> [!TIP]
> For troubleshooting and frequently asked questions about dump collection, see [Dumps: FAQ](faq-dumps.yml).

## Analyze dumps

* Navigate to [Debug Linux dumps](debug-linux-dumps.md) for information regarding analyzing dumps collected on Linux.
Expand Down
43 changes: 38 additions & 5 deletions docs/core/diagnostics/faq-dumps.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,15 +11,48 @@ sections:
- name: Collecting dumps on macOS and Linux
questions:
- question: |
Why do I only get dumps on Linux if [dotnet-dump](dotnet-dump.md) or my [crashing process](dumps.md#collect-dumps-on-crash) is running elevated?
answer: |
On Linux-based systems, `/proc/sys/kernel/ptrace_scope` controls who can call `ptrace` and what processes they can trace. We recommend it being set to `1` - meaning only processes in the same ancestry chain can use traced. Any value higher than this requires elevation or might disable `ptrace` altogether.
Why is dump collection failing on Linux?
answer: |
In order to implement dump collection, .NET processes spawn a child process called createdump. This child process uses the Linux API [ptrace()](https://www.man7.org/linux/man-pages/man2/ptrace.2.html) and also reads from the [/proc](https://www.kernel.org/doc/html/latest/filesystems/proc.html) filesystem to access thread and memory data that is written into the dump file. Although the API usage is allowed by the default security settings on many Linux distros, sometimes a less common security configuration will deny access. You might see output from the createdump process written on the console of the application being dumped such as:

```
[createdump] The process or container does not have permissions or access: open(/proc/1234/mem) FAILED Permission denied (13)
```

One reason that access can be denied is if a security sandbox intercepts the call using a [seccomp BPF filter](https://www.kernel.org/doc/html/v4.19/userspace-api/seccomp_filter.html). For applications running in a container using Open Container Initiative technology, the `seccomp` profile must allow for calls to `ptrace`. For example, `Docker` uses [containerd](https://github.com/moby/containerd) under the hood as a container runtime. When initializing, it specifies a default [seccomp profile](https://github.com/moby/moby/blob/master/profiles/seccomp/default.json) that allows `ptrace` only if the container host has a kernel version higher than 4.8 or if the `CAP_SYS_PTRACE` capability was specified on the container.

If the calls aren't intercepted, then the kernel does a variety of built-in access checks. The docs for [ptrace()](https://www.man7.org/linux/man-pages/man2/ptrace.2.html) include a detailed description near the end, titled "Ptrace access mode checking", that describes how these are done. Accessing the /proc filesystem also uses a variation of the same ptrace access mode checking. What follows is an abbreviated summary of the security checks performed and places where access might be denied:

- Either the calling process needs to have the same user ID as the target process, or the calling process needs to have CAP_SYS_PTRACE. If neither of these is true, access is denied. Since the .NET runtime doesn't do anything to change the user account when launching createdump, the user IDs should match and this step should succeed.
- If createdump doesn't have CAP_SYS_PTRACE (it doesn't by default), then the target process being dumped needs to be marked "dumpable". By default, most processes on Linux are dumpable, but you can change this setting by calling [prctl()](https://www.man7.org/linux/man-pages/man2/prctl.2.html) with the PR_SET_DUMPABLE option. If you add capabilities to a process using the setcap tool, this can also cause a process to stop being dumpable. For a more detailed description of the dumpable setting and what causes it to be disabled, see [the Linux documentation](https://www.man7.org/linux/man-pages/man2/PR_SET_DUMPABLE.2const.html).
- All of the enabled [Linux security modules](https://www.kernel.org/doc/html/v4.16/admin-guide/LSM/index.html) (LSMs) are enumerated and each of them must approve the access. Unfortunately, if an LSM denies the access, there's no uniform Linux reporting mechanism to know which one is responsible. Instead you need to determine which ones are enabled on your system and then investigate each individually. You can determine which LSMs are active by running: `cat /sys/kernel/security/lsm`. Although any LSM could be responsible, [Yama](https://www.kernel.org/doc/html/v4.16/admin-guide/LSM/Yama.html), [SELinux](https://selinuxproject.org/page/Main_Page), and [AppArmor](https://gitlab.com/apparmor/apparmor/-/wikis/GettingStarted) are frequently the relevant ones.

AppArmor and SELinux both have rich configuration and reporting mechanisms, so if you need to learn how to work with them, it's best to view each project's own documentation. Yama only has a single configuration setting, which can be displayed by running:

```
cat /proc/sys/kernel/yama/ptrace_scope
```

This command outputs a number indicating the current Yama ptrace security policy:

- 0: Classic ptrace permissions.
- 1: Restricted ptrace.
- 2: Admin-only attach.
- 3: No attach.

Yama should grant access for createdump under policies 0 and 1, but expect access will be denied under policies 2 and 3. Policy 3 always denies access, and policy 2 doesn't work by default because createdump normally does not have the capability CAP_SYS_PTRACE.

- question: |
Why do I only get dumps on Linux if dotnet-dump or my crashing process is running elevated?
answer: |
Some Linux-based systems are configured with security policies that require any process collecting a dump to have the capability CAP_SYS_PTRACE. Normally processes do not have this capability but running elevated is one way to enable it. For a fuller description of how Linux security policies impact dump collection, see 'Why is dump collection failing on Linux?'.

- question: |
Why can't I collect dumps when running inside a container?
answer: |
For applications running under any Open Container Initiative technology, the `seccomp` profile must allow for calls to `ptrace`. For example, `Docker` uses [containerd](https://github.com/moby/containerd) under the hood as a container runtime. When initializing the runtime, it specifies a default [seccomp profile](https://github.com/moby/moby/blob/master/profiles/seccomp/default.json) that allows `ptrace` only if the container host has a kernel version higher than 4.8 or if the `CAP_SYS_PTRACE` capability was specified.

For applications running under any Open Container Initiative technology, the `seccomp` profile must allow for calls to [ptrace()](https://www.man7.org/linux/man-pages/man2/ptrace.2.html). For example, `Docker` uses [containerd](https://github.com/moby/containerd) under the hood as a container runtime. When initializing the runtime, it specifies a default [seccomp profile](https://github.com/moby/moby/blob/master/profiles/seccomp/default.json) that allows `ptrace` only if the container host has a kernel version higher than 4.8 or if the `CAP_SYS_PTRACE` capability was specified.

For a fuller description of how Linux security policies impact dump collection, see the question 'Why is dump collection failing on Linux?'.
- question: |
Why can't I collect dumps on macOS?
answer: |
Expand Down
2 changes: 2 additions & 0 deletions docs/navigate/tools-diagnostics/toc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -485,6 +485,8 @@ items:
href: ../../core/diagnostics/debug-windows-dumps.md
- name: Collect dumps on crash
href: ../../core/diagnostics/collect-dumps-crash.md
- name: Frequently asked questions
href: ../../core/diagnostics/faq-dumps.yml
- name: Symbols
href: ../../core/diagnostics/symbols.md
- name: EventPipe
Expand Down
Loading