Skip to content

Reformatted debug mem leak doc #608

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Mar 23, 2024
Merged

Reformatted debug mem leak doc #608

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
20ebdfc
Reformatted and added spaces before backticks to correct content alig…
RobinBateman808 Mar 20, 2024
706f36f
Update memory-leaks-and-usage.md
RobinBateman808 Mar 20, 2024
c6f17a6
Tidy up code formats
0xTim Mar 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 41 additions & 19 deletions documentation/server/guides/memory-leaks-and-usage.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,36 +12,37 @@ It’s important to note, however, that a gradual increase in memory usage over

Debugging memory leaks in Swift on macOS and Linux environments can be done using different tools and techniques, each with distinct strengths and usability.

Basic troubleshooting steps include:
**Basic troubleshooting steps include:**

1. Using profiling tools, provided by the respective operating systems and development environments, to identify and analyze memory usage.

**For macOS**, [Memory Graph Debugger](https://developer.apple.com/documentation/xcode/gathering-information-about-memory-use#Inspect-the-debug-memory-graph) and this [Detect and diagnose memory issues](https://developer.apple.com/videos/play/wwdc2021/10180/) video are helpful. You can also use the [Xcode Instruments](https://help.apple.com/instruments/mac/10.0/#/dev022f987b) tool for various profiling instruments including the [Allocations instrument](https://developer.apple.com/documentation/xcode/gathering-information-about-memory-use#Profile-your-app-using-the-Allocations-instrument) to track memory allocation and deallocation in your Swift code.
**For macOS**, [Memory Graph Debugger](https://developer.apple.com/documentation/xcode/gathering-information-about-memory-use#Inspect-the-debug-memory-graph) and this [Detect and diagnose memory issues](https://developer.apple.com/videos/play/wwdc2021/10180/) video are helpful. You can also use the [Xcode Instruments](https://help.apple.com/instruments/mac/10.0/#/dev022f987b) tool for various profiling instruments including the [Allocations instrument](https://developer.apple.com/documentation/xcode/gathering-information-about-memory-use#Profile-your-app-using-the-Allocations-instrument) to track memory allocation and deallocation in your Swift code.

**For Linux**, you can use tools like [Valgrind](https://valgrind.org/) or [Heaptrack](https://github.com/KDE/heaptrack) to profile your application as shown in the examples below. Although these tools are primarily used for C/C++ code, they can also work with Swift.
**For Linux**, you can use tools like [Valgrind](https://valgrind.org/) or [Heaptrack](https://github.com/KDE/heaptrack) to profile your application as shown in the examples below. Although these tools are primarily used for C/C++ code, they can also work with Swift.

2. Reviewing code and identifying potential leaks to examine your code for any potential areas where memory leaks may occur. Common sources of leaks include retained references or unbalanced retain-release cycles, which rarely apply to Swift since it performs [automatic reference counting (ARC)](https://docs.swift.org/swift-book/documentation/the-swift-programming-language/automaticreferencecounting/).

> Note: Memory leaks can occur in Swift if there are substantial reference cycles between objects that involve closures or if objects hold references to external resources that are not released properly. However, the likelihood of such issues is significantly reduced through the automatic memory management's ability to add and remove references, making sources of leaks like retained references and unbalanced retain-release cycles less common in Swift code.
**Note:** Memory leaks can occur in Swift if there are substantial reference cycles between objects that involve closures or if objects hold references to external resources that are not released properly. However, the likelihood of such issues is significantly reduced through the automatic memory management's ability to add and remove references, making sources of leaks like retained references and unbalanced retain-release cycles less common in Swift code.

3. Enabling debug memory allocation features allows you to get additional information about objects and their memory allocations.

**On macOS**, you can enable Zombie Objects using Xcode or use [MallocStackLogging](https://developer.apple.com/videos/play/wwdc2022/10106/) to detect over-released or accessed deallocated objects.
**On macOS**, you can enable Zombie Objects using Xcode or use [MallocStackLogging](https://developer.apple.com/videos/play/wwdc2022/10106/) to detect over-released or accessed deallocated objects.

To enable Zombie Objects:
1. Open your Xcode project.
2. Go to the **Edit Scheme** menu by clicking on the scheme dropdown in the toolbar.
3. In the scheme editor window, select the **Run** tab.
4. Choose the **Diagnostics** tab.
5. Under **Memory Management**, check the box next to **Enable Zombie Objects**.
To enable Zombie Objects:
1. Open your Xcode project.
2. Go to the **Edit Scheme** menu by clicking on the scheme dropdown in the toolbar.
3. In the scheme editor window, select the **Run** tab.
4. Choose the **Diagnostics** tab.
5. Under **Memory Management**, check the box next to **Enable Zombie Objects**.

**On Linux**, Swift has built-in LeakSanitizer support that can be enabled using the `-sanitize=leak` compiler flag.
**On Linux**, Swift has built-in LeakSanitizer support that can be enabled using the `-sanitize=leak` compiler flag.

### Troubleshooting

This section aims to provide you with helpful server-side troubleshooting techniques to debug leaks and usage using **Valgrind**, **LeakSanitizer**, and **Heaptrack**.

The following **example program** leaks memory. We are using it as an *example only* to illustrate the various troubleshooting methods mentioned below.
The following **example program** leaks memory. We are using it as an *example only* to illustrate the various troubleshooting methods mentioned below.

```
public class MemoryLeaker {
var closure: () -> Void = { () }
Expand All @@ -67,6 +68,7 @@ func myFunctionDoingTheAllocation() {

myFunctionDoingTheAllocation()
```

## Debugging leaks with Valgrind
Valgrind is an open-source framework for debugging and profiling Linux applications. It provides several tools, including Memcheck, which can detect memory leaks, invalid memory accesses, and other memory errors. Although Valgrind is primarily focused on C/C++ applications, it can also be used with Swift on Linux.

Expand All @@ -86,6 +88,7 @@ brew install valgrind
Valgrind should be successfully installed on your system using the system package manager.

5. Once you've compiled your program (in this case, a binary named `test`), run the following `valgrind` command to enable full leak checking:

```
valgrind --leak-check=full ./test
```
Expand All @@ -95,11 +98,13 @@ valgrind --leak-check=full ./test
1. Install Swift on your Linux system. You can download and install Swift from the [official website](https://swift.org/download/).

2. Install Valgrind on your Linux system by using your package manager. For example, if you are using Ubuntu, you can run the following command:

```
sudo apt-get install valgrind
```

3. Once Valgrind is installed, run the following command:

```
valgrind --leak-check=full swift run
```
Expand Down Expand Up @@ -139,6 +144,7 @@ The `valgrind` command analyzes the program for any memory leaks and shows the r
```

The following trace block (from above) indicates a memory leak.

```
==1== 32 bytes in 1 blocks are definitely lost in loss record 1 of 4
==1== at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
Expand All @@ -152,6 +158,7 @@ The following trace block (from above) indicates a memory leak.
However, since Swift uses name mangling for function and symbol names, the stack traces may not be straightforward to understand.

To demangle the Swift symbols in the stack traces, run the `swift demangle` command:

```
swift demangle <mangled_symbol>
```
Expand All @@ -160,9 +167,10 @@ Replace `<mangled_symbol>` with the mangled symbol name shown in the stack trace

`swift demangle $s4test12MemoryLeakerCACycfC`

> Note: `swift demangle` is a Swift command line utility and should be available if you have the Swift toolchain installed.
**Note:** `swift demangle` is a Swift command line utility and should be available if you have the Swift toolchain installed.

The utility will demangle the symbol and display a human-readable version as follows:

```
==1== 32 bytes in 1 blocks are definitely lost in loss record 1 of 4
==1== at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
Expand Down Expand Up @@ -190,16 +198,19 @@ Here are the steps:

1. Open a terminal session and navigate to your Swift package directory.
2. Set the `ASAN_OPTIONS` environment variable to enable AddressSanitizer and configure its behavior. You can do this by running the command:

```
export ASAN_OPTIONS=detect_leaks=1
```

3. Run `swift build` with the additional option to enable [Address Sanitizer](https://developer.apple.com/documentation/xcode/diagnosing-memory-thread-and-crash-issues-early):

```
swift build --sanitize=address
```

The build process will compile your code with AddressSanitizer enabled, which automatically looks for leaked memory blocks. If any memory leaks during the build are detected, it will output the information (similar to Valgrind) as shown in the example below:

```
=================================================================
==478==ERROR: LeakSanitizer: detected memory leaks
Expand All @@ -213,31 +224,36 @@ Direct leak of 32 byte(s) in 1 object(s) allocated from:

SUMMARY: AddressSanitizer: 32 byte(s) leaked in 1 allocation(s).
```

Currently, the output doesn’t provide a human-readable representation of the function names because [LeakSanitizer doesn't symbolicate stack traces on Linux](https://github.com/apple/swift/issues/55046).

However, you can symbolicate it using `llvm-symbolizer` or `addr2line` if you have `binutils` installed.

To install `binutils` for Swift on a server running Linux, follow these steps:
1. Connect to your Swift server through SSH using a terminal.
2. Update the package lists by running the following command:

```
sudo apt update
```

3. Install `binutils` by running the following command:

```
sudo apt install binutils
```

4. This will install `binutils` and its related tools for working with binaries, object files, and libraries, which can be useful for developing and debugging Swift applications on Linux.

You can now run the following command to demangle the symbols in the stack traces:

```
# /tmp/test+0xc62ce
addr2line -e /tmp/test -a 0xc62ce -ipf | swift demangle
addr2line -e /tmp/test -a 0xc62ce -ipf | swift demangle
```

In this example, the allocation that leaked is coming from:

```
0x00000000000c62ce: test.myFunctionDoingTheAllocation() -> () at crtstuff.c:?
```
Expand All @@ -256,13 +272,15 @@ A GUI front-end analyzer `heaptrack_gui` is available in addition to command lin

Using a different example, here’s a short how-to using [Ubuntu](https://www.swift.org/download/) to analyze transient usage.
1. Install `heaptrack` by running this command:

```
sudo apt-get install heaptrack
```

2. Run the binary twice using `heaptrack`. The first run provides a baseline for `main`.

```
> heaptrack .build/x86_64-unknown-linux-gnu/release/test_1000_autoReadGetAndSet
heaptrack .build/x86_64-unknown-linux-gnu/release/test_1000_autoReadGetAndSet
heaptrack output will be written to "/tmp/.nio_alloc_counter_tests_GRusAy/heaptrack.test_1000_autoReadGetAndSet.84341.gz"
starting application, this might take some time...
...
Expand All @@ -276,8 +294,9 @@ Heaptrack finished! Now run the following to investigate the data:
```

3. Then run it a second time for the `feature branch` by changing the branch and recompiling.

```
> heaptrack .build/x86_64-unknown-linux-gnu/release/test_1000_autoReadGetAndSet
heaptrack .build/x86_64-unknown-linux-gnu/release/test_1000_autoReadGetAndSet
heaptrack output will be written to "/tmp/.nio_alloc_counter_tests_GRusAy/heaptrack.test_1000_autoReadGetAndSet.84372.gz"
starting application, this might take some time...
...
Expand All @@ -294,13 +313,15 @@ ubuntu@ip-172-31-25-161 /t/.nio_alloc_counter_tests_GRusAy>
The output shows 673989 allocations in the `feature branch` version and 319347 in `main`, indicating a regression.

4. Run the following command to analyze the output as a diff from these runs using `heaptrack_print` and pipe it through `swift demangle` for readability:

```
heaptrack_print -T -d heaptrack.test_1000_autoReadGetAndSet.84341.gz heaptrack.test_1000_autoReadGetAndSet.84372.gz | swift demangle
```

> Note: `-T` outputs the temporary allocations, providing transient allocations and not leaks. If leaks are detected, remove `-T`.
**Note:** `-T` outputs the temporary allocations, providing transient allocations and not leaks. If leaks are detected, remove `-T`.

Scroll down to see the transient allocations (output may be long):

```
MOST TEMPORARY ALLOCATIONS
307740 temporary allocations of 290324 allocations in total (106.00%) from
Expand Down Expand Up @@ -339,6 +360,7 @@ swift_slowAlloc
```

Looking at the output above, we can see the extra transient allocations were due to extra debug printing and querying of environment variables as shown below:

```
NIO.URing.getEnvironmentVar(Swift.String) -> Swift.String?
at /home/ubuntu/swiftnio/swift-nio/Sources/NIO/LinuxURing.swift:291
Expand All @@ -348,7 +370,7 @@ NIO.URing._debugPrint(@autoclosure () -> Swift.String) -> ()

In this example, the debug prints are only for testing and would be removed from the code before the branch is merged.

> Tip: Heaptrack can also be [installed on an RPM-based distribution](https://rhel.pkgs.org/8/epel-x86_64/heaptrack-1.2.0-7.el8.x86_64.rpm.html) to debug transient memory usage. You may need to consult the distribution's documentation for the specific repository setup steps. When Heaptrack is installed correctly, it should display its version and usage information.
**Tip:** Heaptrack can also be [installed on an RPM-based distribution](https://rhel.pkgs.org/8/epel-x86_64/heaptrack-1.2.0-7.el8.x86_64.rpm.html) to debug transient memory usage. You may need to consult the distribution's documentation for the specific repository setup steps. When Heaptrack is installed correctly, it should display its version and usage information.

### Limitations

Expand Down