Skip to content

Debug builds and split DWARF

Mattias Carlsson edited this page Sep 3, 2026 · 2 revisions

Debug builds and split DWARF

For the flag itself — the three levels and what each costs at runtime — see Documentation/usage.md. This page covers the part that does not fit a reference: where the debug information actually lives after --debug, and the one way to lose it.

The short version

A --debug build writes most of its DWARF into .dwo files that sit beside the objects in packages/, not into the archives it installs and not into the prefix. The installed prefix is therefore much smaller than a debug prefix used to be — and it is no longer self-contained. Delete packages/ and the binaries still run, but a debugger can no longer step into them.

If you only remember one thing: packages/ is load-bearing for as long as you want to debug the prefix you installed from it.

Why the split exists

Every debug level emits the same -g3. That was true before this change and is still true: the levels trade optimization and assertions, never symbol depth (lib/flags.sh, the level table). So an operator who wanted cheaper links had no lever short of not using --debug at all — which is the one thing they had asked for.

The prefix is built --disable-shared, so every consumer statically links every archive, and every archive carried its full DWARF inside. Measured in crippledgeek/rdlp, which links this prefix: 86 test binaries over 100 MB, many near 1 GB each, and a target/debug of 94 GB. Peak RSS during parallel link steps was enough to push a 15 GB machine into swap, because each concurrent linker holds its own copy of that DWARF in memory.

-gsplit-dwarf moves the payload out of the object into a sibling .dwo and leaves a skeleton behind. ar archives the object alone, so the .a a linker reads collapses toward its non-debug size, while gdb still resolves the full information through DW_AT_comp_dir + DW_AT_dwo_name.

What it actually bought

Both columns are --debug=full builds of the same tree. "Inline" is the measurement from #92; "split" was measured on the installed prefix on 2026-09-03.

Archive Inline Split Factor
libavcodec.a 278 MB 42.4 MiB 6.6x
libavformat.a 124 MB 12.2 MiB 10.2x
libavfilter.a 116 MB 16.3 MiB 7.1x
libavutil.a 17 MB 3.0 MiB 5.7x

Aggregate, same builds:

Inline Split
All static archives in the prefix 3.0 GB 810 MB
Installed prefix total 5.1 GB 2.4 GB

The debug information did not shrink — it moved. It now lives in 12,930 .dwo files totalling 1.57 GiB under packages/, which is why the next section matters.

The ratio holds at every level rather than only at -O0. Measured on real libavcodec sources with this tree's own configured FFmpeg CFLAGS (gcc 16.2.1), 23 of the largest objects: 12896 KiB → 4332 KiB at -O0 -g3 (3.0x), and 8796 KiB → 2484 KiB at -O2 -g3 (3.5x). symbols benefits as much as full.

The failure mode, and it is quiet

A debugger that cannot find the .dwo files does not degrade gracefully. Driven, not assumed: gdb reports

Could not find DWO CU <path>.dwo

and then places no breakpoint in that compilation unit. The binary still links, still runs, and behaves identically. Only the debugger notices, and it notices one CU at a time.

Three ways to lose them:

  1. mediaforge.sh clean removes the build trees, .dwo included. lib/cleanup.sh warns before it does.
  2. Copying the prefix to another machine without packages/. The prefix is not portable on its own — this is the known gap tracked in #94.
  3. Rebuilding a recipe. New objects, new .dwo; anything still linked against the old archive is now looking for files that were replaced.

The skeleton left in the object is what keeps this from being total: a backtrace still resolves function names. It is source-level stepping, locals and macros that go.

Verifying it works

# The binary should still say "with debug_info, not stripped"
file ~/.local/mediaforge/bin/ffmpeg

# The .dwo population that backs it
find packages -name '*.dwo' | wc -l

# Source-level debugging against the installed prefix
gdb --batch -ex 'break av_packet_alloc' -ex 'info breakpoints' \
    ~/.local/mediaforge/bin/ffmpeg

A breakpoint resolving to libavcodec/packet.c means the .dwo lookup is working. Could not find DWO CU means it is not — check that packages/ still holds the tree the prefix was built from.

-g3 survives the move: info macro still answers in a binary linked against a split archive, so the macro payload that makes -g3 worth having over -g2 is intact.

Platform and tooling notes

macOS — a no-op rather than a hazard. Splitting is an ELF feature; clang for a Mach-O target accepts the flag, writes no .dwo, and emits a byte-identical object (measured, clang 22.1.8, -target arm64-apple-darwin). Nothing to retain, nothing to lose.

ccache — handled. A cache hit restores the .dwo beside the object (measured, ccache 4.13.6). This matters because lib/ccache.sh puts ccache in front of every compile it can.

cmake — the split survives cmake's flag ordering. cmake appends CMAKE_<LANG>_FLAGS_<CONFIG> after CMAKE_<LANG>_FLAGS, so the compile line really does read -g3 -gsplit-dwarf ... -g -DNDEBUG. A trailing plain -g does not downgrade -g3, and -gsplit-dwarf is not a -g level at all, so nothing cmake appends turns it off. Measured on a macro-defining probe.

Rust and CUDA are not covered. rav1e (cargo) and nv-codec (nvcc) read no CFLAGS and each spell this differently — cargo's is split-debuginfo, nvcc's is different again. Both still ship their DWARF inside their objects. This is recorded rather than discovered: if your link is still heavy, those two are why.

See also

  • Documentation/usage.md — the --debug flag reference
  • #92 — the measurement and the change
  • #94 — making the split optional, for a portable prefix

Clone this wiki locally