Fixing Meson build types in a mostly backwards-compatible manner

[mcatanzaro]

Let's discuss Jussi's blog post Making decision without all the information is tricky, a case study, which is a response to my Rule #4: Be Very Careful with Meson’s Build Types. There are several problems with Meson's current build types, but the most serious problem is that debug means different things on different platforms. Consider this table from my blog post:

CMake Build Type	Meson Build Type	Meson debug Option	Production Build?  (excludes Windows)
Release	release	false	Yes
Debug	debug	true	No
RelWithDebInfo	debugoptimized	true	Yes, be careful!
MinSizeRel	minsize	true	Yes, be careful!
N/A	plain	false	Yes

This is already messy before you consider that everything is different on Windows than it is everywhere else. (That table does not apply to Windows.) In my blog post, I recommend projects distinguish between production builds and development builds and follow the following rule: you have a non-production (debug) build if debug is true and if optimization is 0 or g; otherwise, you have a production build. This works everywhere except Windows, but it is very confusing and we can do better.

Here is my strawman proposal to improve things without breaking backwards-compatibility too much:

• Add a new debuginfo variable that will be separate from debug to eliminate confusion about the meaning of debug. Currently, the debug variable means "debuginfo" on most platforms but "debug checks" on Windows. That's inconsistent.
• Deprecate the debugoptimized build type without changing its behavior.
• Introduce a new releasewithdebuginfo build type which sets debug to false, debuginfo to true, and optimization to 2. This replaces debugoptimized.
• Change minsize to set debug to false by default. This is not quite backwards-compatible, but it seems like the right choice because debuginfo is huge, and it doesn't make sense to have it in a minsize build unless you have infrastructure to split it out, which Meson does not provide. It's also needed for the next step. Alternative: if worried about backwards-compatibility, deprecate minsize and introduce a new minsizerelease.

Then we have the following non-deprecated build types, which work the same on Windows as everywhere else:

CMake Build Type	Meson Build Type	Meson debug Option	Meson debuginfo Option	Production Build?
Release	release	false	false	Yes
Debug	debug	true	true	No
RelWithDebInfo	releasewithdebuginfo	false	true	Yes
MinSizeRel	minsize	false	false	Yes
N/A	plain	false	false	Yes

And now everything works nicely, assuming you do not use the deprecated debugoptimized build type, which would work as well as it does today.

This is surely not what Jussi would design if starting from square one, but it seems like a minimally-invasive way to significantly improve what we have now. We could do even better if willing to make bigger changes.

[tristan957]

I definitely think the minsize being debug=true by default is wrong based on the logic in your argument.

[jpakkane]

One thing I considered was creating a new option, say intent with correct values. Setting one will set the other one to something sensible. Then we can deprecate buildtype and hopefully eventually remove it.

[jpakkane]

But you can't really do that because it carries conceptual information that is not available anywhere else.

Suppose you have a project with two implementations of some algorithm. One is as performant as possible and the other is as small (compiled code-wise) as possible. When you are building for size-constrained targets you want to use the latter one and otherwise the first one. The kicker is that sometimes -O2 produces smaller code than -Os. So there are cases where your "build type" or "intent" is conceptually "optimize for size" but the optimization level is O2.

[eli-schwartz]

So, set(CMAKE_C_FLAGS_MINSIZE -O2) then?

[mcatanzaro]

I think buildtype is already the notion of intent. We should start by deprecating get_option('buildtype').

That's what I thought too, but I'm not sure it's actually true. Sometimes you really need -O2 -g in a development build because you are profiling something, for example, but still need to have developer stuff enabled, like loading things from source directory instead of install prefix. Decoupling intent from compiler flags and allowing the compiler flags to be set separately seems useful, as long as we're careful with defaults.

[mcatanzaro]

So from Jussi's blog post, we have the following proposed intents:

development
releaseperf (maximize performance)
releasesize (minimize size)

It might also contain the following:
distro (when building distro packages)
fuzzing

But I wonder if we can simplify this down to just development and release?

• To choose between -O2 and -Os, you could touch the optimization option directly instead of using build type.
• The name distro is very specific and more limiting than plain was. It's also not needed: all distros really need is a way to not inject -O2 -g and that's it. This could easily be a separate option; the only hard part would be naming it. no-standard-flags or add-build-flags or something like that. These are bad names, but you get the idea.
• I have no clue why fuzzing would be different from any other build intent. You can fuzz development builds or release builds as you prefer, right?

[jpakkane]

I have no clue why fuzzing would be different from any other build intent.

Because if you know that the build is used only for fuzz testing you can skip building all other unit tests and possibly even the main binaries. Some libraries have also custom defines to disable things like checksum calculations in protocol fuzzing.

Dunno if this is an important enough use case but it seemed useful at least.

[jpakkane]

Introduce a new releasewithdebuginfo build type

Having both a release-with-debuginfo and plain release is the one thing I really wish I hadn't copied from CMake directly. It makes absolutely no sense to build a release without debug info. You pretty much always need it because otherwise you can't debug crashes from the field. It sorta makes sense for CMake because they don't have individual toggles for optimization and debug info but because we do the situation is not great. And further ...

debuginfo is huge, and it doesn't make sense to have it in a minsize build unless you have infrastructure to split it out, which Meson does not provide.

We do provide strip support which is 90% of the work. The only real problem would be toolchains that do not provide it (and do not do the VS thing of putting it in a separate file). The only real case I can think of is embedded stuff but the toolchains I have worked on do the stripping automatically when converting the binary to a format needed for flashing. There may, of course, be toolchains that don't do that but FWICT they are a small minority.

[mcatanzaro]

Having both a release-with-debuginfo and plain release is the one thing I really wish I hadn't copied from CMake directly. It makes absolutely no sense to build a release without debug info. You pretty much always need it because otherwise you can't debug crashes from the field.

Use cases to not have debuginfo:

• Embedded systems where storage is too small to support it
• Proprietary software

We do provide strip support which is 90% of the work. The only real problem would be toolchains that do not provide it (and do not do the VS thing of putting it in a separate file). The only real case I can think of is embedded stuff but the toolchains I have worked on do the stripping automatically when converting the binary to a format needed for flashing. There may, of course, be toolchains that don't do that but FWICT they are a small minority.

I didn't know about this. The stripped debuginfo gets installed separately?

[jpakkane]

Embedded systems where storage is too small to support it

As mentioned you either strip it out or it gets dropped on flashing upload. I don't think I have ever seen any dev setup that would not have fit these two. If someone has examples, add replies to this message.

Proprietary software

You still need the debug info to decipher crashes from users. I have not used them myself but FWICT this is exactly what Microsoft's symbol servers do. When I last asked Flatpak did not do this, but they seemed to be interested in providing it at some point in the future.

I didn't know about this. The stripped debuginfo gets installed separately?

Not on device. You still have it on your own and can use it for e.g. on-device debugging (again, I have never done this myself, but I've been told it's a thing).

[xclaesse]

Note that meson could split debug into separate files itself: #10064

[rgommers]

It makes absolutely no sense to build a release without debug info.

To the use cases already, I'd also like to add that for Python packages (a la NumPy and SciPy), wheels on PyPI have to be shipped without debug info. The size overhead is ~3x, and when you have ~100 million downloads/month, that extra size is definitely not justified by possible getting a couple of extra stack traces a release. We've released with debug info by accident once or twice, and that gets treated as a critical bug in the release process (installs don't fit into AWS Lambda anymore and other such impacts).

[ALTracer]

Chiming in for flavors of debug symbols in embedded. MMU microprocessors, like Cortex-A with full-blown embedded Linux distros, can do on-device debugging with full gdb (not remote gdbserver in userspace), and for that I usually push -O2 or -Og with -g ELF binaries (and the equivalent of libc6-dbg etc.); sometimes the symbols are separated into program.debug so I can use the stripped binary without the symbols storage overhead elsewhere. Otherwise I set up an ssh connection, spin up a gdbserver, grab the sysroot environment and use the developer workstation to perform some remote debugging for faster/fuller symbol resolution.

A microcontroller, like Cortex-M with baremetal/standalone firmware, cares not for ELF/DWARF (caveat NuttX RTOS with ELF binfmt loader) contents and will only get flashed with .text/.rodata/.data from .hex/.srec/.bin; BUT utilities like gdb know how to flash just the "ROM" sections, when given the entire ELF file, and it makes no sense to strip it or separate debuginfo otherwise. And for MaskROM bootloader protocol programs (think lpcflash, stm32flash, dfu-util, imx6 uuu) will only use the .bin, which is fairly distantly related to the original .elf/.axf .

Side note: what if Meson handled compile setup of giant software suites like Firefox, LibreOffice, Webkit-GTK, QtWebEngine? Building these already is a huge pressure on RAM, and building with debug symbols has even higher requirements (and compilers don't bother with memory management much). The linking step is a separate problem -- the linker will consume proportionally more RAM to resolve all the debug symbols. I'll also mention LTO (but I don't understand it enough).
For that you'd want Release as in normal optimizations but no debug info, as Gentoo users do when compiling locally, and only rebuild with debuginfo when something goes wrong and you want to investigate it instead of rolling back to a non-broken version. The other big packages with long compile times I have on my Gentoo are gcc, llvm, clang, rust toolchains, but they are not subject to debugging.

[matthiasclasen]

My vision for better buildtype would include a way for projects to define their own, declaratively (presets for options), since I believe it is pretty hopeless to come up with the one true set of build types that fit every circumstance.

[eli-schwartz]

set(CMAKE_C_FLAGS_MINSIZE -O2)

\o/