add ReleaseSmall mode

[andrewrk]

We currently have these build modes:

• Debug (safety on, fast compilation, slow runtime performance, big binary) (llvm -O0)
• ReleaseSafe (safety on, slow compilation, medium runtime performance, big binary) (llvm -O3)
• ReleaseFast (safety off, slow compilation, fast runtime performance, big binary) (llvm -O3)

Add:

• ReleaseSmall (safety off, slow compilation, medium-fast runtime performance, small binary) (llvm -Os)

This also lets libraries check for the release mode and make different decisions. E.g. maybe the float printing code uses a slower implementation without a table of precomputed information in ReleaseSmall mode.

[PavelVozenilek]

Entities are not to be multiplied without necessity.

There are two "natural" build modes:

1. debug (all checks, all tests)
2. release (no checks, no tests), speed or size optimized

There could be third mode, "custom", allowing all possible combinations: release with tests, release with safety checks, debug without tests (for fast recompilation), release with "performance measuring tests", variant with embedded web server to provide insight into app's internal state, etc.

[hasenj]

Does LLVM support tree shaking or dead code elimination? If yes, why should be there be a special mode for this instead of being the default in release mode? If not, what exactly would happen in ReleaseSmall mode that would enable smaller binaries?

[tiehuis]

Loop unrolling and auto-vectorization are two examples which you may not want that otherwise are likely to occur in Release mode without an explicit size setting.

Right now it seems that we only either forward the general CodeGenOpt::Aggressive or CodeGenOpt::None levels to llvm which according to the source corresponds to -O0 and -O3 options.

I had a bit of a look at the specific passes that are enabled by these options for some specific examples but its a bit unwieldy to find at an immediate glance in llvm.

[hasenj]

When people are concerned with binary size, aren't they usually concerned about unused libraries? e.g. people complaining about the size of a hello world program in go-lang.

What's the use case for people who want to prevent loop unrolling? I mean how many kilobytes would it save? If it's really required, wouldn't it be better left as a @pragma or something like that?

[tiehuis]

Sure, that is a problem, but I think given that we typically compile all source as one entire unit (plus lto) and dead code elimination is performed by llvm (pretty sure even at minimal optimization level?) there isn't much extra in the standard case to do here.

I suspect the savings achieved by size targetted optimizations are more than you'd find at first glance. A good example can be seen for sqlite under different compilation settings. https://sqlite.org/footprint.html

This would be especially useful for size-constrained embedded programs, too.

[PavelVozenilek]

Possible way to deal with ever expanding number of compiler modes:

1. There are two basic modes: debug and release.

2. Both these modes could be modified by certain number of options. Command line could be one of:
   --release
--release+small (means optimized for size)
   --release+tests (with tests)
   --release+checks
--release+tests+checks
   ... possibly other valid combinations
   --debug
--debug-tests
--debug+small
--debug-tests+small-checks (shortest compilation time)
   ... possibly other valid combinations

3. builtin could provide a structure with flags, what is enabled and disabled. This may be handy for some meta-meta programming.

4. If one decides to add new mode (e.g. automatic collection of performance data) it would be enough to add +/- option and modify the builtin struct.