Implement control over symbol stripping

I'm developing an fbc package for Gentoo, but there's a behavior in fbc that causes problems for the Gentoo build system. Gentoo packages are required to build executable with the symbol table intact (i.e. "not stripped" executables).

However, fbc automatically passes the -s option down to the linker if debugging/profiling is not enabled; this strips all symbol information from the executable. Since the addition of the -s option cannot be controlled, fbc executables cause the Gentoo build system to fail.

It is true that symbols will not be stripped if debugging is enabled, but this bloats the executables with additional information that is not used (e.g. debug_info). There are situations where symbols are useful outside of source code debugging, where debug_info and such are not neeeded.; for example, a live memory analyzer can keep track of symbol values if they are simply present in the symbol table without the need of additional debugging info.

I propose the following new compiler option:

  -strip           Omit all symbol information from the output file

It will be useful to have a way to manually control whether symbols are stripped or not. That should give user much finer control and freedom over the resulting executables.

[dkl]

That reminds me of https://www.freebasic.net/forum/viewtopic.php?f=17&t=22958 which was about getting a "release" build but with debug info and symbols, which seems to be somewhat similar.

Anyways, an option to disable the -s during linking sounds useful for that. It should probably remain the default for non-debug builds though.

[jayrm]

@vilhelmgray, is the change desired for just building the fbc package? Or is change intended to be global, like for all fbc built executables? If we add '-strip' option only, what is the justification that not stripping symbols by default is the way to go on all platforms for all users?

Most users typically don't seem to care about executable symbols or the benefits and are only looking at binary file size, so stripping all symbols was selected as default.

If it's only desired for building fbc package, then can achieve same result with a '-nostrip' option, and apply through fbc's makefile.

If preserving symbols is the norm on gentoo linux and meant for all executables, ones built by fbc also, AND we want to keep the strip all symbols default on other systems, then it's more like a configuration option for the fbc package build selected by the package maintainer. In which case, we probably need with both a '-nostrip' and a '-strip' option available, where one is selected by default depending on the package build options.

That's a good question, whether all executables compiled by fbc by default should be stripped or not. I decided to investigate why Gentoo requires symbols, and why gcc does not strip all symbols by default.

After speaking with some members of Gentoo and GCC on their IRC, I found that there are some important use cases for symbols in ELF files -- I can't speak about Windows EXEs since I didn't investigate, but I imagine similar behavior is present.

• Keeping the symbol table intact allows it to be exported when linking with system libraries and plugins.

  In particular, the symbol table is essential for plugin interactions because the offset of functions in your executable must be reported to the plugin so that it can hook correctly. In a plugin interaction scenario, the executable finds the plugin function calls by referring to its relocation table information -- and vice versa, the plugin will make calls to the executable via the executable's relocation table information. If this table is stripped, then plugins won't be able to call back to the executable.

• There's also a security aspect to the symbol table. Modern Linux systems (Gentoo, Debian, Ubuntu, Fedora, etc.) all enable Address Space Layout Randomization by default in order to protect the system from malicious code execution. In fact, even FreeBSD has ASLR support in its releases now as well.

  ASLR works by taking the relocation table information (Procedure Linkage Table and Global Offsets Table) and randomizing the memory locations of the respective functions and variables. This makes it significantly more difficult for a malicious entity to hijack the system via a return-to-libc attack or similar vector. Such memory randomization would not be possible without the symbols table intact.

For such reasons, it makes sense for the default behavior of fbc to simply defer to the linker in deciding whether to strip symbols by default. It's apparent that modern systems such as Linux and FreeBSD expect the symbol tables of executables to be intact for certain optimizations, security protections, and improvements to take place. So the most elegant solution is to let the linker on these systems decide, which is why gcc does not strip all symbols by default.

However, there are those users who need to strip symbols for one reason or another -- e.g. perhaps a FreeBASIC bowling scorecard program running on a storage space limited MS-DOS system that is air-gaped from the malware-infested Internet. For those users who actually care about the space taken up by the symbol table, they can use this new '-strip' fbc option to remove it. Note as well that an explicit '-nostrip' would not be necessary for the same reason gcc lacks such an option.

[jayrm]

Thank you for investigating. It's helpful to know the background.

fbc's defaults and gnu's defaults are currently polar opposites:

1. gcc and ld keeps everything unless told to remove it
2. fbc removes everything unless told to keep it (-g/-profile)

I think the goal is that fbc does something reasonable by default for simplicity, but can do something else complex with additional options. But as this issue points out, the requirements for building for gentoo can't be done simply.

The idea of deferring to the system's linker in deciding whether to strip symbols by default, for best compatibility with the target system, sounds reasonable in principal. However, there are some consequences for the user with this new default.

ld's default behaviour is to preserve all symbols unless user specifies otherwise (except --strip-discarded, enabled by default). Which means that ld when invoked from fbc will preserve debugging symbols by default, if they exist in any library or object module, unless user explicitly strips them '-Wl --strip-debug', or user invokes 'strip --strip-debug' as separate step, or new fbc option '-strip' as proposed is used.

I am almost certain we will get some push back from the dos/windows users with this change if only '-strip' option is added, with file sizes doubled in size by default, and much more if some library with debug info is linked. Not sure how Linux users will respond. For me personally, it makes no difference as I almost always build and use all full debugging symbol versions of everything.

So, to move forward, I think what I'd like to see is:

• fbc does not use 'ld -s' aka '--strip-all' by default; this is inline with gnu binutils (on all platforms, I think)
• add 'ENABLE_STRIPALL' makefile & compiler build option so that '--strip-all' can be enabled by default if desired, and enable 'ENABLE_STRIPALL' by default in the makefile on at least dos/windows targets
• add fbc '-strip' option, explicitly invokes '--strip-all' with ld
• add fbc '-nostrip' option, explicitly inhibits '--strip-all' with ld

I think this will give us the flexibility needed. fbc is not specifically a build/make tool, but we are using it like one performing all the steps of compiling and linking. And the debug symbol generation or removal should be separate issue, as that introduces the idea of "release" versus "debug" builds, which as far as I can tell, not a concept that exists in gnu binutils.

So, to move forward, I think what I'd like to see is:

* fbc does not use 'ld -s' aka '--strip-all' by default; this is inline with gnu binutils (on all platforms, I think)

* add 'ENABLE_STRIPALL' makefile & compiler build option so that '--strip-all' can be enabled by default if desired, and enable 'ENABLE_STRIPALL' by default in the makefile on at least dos/windows targets

* add fbc '-strip' option, explicitly invokes '--strip-all' with ld

* add fbc '-nostrip' option, explicitly inhibits '--strip-all' with ld

This is an excellent idea. As you stated, it should give us the flexibility to choose whether to strip symbols or not, both for end-users and package managers.

I'll implement this and force-push it to this branch when I get the chance then. We should end up with three new options: ENABLE_STRIPALL as a build option (default enabled for dos/windows targets), and -strip/-nostrip as fbc options.

The following options are introduced by the latest commit I've pushed:

• -strip fbc option
  • strip all symbol information
• -nostrip fbc option
  • do not strip symbol information
• -d ENABLE_STRIPALL compiler build option
  • configure fbc to default to stripping symbols
• ENABLE_STRIPALL makefile option
  • use -d ENABLE_STRIPALL for fbc on select targets (dos and win32)

With these options a package mantainer can configure fbc to strip symbols by default by passing -d ENABLE_STRIPALL (the makefile is configured to do so for dos and win32 targets). Regardless of how fbc has been configured, an end-user can override the default by passing the explicit -strip or -nostrip option to fbc.

[jayrm]

Thanks. Fixed in #141

[jayrm]

Wiki pages added:
Compiler Option: -nostrip
Compiler Option: -strip

Wiki pages updated:
Compiler Options