Underscore prefix detection broken if -g specified

[ePirat]

This is not an issue currently, due to #5481 hiding this bug but once that is solved, it will become an issue again.

If the cflags contain -g, the test for the underscore symbol would incorrectly return as soon as it finds the symbol with or without underscore, but when debug symbols are enabled using -g, this will yield the false result. This can easily be illustrated by a small testcase:

symbol_name = b'meson_uscore_prefix'
with open('out.obj', 'rb') as o:
    for line in o:
        # Check if the underscore form of the symbol is somewhere
        # in the output file.
        if b'_' + symbol_name in line:
            print('True')
        # Else, check if the non-underscored form is present
        elif symbol_name in line:
            print('False')

Would print:

False
True

as it first finds the symbol without the underscore, only later finds the symbol with underscore.

Of course this is not an issue when using the meson buildtype option, as meson will not use the -g argument from that for the test, but is an issue when -g is contained in externally provided flags by env variables or crossfile.

[ePirat]

@jpakkane Possible solutions for this:

1. Search the whole file and if both the underscore and non-underscore variant are found, assume symbols have underscore prefix. (This might still produce wrong results in some cases)

2. Have the compiler output assembly and check the assembly. (This is probably the most reliable we can do without using external tools like nm)

3. Use nm on the resulting binary and parse the output of nm. (This is probably the slowest of all approaches, and most complex one as we would need to check for nm or equivalent tool and parse the output)

Personally I am in favor of the second option.

[jpakkane]

Or possibly filter out -g from feature test arguments?

[ePirat]

@jpakkane That might work, although not sure whats the easiest way to do that? Especially if any compiler would use multiple args for that.

Is there a way to filter part fo a list with another list in order, like:

['foo', 'bar', 'baz', 'xyz', 'abc'] filter with ['baz', 'xyz'] to get ['foo', 'bar', 'abc']

[jpakkane]

filtered = [x for x in original_list if x not in things_to_take_out]

[ePirat]

That won't respect order though, so could lead to wrong, confusing, results:

>>> original_list = ['-x', 'foo', '-x', 'debug']
>>> filter_list = ['-x', 'debug']
>>> [x for x in original_list if x not in filter_list]
['foo']

[ePirat]

@jpakkane So the filtering approach is not really that feasible given that there are other flags than -g that can enable debug symbols. So probably it needs a more complex check that is more reliable to debug symbols presence.

[lrusak]

I just ran into this when updating from meson 0.51.0 and after 976c303 I had to strip -g from our cross file flags. Any updates to solving this?

[eli-schwartz]

-g0 negates -g and handles ordering just fine. Can we know ahead of time which potential arguments need to be removed and can they all be negated?

[lrusak]

I've also had to drop -gdwarf-2 as that broke the build also

[xry111]

flto in CFLAGS also causes the problem. And @ePirat 's option 2 won't fix it.

[ePirat]

@xry111 Why would 2. not work here?

[xry111]

@ePirat with -flto the outputed "assembly" contains strings like .gnu.lto_foo.0 so _foo will be found.

[wantehchang]

Similar to the problem with GCC's -flto flag reported by @xry111, symbols_have_underscore_prefix() also returns True incorrectly if Clang's -fprofile-instr-generate flag is in CFLAGS. I think this is because the object file generated by the compiler contains strings like __profc_foo.

The following commands reproduce a linker error caused by this bug when building the dav1d library with Clang's -fprofile-instr-generate flag in CFLAGS on Linux:

$ git clone -b 0.9.2 --depth 1 https://code.videolan.org/videolan/dav1d.git
$ cd dav1d
$ mkdir build
$ cd build
$ export CC=clang
$ export CFLAGS=-fprofile-instr-generate
$ meson ..
$ ninja

You will see a lot of "undefined reference" linker errors. The problem is that the following code in dav1d/meson.build malfunctions and incorrectly adds an underscore prefix to symbols defined in assembly language files:

if cc.symbols_have_underscore_prefix()
    cdata.set10('PREFIX', true)
    cdata_asm.set10('PREFIX', true)
endif

[jannau]

upstream dav1d will move away from meson's cc.symbols_have_underscore_prefix(). See https://code.videolan.org/videolan/dav1d/-/merge_requests/1296
We need it only for external asm. The systems with underscore symbol prefix are well known and have to be supported explicitly in the form of asm code anyway. Adding an additional condition to the OS/architecture list when asm is added for a new system should not be a problem. I don't expect that there will be new OS/architecture combinations which use underscore symbol prefixes.

[nirbheek]

As per discussion on IRC / Matrix with @ePirat, underscore detection will be moved to a hardcoded list based on toolchain and platform, and we will move the current broken compile check to a unit test that compares the compile check against the hardcoded list.

We could use things like nm or manually parsing object files, but that is actually harder to maintain than this system. Especially if new object formats / OSes / toolchains need to be added.

[nirbheek]

@ePirat do you have a draft branch / PR to implement the hard-coded list of underscore + toolchain + platform combination? We hit this in gstreamer again recently: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/978

[ePirat]

@nirbheek Doing that is really tricky given the amount of platforms meson already supports, but I found a much better solution, will submit the PR for this probably tomorrow evening.

[ePirat]

@nirbheek Sorry for the delay, was busy investigating other issues first. Here is the PR: #9989