Assembler test fails for 32-bit x86 target on 64-bit host

[puleglot]

Recently introduced try_compile test (a710032) doesn't work for 32-bit x86 target on 64-bit host. I export "ASFLAGS=--32" variable, "--32" is being added to the linker flags and test fails.

.....
-- The ASM-ATT compiler identification is GNU
-- Found assembler: /usr/bin/as
CMake Warning at loader/CMakeLists.txt:122 (message):
  Could not find working x86 GAS assembler

  The build will fall back on building with C code

  Note that this may be unsafe, as the C code requires tail-call
  optimizations to remove the stack frame for certain calls.  If the compiler
  does not do this, then unknown device extensions will suffer from a
  corrupted stack.
.....

With --debug-trycompile cmake option I get this:

# pwd
/var/tmp/portage/media-libs/vulkan-loader-1.0.54.0-r1/work/Vulkan-LoaderAndValidationLayers-sdk-1.0.54.0-abi_x86_32.x86/loader/CMakeFiles/CMakeTmp
# cat CMakeFiles/cmTC_12db8.dir/link.txt
/usr/bin/ld  --32 -I"/var/tmp/portage/media-libs/vulkan-loader-1.0.54.0-r1/work/Vulkan-LoaderAndValidationLayers-sdk-1.0.54.0-abi_x86_32.x86/loader"     CMakeFiles/cmTC_12db8.dir/asm_test.asm.o  -o cmTC_12db8
# /usr/bin/ld  --32 -I"/var/tmp/portage/media-libs/vulkan-loader-1.0.54.0-r1/work/Vulkan-LoaderAndValidationLayers-sdk-1.0.54.0-abi_x86_32.x86/loader"     CMakeFiles/cmTC_12db8.dir/asm_test.asm.o  -o cmTC_12db8
/usr/bin/ld: unrecognized option '--32'
/usr/bin/ld: use the --help option for usage information
#

My cmake version: 3.7.2.

[johnzupin]

Looks like the CMAKE_ASM-ATT_FLAGS variables in loader/CMakeList.txt needs to be cached.

I'm doing some more testing and will create a pull request today.

[johnzupin]

I don't think this was a fix actually, still looking for solutions...

[ikelos]

This has been reported to Gentoo as bug 625844. They report that this is an issue in 1.0.54.0 and is still present in 1.0.61.1. I'm sure there would be some users on there happy to help test out potential patches if that would help?

[ikelos]

@johnzupin I was wondering how finding a solution for this has been going?

[johnzupin]

@lenny-lunarg: Can you review my fix when you get in on next week please.

[johnzupin]

@lenny-lunarg

[lenny-lunarg]

This should now be resolved with commit 488a6ea. The problem was that the try_compile test was also attempting to link with the --32 argument. This should now be resolved.

[saboya]

Should this work when backported to 1.0.61.1? Tried it on Gentoo and it didn't work for me.

[chewi]

@saboya, this commit only addresses part of the problem on Gentoo. An eclass change will deal with the rest.