Assembly files fail to build with CMake >= 3.19

[Jokeren]

Intention
Build dyninst on LLNL lassen using spack

Describe the bug
Failed to build

To Reproduce
spack install --only dependencies hpctoolkit ^dyninst@master

Expected behavior
Build successfully

System (please complete the following information):

• What platform? Power 9
• What commit? master
• Maybe libc version? gcc-8.3.1

@hainest, Xiaozhu said you have been testing on lassen for a while, not sure if there's something wrong with my installation command.

[hainest]

Are you using a local spack (i.e., one you installed yourself)? If so, what is HEAD?

Could you post the concretization of hpctoolkit ^dyninst@master?

Did your error look something like this?

379                     if (ch == EOF) break;
380                            ^~
381    [ 53%] Building C object CMakeFiles/dyninstAPI_RT_static.dir/src/RTspace.S.o
382    Scanning dependencies of target dynElf
383    make[2]: Leaving directory '/tmp/haines8/spack-stage/spack-stage-dyninst-master-mvasx2jj7xnfupjc4pel6lahuwrzxuvy/spack-build-mvasx2j'
384    make  -f elf/CMakeFiles/dynElf.dir/build.make elf/CMakeFiles/dynElf.dir/build
>> 385    /var/tmp/haines8/spack-stage/spack-stage-dyninst-master-mvasx2jj7xnfupjc4pel6lahuwrzxuvy/spack-src/dyninstAPI_RT/src/RTspace.S:1:1: error: expect
			ed identifier or '(' before '.' token
386     .file "RTspace.s"

[Jokeren]

@hainest Yes, that's like the error I saw. I was using spack@develop

hpctoolkit@2020.06.12%gcc@8.3.1~all-static~bgq~cray~cuda~mpi+papi arch=linux-rhel7-power9le
    ^binutils@2.34%gcc@8.3.1+gold~headers~interwork~ld+libiberty~lto~nls~plugins arch=linux-rhel7-power9le
        ^texinfo@6.5%gcc@8.3.1 patches=12f6edb0c6b270b8c8dba2ce17998c580db01182d871ee32b7b6e4129bd1d23a,1732115f651cff98989cb0215d8f64da5e0f7911ebf0c13b064920f088f2ffe1 arch=linux-rhel7-power9le
            ^perl@5.30.3%gcc@8.3.1+cpanm+shared+threads arch=linux-rhel7-power9le
                ^gdbm@1.18.1%gcc@8.3.1 arch=linux-rhel7-power9le
                    ^readline@8.0%gcc@8.3.1 arch=linux-rhel7-power9le
                        ^ncurses@6.2%gcc@8.3.1~symlinks+termlib arch=linux-rhel7-power9le
                            ^pkgconf@1.7.3%gcc@8.3.1 arch=linux-rhel7-power9le
        ^zlib@1.2.11%gcc@8.3.1+optimize+pic+shared arch=linux-rhel7-power9le
    ^boost@1.73.0%gcc@8.3.1+atomic+chrono~clanglibcpp~container~context~coroutine+date_time~debug+exception~fiber+filesystem+graph~icu+iostreams+locale+log+math~mpi+multithreaded~numpy~pic+program_options~python+random+regex+serialization+shared+signals~singlethreaded+system~taggedlayout+test+thread+timer~versionedlayout+wave cxxstd=98 patches=246508e052c44b6f4e8c2542a71c06cacaa72cd1447ab8d2a542b987bc35ace9,4dd507e1f5a29e3b87b15321a4d8c74afdc8331433edabf7aeab89b3c405d556 visibility=global arch=linux-rhel7-power9le
        ^bzip2@1.0.8%gcc@8.3.1+shared arch=linux-rhel7-power9le
            ^diffutils@3.7%gcc@8.3.1 arch=linux-rhel7-power9le
                ^libiconv@1.16%gcc@8.3.1 arch=linux-rhel7-power9le
    ^dyninst@master%gcc@8.3.1+openmp~stat_dysect~static build_type=RelWithDebInfo arch=linux-rhel7-power9le
        ^cmake@3.17.1%gcc@8.3.1~doc+ncurses+openssl+ownlibs~qt patches=1c540040c7e203dd8e27aa20345ecb07fe06570d56410a24a266ae570b1c4c39 arch=linux-rhel7-power9le
            ^openssl@1.1.1g%gcc@8.3.1+systemcerts arch=linux-rhel7-power9le
        ^elfutils@0.179%gcc@8.3.1+bzip2~nls+xz arch=linux-rhel7-power9le
            ^m4@1.4.18%gcc@8.3.1+sigsegv patches=3877ab548f88597ab2327a2230ee048d2d07ace1062efe81fc92e91b7f39cd00,fc9b61654a3ba1a8d6cd78ce087e7c96366c290bc8d2c299f09828d793b853c8 arch=linux-rhel7-power9le
                ^libsigsegv@2.12%gcc@8.3.1 arch=linux-rhel7-power9le
            ^xz@5.2.5%gcc@8.3.1 arch=linux-rhel7-power9le
        ^intel-tbb@2020.2%gcc@8.3.1+shared+tm cxxstd=default patches=62ba015ebd1819c45bef47411540b789b493e31ca668c4ff4cb2afcbc306b476,ce1fb16fb932ce86a82ca87cf0431d1a8c83652af9f552b264213b2ff2945d73 arch=linux-rhel7-power9le
        ^libiberty@2.33.1%gcc@8.3.1+pic arch=linux-rhel7-power9le
    ^gotcha@1.0.3%gcc@8.3.1~test build_type=RelWithDebInfo arch=linux-rhel7-power9le
    ^libdwarf@20180129%gcc@8.3.1 arch=linux-rhel7-power9le
    ^libmonitor@2019.05.31%gcc@8.3.1~bgq+hpctoolkit~krellpatch arch=linux-rhel7-power9le
    ^libunwind@1.4.0%gcc@8.3.1+xz~zlib arch=linux-rhel7-power9le
    ^mbedtls@2.16.1%gcc@8.3.1+pic build_type=Release arch=linux-rhel7-power9le
    ^papi@6.0.0.1%gcc@8.3.1+example~infiniband~lmsensors~powercap~rapl~sde+shared~static_tools arch=linux-rhel7-power9le
    ^xerces-c@3.2.2%gcc@8.3.1 cxxstd=default transcoder=iconv arch=linux-rhel7-power9le

[hainest]

I've been able to replicate this on my home machine (Ubuntu 20, x86), so it's not just an issue on Lassen. It looks like gcc doesn't like our hand-written GAS assembly anymore when compiling under spack. I don't normally use spack to build Dyninst, so I've not seen this error. My normal means of building it using the build script is working fine on all 12 platforms we test on, including Lassen. I'm bisecting spack to see if I can find when it broke.

[hainest]

It looks like CMake-3.19.0 broke ASM handling. spack install --only dependencies hpctoolkit ^dyninst@master ^cmake@3.18.1 works fine both on my machine and on Lassen.

[hainest]

@Jokeren The problem is that CMake 3.19 started passing the -x <LANG> flag during compilation. We were incorrectly stating that .S files were C-language files, so gcc refused to compile them. I'm testing a PR now to fix that.

[Jokeren]

@hainest Thanks for your investigation!