Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SR-13082] llvm-project: dd_interceptors.cpp:225:20: error: redefinition of 'realpath' #55528

Closed
swift-ci opened this issue Jun 25, 2020 · 15 comments
Labels
bug A deviation from expected or documented behavior Compiler

Comments

@swift-ci
Copy link
Collaborator

swift-ci commented Jun 25, 2020

Previous ID SR-13082
Radar None
Original Reporter soloturn (JIRA User)
Type Bug
Status Closed
Resolution Won't Do
Environment

the build script:
https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=swift-language-git

Additional Detail from JIRA
Votes 1
Component/s Compiler
Labels Bug
Assignee None
Priority Medium

md5: 4d6973609f40b3d15daca4d16d659e9b

Issue Description:

building swift version from git on arch linux gives the following error:


[1/1][100%][0.340s] Cleaning all built files...
 Cleaning... 1004 files.
 [1569/1688][ 92%][4697.448s] Building CXX object tools/clang/tools/extra/clangd/CMakeFiles/obj.clangDaemon.dir/CodeComplete.cpp.o
 /src/swift-language-git/src/llvm-project/clang-tools-extra/clangd/CodeComplete.cpp:80:11: warning: enumeration value 'CommentTag' not handled in switch [-Wswitch]
 switch (Kind) {
 ^
 1 warning generated.
 [1585/1688][ 93%][4744.150s] Building CXX object tools/clang/tools/extra/clangd/CMakeFiles/obj.clangDaemon.dir/Protocol.cpp.o
 /src/swift-language-git/src/llvm-project/clang-tools-extra/clangd/Protocol.cpp:231:11: warning: enumeration value 'CommentTag' not handled in switch [-Wswitch]
 switch (Kind) {
 ^
 1 warning generated.
 [1586/1688][ 93%][4745.381s] Building CXX object tools/clang/tools/extra/clangd/CMakeFiles/obj.clangDaemon.dir/Quality.cpp.o
 /src/swift-language-git/src/llvm-project/clang-tools-extra/clangd/Quality.cpp:119:11: warning: enumeration value 'CommentTag' not handled in switch [-Wswitch]
 switch (D.Kind) {
 ^
 1 warning generated.
 [1677/1688][ 99%][4998.909s] Performing build step for 'compiler-rt'
 [1005/1355][ 74%][83.818s] Building CXX object lib/tsan/dd/CMakeFiles/clang_rt.dd-x86_64.dir/dd_interceptors.cpp.o
 FAILED: lib/tsan/dd/CMakeFiles/clang_rt.dd-x86_64.dir/dd_interceptors.cpp.o 
 /src/swift-language-git/src/build/Ninja-ReleaseAssert/llvm-linux-x86_64/./bin/clang++ -I/src/swift-language-git/src/llvm-project/compiler-rt/lib/tsan/dd/../.. -D_FORTIFY_SOURCE=2 -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -Wall -std=c++14 -Wno-unused-parameter -O3 -DNDEBUG -m64 -fno-lto -fPIC -fno-builtin -fno-exceptions -fomit-frame-pointer -funwind-tables -fno-stack-protector -fno-sanitize=safe-stack -fvisibility=hidden -fno-lto -O3 -gline-tables-only -Wno-gnu -Wno-variadic-macros -Wno-c99-extensions -Wno-non-virtual-dtor -fno-rtti -MD -MT lib/tsan/dd/CMakeFiles/clang_rt.dd-x86_64.dir/dd_interceptors.cpp.o -MF lib/tsan/dd/CMakeFiles/clang_rt.dd-x86_64.dir/dd_interceptors.cpp.o.d -o lib/tsan/dd/CMakeFiles/clang_rt.dd-x86_64.dir/dd_interceptors.cpp.o -c /src/swift-language-git/src/llvm-project/compiler-rt/lib/tsan/dd/dd_interceptors.cpp
 /src/swift-language-git/src/llvm-project/compiler-rt/lib/tsan/dd/dd_interceptors.cpp:225:1: warning: attribute declaration must precede definition [-Wignored-attributes]
 INTERCEPTOR(char\*, realpath, const char \*path, char \*resolved_path) {
 ^
 /src/swift-language-git/src/llvm-project/compiler-rt/lib/tsan/dd/../../interception/interception.h:229:3: note: expanded from macro 'INTERCEPTOR'
 DECLARE_WRAPPER(ret_type, func, __VA_ARGS__) \
 ^
 /src/swift-language-git/src/llvm-project/compiler-rt/lib/tsan/dd/../../interception/interception.h:154:20: note: expanded from macro 'DECLARE_WRAPPER'
 __attribute__((weak, alias("__interceptor_" #func), visibility("default")));
 ^
 /usr/include/bits/stdlib.h:37:8: note: previous definition is here
 __NTH (realpath (const char \*__restrict __name, char \*__restrict __resolved))
 ^
 /src/swift-language-git/src/llvm-project/compiler-rt/lib/tsan/dd/dd_interceptors.cpp:225:20: error: redefinition of 'realpath'
 INTERCEPTOR(char\*, realpath, const char \*path, char \*resolved_path) {
 ^
 /usr/include/bits/stdlib.h:37:8: note: previous definition is here
 __NTH (realpath (const char \*__restrict __name, char \*__restrict __resolved))
 ^
 /src/swift-language-git/src/llvm-project/compiler-rt/lib/tsan/dd/dd_interceptors.cpp:225:1: warning: attribute declaration must precede definition [-Wignored-attributes]
 INTERCEPTOR(char\*, realpath, const char \*path, char \*resolved_path) {
 ^
 /src/swift-language-git/src/llvm-project/compiler-rt/lib/tsan/dd/../../interception/interception.h:229:3: note: expanded from macro 'INTERCEPTOR'
 DECLARE_WRAPPER(ret_type, func, __VA_ARGS__) \
 ^
 /src/swift-language-git/src/llvm-project/compiler-rt/lib/tsan/dd/../../interception/interception.h:154:57: note: expanded from macro 'DECLARE_WRAPPER'
 __attribute__((weak, alias("__interceptor_" #func), visibility("default")));
 ^
 /usr/include/bits/stdlib.h:37:8: note: previous definition is here
 __NTH (realpath (const char \*__restrict __name, char \*__restrict __resolved))
 ^
 2 warnings and 1 error generated.
 [1010/1355][ 74%][89.389s] Building CXX object lib/tsan/CMakeFiles/clang_rt.tsan-x86_64.dir/rtl/tsan_interceptors_posix.cpp.o
 ninja: build stopped: subcommand failed.
 [1685/1688][ 99%][5035.713s] Linking CXX executable bin/c-index-test
 FAILED: tools/clang/runtime/compiler-rt-stamps/compiler-rt-build 
 cd /src/swift-language-git/src/build/Ninja-ReleaseAssert/llvm-linux-x86_64/tools/clang/runtime/compiler-rt-bins && /usr/bin/cmake --build . && /usr/bin/cmake -E touch /src/swift-language-git/src/build/Ninja-ReleaseAssert/llvm-linux-x86_64/tools/clang/runtime/compiler-rt-stamps//compiler-rt-build
 ninja: build stopped: subcommand failed.
 ERROR: command terminated with a non-zero exit status 1, aborting

==> ERROR: A failure occurred in build().
 Aborting...

 

@swift-ci
Copy link
Collaborator Author

swift-ci commented Jun 30, 2020

Comment by solo turn (JIRA)

retested with e80a6b0 llvm-project, now fixed.

@swift-ci
Copy link
Collaborator Author

swift-ci commented Jul 7, 2020

Comment by solo turn (JIRA)

hi theindigamer (JIRA User) i picked up the arch linux packages recently from git, as well as from release. both do not compile. the intention is to first focus on the git version, then it will trickle down to a working release over time. for this purpose i created now a daily build on github as i did not know how to integrate one in your jenkins. i am not sure how to proceed with this error.

the error is always:
/github/workspace/swift-language-git/src/llvm-project/compiler-rt/lib/tsan/dd/dd_interceptors.cpp:225:20: error: redefinition of 'realpath'
INTERCEPTOR(char*, realpath, const char *path, char *resolved_path) {

the build is here:

the arch packages:

doing a build with
LDFLAGS='-ldl -lpthread' python swift/utils/build-script --preset=buildbot_linux,no_test install_destdir="$srcdir/build" installable_package="$srcdir/swift-ar
ch-pkg.tar.gz"

@typesanitizer
Copy link

typesanitizer commented Jul 8, 2020

soloturn (JIRA User), one possibility is that /usr/include/bits/stdlib.h is being (unexpectedly) included, causing the redefinition error. I don't know why that would be happening, but you could try using --extra-cmake-options="-DCMAKE_IGNORE_PATH=/usr/include/bits" or temporarily removing that header from /usr/include/bits to check if that fixes the issue.

@swift-ci
Copy link
Collaborator Author

swift-ci commented Jul 10, 2020

Comment by solo turn (JIRA)

theindigamer (JIRA User) tried so, https://github.com/soloturn/swift-aur/runs/852851416?check_suite_focus=true#step:3:324

```

  • env /usr/sbin/cmake -G Ninja -DCMAKE_C_COMPILER:PATH=/usr/sbin/clang -DCMAKE_CXX_COMPILER:PATH=/usr/sbin/clang++ -DCMAKE_LIBTOOL:PATH=/usr/sbin/libtool -DLLVM_VERSION_MAJOR:STRING=10 -DLLVM_VERSION_MINOR:STRING=0 -DLLVM_VERSION_PATCH:STRING=0 -DCLANG_VERSION_MAJOR:STRING=10 -DCLANG_VERSION_MINOR:STRING=0 -DCLANG_VERSION_PATCH:STRING=0 -DCMAKE_MAKE_PROGRAM=/usr/sbin/ninja -DCMAKE_BUILD_TYPE:STRING=Release -DCMAKE_IGNORE_PATH=/usr/include/bits /github/workspace/swift-language-git/src/cmark
    ```

but for the configure step of compiler-rt it does not do it or has no effect:
https://github.com/soloturn/swift-aur/runs/852851416?check_suite_focus=true#step:3:4892

the patch applied is here:
https://aur.archlinux.org/cgit/aur.git/commit/?h=swift-language-git&id=6d3213d1aec00aa552cfef393134ffaf49752688

will try to create an image which lacks the header file.

@swift-ci
Copy link
Collaborator Author

swift-ci commented Jul 11, 2020

Comment by solo turn (JIRA)

thanks theindigamer (JIRA User), i deleted bits/stdlib.h and references which helped: soloturn/build-aur-action@48ca1a3

/usr/include/stdlib.h has:

/* Define some macros helping to catch buffer overflows. */
#if __USE_FORTIFY_LEVEL > 0 && defined __fortify_function
# include <bits/stdlib.h>
#endif

deleting the if, and the header file can be done on this docker image, but on normal operations this will not be sustainable.

arch linux has a simpler package structure than other distributions, as it does not split between runtime and -dev packages, make the swift compile work for arch should fix it for any linux. how would you recommend to fix the detection / usage of the file in the original swift build?

@typesanitizer
Copy link

typesanitizer commented Jul 20, 2020

Sorry, I forgot to respond here. Ok, so at least we understand what is causing the issue. I'm not sure what exactly is setting the __USE_FORTIFY_LEVEL setting (which apparently seems to be required), since it isn't mentioned in the Swift or LLVM code I'm looking at. Do you know what's going on there? As a hack, it might be worthwhile experimenting with:

  • Setting {{_USE_FORTIFY_LEVEL=0}} in CMake (maybe pass {{-D_USE_FORTIFY_LEVEL=0 }}to CMake?)

  • Somehow undef-ing __fortify_function (again, I don't know what is defining this).

  • If <bit/stdlib.h> has a header guard say BITS_STDLIB_H, try defining that header guard in CMake (-DBITS_STDLIB_H=1) so that even if the header is included, it just expands to empty.

Also, from what I can tell, this doesn't have something to do with Swift specifically... It seems like an LLVM configuration issue... might be worth asking on the llvm-dev mailing list or filing a bug report against LLVM in case anyone has run into this before. It might be worth checking how the LLVM packages are built for Arch Linux and if they have any special hacks.

@swift-ci
Copy link
Collaborator Author

swift-ci commented Jul 25, 2020

Comment by solo turn (JIRA)

arch llvm-10.0.1 package is here:

https://www.archlinux.org/packages/extra/x86_64/llvm/

there is no patch to the code coming from the release tarball or git, the build script does:

cmake .. -G Ninja \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=/usr \
-DLLVM_HOST_TRIPLE=$CHOST \
-DLLVM_BUILD_LLVM_DYLIB=ON \
-DLLVM_LINK_LLVM_DYLIB=ON \
-DLLVM_INSTALL_UTILS=ON \
-DLLVM_ENABLE_RTTI=ON \
-DLLVM_ENABLE_FFI=ON \
-DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=AVR \
-DLLVM_BUILD_TESTS=ON \
-DLLVM_BUILD_DOCS=ON \
-DLLVM_ENABLE_SPHINX=ON \
-DLLVM_ENABLE_DOXYGEN=OFF \
-DSPHINX_WARNINGS_AS_ERRORS=OFF \
-DLLVM_BINUTILS_INCDIR=/usr/include
ninja all ocaml_doc

arch uses "makepkg" which sets in its config the host triple (e.g. x86_64-pc-linux-gnu" to "chost" which is set:

chost = run_command(cc, '-dumpmachine').stdout().strip()
carch = chost.split('-')[0]

when packaging, llvm-config.h is provided additionally to distinguish between 32 bit and 64 bit:

https://github.com/archlinux/svntogit-packages/blob/packages/llvm/trunk/llvm-config.h

#include <bits/wordsize.h>

#if __WORDSIZE == 32
#include "llvm-config-32.h"
#elif __WORDSIZE == 64
#include "llvm-config-64.h"
#else
#error "Unknown word size"
#endif

@swift-ci
Copy link
Collaborator Author

swift-ci commented Jul 26, 2020

Comment by solo turn (JIRA)

then there is compiler-rt:
https://github.com/archlinux/svntogit-packages/blob/packages/compiler-rt/trunk/PKGBUILD

no patch or hack, it does:

cmake .. -G Ninja
-DCMAKE_BUILD_TYPE=Release
-DCMAKE_INSTALL_PREFIX=/usr
ninja

and the apple build does, according to:
https://github.com/soloturn/swift-aur/runs/912654366?check_suite_focus=true

/github/workspace/swift-language-git/src/build/buildbot_linux/llvm-linux-x86_64/./bin/clang++ -I/github/workspace/swift-language-git/src/llvm-project/compiler-rt/lib/tsan/dd/../.. -D_FORTIFY_SOURCE=2 -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -Wall -std=c++14 -Wno-unused-parameter -O3 -DNDEBUG -m64 -fno-lto -fPIC -fno-builtin -fno-exceptions -fomit-frame-pointer -funwind-tables -fno-stack-protector -fno-sanitize=safe-stack -fvisibility=hidden -fno-lto -O3 -gline-tables-only -Wno-gnu -Wno-variadic-macros -Wno-c99-extensions -Wno-non-virtual-dtor -fno-rtti -MD -MT lib/tsan/dd/CMakeFiles/clang_rt.dd-x86_64.dir/dd_interceptors.cpp.o -MF lib/tsan/dd/CMakeFiles/clang_rt.dd-x86_64.dir/dd_interceptors.cpp.o.d -o lib/tsan/dd/CMakeFiles/clang_rt.dd-x86_64.dir/dd_interceptors.cpp.o -c /github/workspace/swift-language-git/src/llvm-project/compiler-rt/lib/tsan/dd/dd_interceptors.cpp

@typesanitizer
Copy link

typesanitizer commented Jul 27, 2020

Looking at the address sanitizer code, I do see there's a macro -D_FORTIFY_SOURCE, along with

clang/test/Driver/darwin-asan-nofortify.c
1:// Make sure AddressSanitizer disables _FORTIFY_SOURCE on Darwin.
5:// CHECK: #define _FORTIFY_SOURCE 0 

I suspect that might be related to the issue you are seeing. I do see a -D_FORTIFY_SOURCE=2 in the invocation you pasted. Maybe you can try overriding that setting somehow from the CMake settings?

@swift-ci
Copy link
Collaborator Author

swift-ci commented Aug 1, 2020

Comment by solo turn (JIRA)

@natecook1000 you wrote the python build-script. what makes compiling compiler-rt special in the swift case? from the command line i see it is using /src/swift-language-git/src/build/Ninja-ReleaseAssert/llvm-linux-x86_64/./bin/clang++ instead of /usr/bin/c++ as the normal package build. how could the two react different to things in includes?

@swift-ci
Copy link
Collaborator Author

swift-ci commented Aug 7, 2020

Comment by solo turn (JIRA)

natesg (JIRA User), theindigamer (JIRA User), i tried to build the arch linux compiler-rt package now with clang, clang++ instead of g++, and it gives the same error. for you guys llvm is "upstream" isn't it and therefor it needs to be fixed there?

i did download the PKGBUILD file from arch: https://github.com/archlinux/svntogit-packages/blob/packages/compiler-rt/trunk/PKGBUILD

and then:
$ export CC=clang
$ export CXX=clang++
$ makepkg -s$

and it really gives the same error when compiling swift.
/usr/bin/clang++ -I../lib/tsan/dd/../.. -D_FORTIFY_SOURCE=2 -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -Wall -std=c++14 -Wno-unused-parameter -O3 -DNDEBUG -m64 -fno-lto -fPIC -fno-builtin -fno-exceptions -fomit-frame-pointer -funwind-tables -fno-stack-protector -fno-sanitize=safe-stack -fvisibility=hidden -fno-lto -O3 -gline-tables-only -Wno-gnu -Wno-variadic-macros -Wno-c99-extensions -Wno-non-virtual-dtor -fno-rtti -MD -MT lib/tsan/dd/CMakeFiles/clang_rt.dd-x86_64.dir/dd_interceptors.cpp.o -MF lib/tsan/dd/CMakeFiles/clang_rt.dd-x86_64.dir/dd_interceptors.cpp.o.d -o lib/tsan/dd/CMakeFiles/clang_rt.dd-x86_64.dir/dd_interceptors.cpp.o -c ../lib/tsan/dd/dd_interceptors.cpp
../lib/tsan/dd/dd_interceptors.cpp:225:1: warning: attribute declaration must precede definition [-Wignored-attributes]
...__NTH (realpath (const char *__restrict __name, char *__restrict __resolved))
^
../lib/tsan/dd/dd_interceptors.cpp:225:20: error: redefinition of 'realpath'
...

@swift-ci
Copy link
Collaborator Author

swift-ci commented Aug 7, 2020

Comment by solo turn (JIRA)

this is upstrean in compiler-rt, will report bug there if they permit.

@typesanitizer
Copy link

typesanitizer commented Aug 7, 2020

> i did download the PKGBUILD file from arch

> and it really gives the same error when compiling swift

Thank you. Yes, that means it should be fixed upstream, not in Swift.

@swift-ci
Copy link
Collaborator Author

swift-ci commented Aug 15, 2020

Comment by solo turn (JIRA)

bug is https://bugs.llvm.org/show_bug.cgi?id=47163

@swift-ci
Copy link
Collaborator Author

swift-ci commented Oct 2, 2020

Comment by Fan Jiang (JIRA)

@compnerd This one is the one I was talking about today 🙂

@swift-ci swift-ci transferred this issue from apple/swift-issues Apr 25, 2022
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug A deviation from expected or documented behavior Compiler
Projects
None yet
Development

No branches or pull requests

2 participants