Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

switched to upstream: getting an Optional.h hasValue assert #9023

Closed
tjenssen opened this issue Jul 18, 2019 · 19 comments
Closed

switched to upstream: getting an Optional.h hasValue assert #9023

tjenssen opened this issue Jul 18, 2019 · 19 comments
Assignees

Comments

@tjenssen
Copy link

tjenssen commented Jul 18, 2019

after setting set EMCC_DEBUG=2
the linker step

emcc:DEBUG: emcc step "calculate system libraries" took 1.13 seconds
emcc:DEBUG: linking: ['C:/dev/emsdk/qt5/qtbase/lib/libqtfreetype.a', 'C:/dev/emsdk/qt5/qtbase/lib/libqtlibpng.a', 'main.obj', 'qmlprojectrunner.js_plugin_import.obj', 'C:/dev/emsdk/qt5/qtbase/plugins/platforms/libqwasm.a', 'C:/dev/emsdk/qt5/qtbase/lib/libQt5EventDispatcherSupport.a', 'C:/dev/emsdk/qt5/qtbase/lib/libQt5FontDatabaseSupport.a', 'C:/dev/emsdk/qt5/qtbase/lib/libqtfreetype.a', 'C:/dev/emsdk/qt5/qtbase/lib/libQt5EglSupport.a', 'C:/dev/emsdk/qt5/qtbase/plugins/imageformats/libqgif.a', 'C:/dev/emsdk/qt5/qtbase/plugins/imageformats/libqicns.a', 'C:/dev/emsdk/qt5/qtbase/plugins/imageformats/libqico.a', 'C:/dev/emsdk/qt5/qtbase/plugins/imageformats/libqjpeg.a', 'C:/dev/emsdk/qt5/qtbase/plugins/imageformats/libqtga.a', 'C:/dev/emsdk/qt5/qtbase/plugins/imageformats/libqtiff.a', 'C:/dev/emsdk/qt5/qtbase/plugins/imageformats/libqwbmp.a', 'C:/dev/emsdk/qt5/qtbase/plugins/imageformats/libqwebp.a', 'C:/dev/emsdk/qt5/qtbase/lib/libQt5Gui.a', 'C:/dev/emsdk/qt5/qtbase/lib/libqtlibpng.a', 'C:/dev/emsdk/qt5/qtbase/lib/libqtharfbuzz.a', 'C:/dev/emsdk/qt5/qtbase/lib/libQt5Core.a', 'C:/dev/emsdk/qt5/qtbase/lib/libqtpcre2.a', 'C:\\Users\\tijensse\\.emscripten_cache\\wasm-obj\\libc.a', 'C:\\Users\\tijensse\\.emscripten_cache\\wasm-obj\\libcompiler_rt.a', 'C:\\Users\\tijensse\\.emscripten_cache\\wasm-obj\\libc-wasm.a', 'C:\\Users\\tijensse\\.emscripten_cache\\wasm-obj\\libc++-noexcept.a', 'C:\\Users\\tijensse\\.emscripten_cache\\wasm-obj\\libc++abi.a', 'C:\\Users\\tijensse\\.emscripten_cache\\wasm-obj\\libc-extras.a', 'C:\\Users\\tijensse\\.emscripten_cache\\wasm-obj\\libgl.a', 'C:\\Users\\tijensse\\.emscripten_cache\\wasm-obj\\libdlmalloc.a', 'C:\\Users\\tijensse\\.emscripten_cache\\wasm-obj\\libpthreads_stub.a', 'C:\\Users\\tijensse\\.emscripten_cache\\wasm-obj\\libcompiler_rt_wasm.a', 'C:\\Users\\tijensse\\.emscripten_cache\\wasm-obj\\libc_rt_wasm.a']

fails with
Assertion failed: hasVal, file C:\b\s\w\ir\cache\builder\emscripten-releases\llvm-project\llvm\include\llvm/ADT/Optional.h, line 177
a very simple c++ application just works

with fastbuild everything was working as expected, not sure what kind of information I can also bring up, just ask me if you need anything!

@sbc100
Copy link
Collaborator

sbc100 commented Jul 18, 2019

Can you post the full error message output from LLVM?

@tjenssen
Copy link
Author

tjenssen commented Jul 18, 2019

not sure what you mean - because there is not much more, but here is the complete output: https://pastebin.com/diJHsuPK

@tjenssen
Copy link
Author

If I do:
em++ -s ERROR_ON_UNDEFINED_SYMBOLS=0 --source-map-base http://localhost:8000/ -o ./qmlprojectrunner.js main.obj qmlprojectrunner.js_plugin_import.obj C:/dev/emsdk/qt5/qtbase/plugins/platforms/libqwasm.a C:/dev/emsdk/qt5/qtbase/lib/libQt5FontDatabaseSupport.a C:/dev/emsdk/qt5/qtbase/lib/libQt5Gui.a C:/dev/emsdk/qt5/qtbase/lib/libQt5Core.a

I am getting the same error, if I remove one of these libs, the error is gone

@tjenssen
Copy link
Author

tjenssen commented Jul 18, 2019

LD_DEBUG seems to be ignored, how can I debug the linker?

edited: this environment variable is for running an application, so ignore this ;)

@sbc100
Copy link
Collaborator

sbc100 commented Jul 18, 2019

You can pass "-verbose" and/or "-mllvm -debug" to wasm-ld. (Or -Wl,-verbose -Wl,-mllvm,-debug to clang or emcc).

You might find that --no-threads is useful too since it avoid interleaved output from the threaded parts.

I didn't know about LD_DEBUG! I will look into adding support for that in lld.

@sbc100
Copy link
Collaborator

sbc100 commented Jul 19, 2019

Normally I would expect a more useful error output from llvm. @dschuff would we expect no backtrace in the llvm assert failure? Should we be building with more debug information in there?

@tjenssen
Copy link
Author

tjenssen commented Jul 19, 2019

ok just added your proposed flags to emcc, this is the output:

applying relocations: .data.__stdout_used count=1
apply reloc: type=getVirtualAddress: R_WASM_SECTION_OFFSET_I32f
apply reloc: type=R_WASM_MEMORY_ADDR_I32 addend=0 index=0 value=3586336 offset=167
 addend=applying relocations: .data.__cxa_terminate_handler count=13439 index=167
 value=apply reloc: type=R_WASM_TABLE_INDEX_I32 addend=03791925 index= offset=0 value=5300 offset=123978
6
apply reloc: type=R_WASM_SECTION_OFFSET_I32 addend=175490applying relocations:  index=.data._ZL5cause167 value=3963976 count= offset=1124001

getVirtualAddress: .L.str.4
apply reloc: type=R_WASM_MEMORY_ADDR_I32 addend=apply reloc: type=R_WASM_SECTION_OFFSET_I32 addend=0 index=352317 value= index=3476268 offset=74167 value=3792009
 offset=124005
apply reloc: type=R_WASM_SECTION_OFFSET_I32 addend=175510 index=167 value=3963996 offset=124028
apply reloc: type=R_WASM_SECTION_OFFSET_I32 addend=3439 index=167 value=3791925 offset=124032

and then many similar lines from 1434273 to 3178780 and then

apply reloc: type=R_WASM_SECTION_OFFSET_I32 addend=405147 index=124 value=34657526 offset=284792
Assertion failed: hasVal, file C:\b\s\w\ir\cache\builder\emscripten-releases\llvm-project\llvm\include\llvm/ADT/Optional.h, line 177
 #0 0x00007ff61554da25 (C:\dev\emsdk\upstream\bin\wasm-ld.exe+0xada25)
 #1 0x00007fff7515d167 (C:\Windows\System32\ucrtbase.dll+0x6d167)
 #2 0x00007fff7515dff1 (C:\Windows\System32\ucrtbase.dll+0x6dff1)
...

@akien-mga
Copy link

akien-mga commented Jul 22, 2019

I get a similar issue compiling Godot Engine: godotengine/godot#30756. Steps to reproduce are included in the linked issue. It used to work fine with 1.38.37 (after working around another issue with computed gotos, fixed in LLVM upstream in between).

On Linux I seem to have some more debug symbols and a more verbose stacktrace:

wasm-ld: /b/s/w/ir/cache/builder/emscripten-releases/llvm-project/llvm/include/llvm/ADT/Optional.h:177: const T &llvm::optional_detail::OptionalStorage<unsigned int, true>::getValue() const & [T = unsigned int]: Assertion `hasVal' failed.
 #0 0x00007fb376481af4 PrintStackTraceSignalHandler(void*) (/home/akien/Projects/godot/emscripten/emsdk/upstream/bin/../lib/libLLVM-9svn.so+0x6c6af4)
 #1 0x00007fb37647f7ae llvm::sys::RunSignalHandlers() (/home/akien/Projects/godot/emscripten/emsdk/upstream/bin/../lib/libLLVM-9svn.so+0x6c47ae)
 #2 0x00007fb376481da8 SignalHandler(int) (/home/akien/Projects/godot/emscripten/emsdk/upstream/bin/../lib/libLLVM-9svn.so+0x6c6da8)
 #3 0x00007fb379294560 __restore_rt (/lib64/libpthread.so.0+0x13560)
 #4 0x00007fb37594ba7a raise (/lib64/libc.so.6+0x3ca7a)
 #5 0x00007fb375934524 abort (/lib64/libc.so.6+0x25524)
 #6 0x00007fb37593440f _nl_load_domain.cold.0 (/lib64/libc.so.6+0x2540f)
 #7 0x00007fb3759409a2 (/lib64/libc.so.6+0x319a2)
 #8 0x00000000006a7ad2 (/home/akien/Projects/godot/emscripten/emsdk/upstream/bin/wasm-ld+0x6a7ad2)
 #9 0x0000000000690267 lld::wasm::InputChunk::writeTo(unsigned char*) const (/home/akien/Projects/godot/emscripten/emsdk/upstream/bin/wasm-ld+0x690267)
#10 0x00000000006befad lld::wasm::CustomSection::writeTo(unsigned char*) (/home/akien/Projects/godot/emscripten/emsdk/upstream/bin/wasm-ld+0x6befad)
#11 0x00000000006bac7d std::_Function_handler<void (), void llvm::parallel::detail::parallel_for_each<__gnu_cxx::__normal_iterator<lld::wasm::OutputSection**, std::vector<lld::wasm::OutputSection*, std::allocator<lld::wasm::OutputSection*> > >, (anonymous namespace)::Writer::writeSections()::$_0>(__gnu_cxx::__normal_iterator<lld::wasm::OutputSection**, std::vector<lld::wasm::OutputSection*, std::allocator<lld::wasm::OutputSection*> > >, __gnu_cxx::__normal_iterator<lld::wasm::OutputSection**, std::vector<lld::wasm::OutputSection*, std::allocator<lld::wasm::OutputSection*> > >, (anonymous namespace)::Writer::writeSections()::$_0)::'lambda'()>::_M_invoke(std::_Any_data const&) (/home/akien/Projects/godot/emscripten/emsdk/upstream/bin/wasm-ld+0x6bac7d)
#12 0x00007fb376417625 std::_Function_handler<void (), llvm::parallel::detail::TaskGroup::spawn(std::function<void ()>)::$_0>::_M_invoke(std::_Any_data const&) (/home/akien/Projects/godot/emscripten/emsdk/upstream/bin/../lib/libLLVM-9svn.so+0x65c625)
#13 0x00007fb376417207 llvm::parallel::detail::(anonymous namespace)::ThreadPoolExecutor::work() (/home/akien/Projects/godot/emscripten/emsdk/upstream/bin/../lib/libLLVM-9svn.so+0x65c207)
#14 0x00007fb375cf23d0 (/lib64/libstdc++.so.6+0xbd3d0)
#15 0x00007fb37928a04c start_thread (/lib64/libpthread.so.0+0x904c)
#16 0x00007fb375a13bcf clone (/lib64/libc.so.6+0x104bcf)
$ emcc -v
emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 1.38.39
clang version 9.0.0 (/b/s/w/ir/cache/git/chromium.googlesource.com-external-github.com-llvm-llvm--project 22c4a147a96447b38ce90e59b27c33079f1aa203)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/akien/Projects/godot/emscripten/emsdk/upstream/bin
shared:INFO: (Emscripten: Running sanity checks)

@kripken kripken added this to Blocker in LLVM Upstream Backend Jul 22, 2019
@supernow
Copy link

I got the same error.
How can I fix it?

@kripken
Copy link
Member

kripken commented Jul 29, 2019

@supernow this bug is still open, and there isn't a known workaround. For now, I'd suggest not using upstream if you hit this bug - use fastcomp instead. But we hope to fix it soon!

@sbc100 sbc100 self-assigned this Aug 16, 2019
@sbc100
Copy link
Collaborator

sbc100 commented Aug 16, 2019

Looks like this is related to debug info. Adding -Wl,--strip-debug fixes the issue. I'm investigating a real fix now.

@sbc100
Copy link
Collaborator

sbc100 commented Aug 19, 2019

Fix should be here: https://reviews.llvm.org/D66435

@sbc100
Copy link
Collaborator

sbc100 commented Aug 20, 2019

Should be fixed now.

@sbc100 sbc100 closed this as completed Aug 20, 2019
llvm-git-migration pushed a commit to llvm/llvm-project that referenced this issue Aug 20, 2019
Debug sections are special in that they can contain relocations against
symbols that are not present in the final output (i.e. not live).
However it is also possible to have R_WASM_TABLE_INDEX relocations
against symbols that don't have a table index assigned (since they are
not address taken by actual code.

Fixes: emscripten-core/emscripten#9023

Differential Revision: https://reviews.llvm.org/D66435

llvm-svn: 369423
dtzWill pushed a commit to llvm-mirror/lld that referenced this issue Aug 20, 2019
Debug sections are special in that they can contain relocations against
symbols that are not present in the final output (i.e. not live).
However it is also possible to have R_WASM_TABLE_INDEX relocations
against symbols that don't have a table index assigned (since they are
not address taken by actual code.

Fixes: emscripten-core/emscripten#9023

Differential Revision: https://reviews.llvm.org/D66435

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369423 91177308-0d34-0410-b5e6-96231b3b80d8
dtzWill pushed a commit to llvm-mirror/llvm that referenced this issue Aug 20, 2019
Debug sections are special in that they can contain relocations against
symbols that are not present in the final output (i.e. not live).
However it is also possible to have R_WASM_TABLE_INDEX relocations
against symbols that don't have a table index assigned (since they are
not address taken by actual code.

Fixes: emscripten-core/emscripten#9023

Differential Revision: https://reviews.llvm.org/D66435

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@369423 91177308-0d34-0410-b5e6-96231b3b80d8
@supernow
Copy link

The latest version is 1.38.42-upstream, however that can not build well without -Wl,--strip-debug flags.
Should I do something else?

@sbc100
Copy link
Collaborator

sbc100 commented Aug 21, 2019

@supernow can you post the output of your build failure? and the output of emcc -v?

@supernow
Copy link

supernow commented Aug 22, 2019

@sbc100

emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 1.38.42
clang version 10.0.0 (/b/s/w/ir/cache/git/chromium.googlesource.com-external-github.com-llvm-llvm--project edfaee08115376467d1c95573de991aebfdaeb42)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /root/emsdk/upstream/bin
Found candidate GCC installation: /usr/lib/gcc/x86_64-redhat-linux/4.8.2
Found candidate GCC installation: /usr/lib/gcc/x86_64-redhat-linux/4.8.5
Selected GCC installation: /usr/lib/gcc/x86_64-redhat-linux/4.8.5
Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Selected multilib: .;@m64
shared:INFO: (Emscripten: Running sanity checks)

without -Wl,--strip-debug flags

wasm-ld: /b/s/w/ir/cache/builder/emscripten-releases/llvm-project/llvm/include/llvm/ADT/Optional.h:177: const T &llvm::optional_detail::OptionalStorage<unsigned int, true>::getValue() const & [T = unsigned int]: Assertion `hasVal' failed.
 #0 0x00007f876fa309f4 PrintStackTraceSignalHandler(void*) (/root/emsdk/upstream/bin/../lib/libLLVM-10svn.so+0x6d79f4)
 #1 0x00007f876fa2e73e llvm::sys::RunSignalHandlers() (/root/emsdk/upstream/bin/../lib/libLLVM-10svn.so+0x6d573e)
 #2 0x00007f876fa30ca8 SignalHandler(int) (/root/emsdk/upstream/bin/../lib/libLLVM-10svn.so+0x6d7ca8)
 #3 0x00007f87729005d0 __restore_rt (/lib64/libpthread.so.0+0xf5d0)
 #4 0x00007f876e7292c7 raise (/lib64/libc.so.6+0x362c7)
 #5 0x00007f876e72a9b8 abort (/lib64/libc.so.6+0x379b8)
 #6 0x00007f876e7220e6 __assert_fail_base (/lib64/libc.so.6+0x2f0e6)
 #7 0x00007f876e722192 (/lib64/libc.so.6+0x2f192)
 #8 0x00000000006ac962 (/root/emsdk/upstream/bin/wasm-ld+0x6ac962)
 #9 0x0000000000695287 lld::wasm::InputChunk::writeTo(unsigned char*) const (/root/emsdk/upstream/bin/wasm-ld+0x695287)
#10 0x00000000006c461d lld::wasm::CustomSection::writeTo(unsigned char*) (/root/emsdk/upstream/bin/wasm-ld+0x6c461d)
#11 0x00000000006c02ed std::_Function_handler<void (), void llvm::parallel::detail::parallel_for_each<__gnu_cxx::__normal_iterator<lld::wasm::OutputSection**, std::vector<lld::wasm::OutputSection*, std::allocator<lld::wasm::OutputSection*> > >, (anonymous namespace)::Writer::writeSections()::$_0>(__gnu_cxx::__normal_iterator<lld::wasm::OutputSection**, std::vector<lld::wasm::OutputSection*, std::allocator<lld::wasm::OutputSection*> > >, __gnu_cxx::__normal_iterator<lld::wasm::OutputSection**, std::vector<lld::wasm::OutputSection*, std::allocator<lld::wasm::OutputSection*> > >, (anonymous namespace)::Writer::writeSections()::$_0)::'lambda'()>::_M_invoke(std::_Any_data const&) (/root/emsdk/upstream/bin/wasm-ld+0x6c02ed)
#12 0x00007f876f9c71a5 std::_Function_handler<void (), llvm::parallel::detail::TaskGroup::spawn(std::function<void ()>)::$_0>::_M_invoke(std::_Any_data const&) (/root/emsdk/upstream/bin/../lib/libLLVM-10svn.so+0x66e1a5)
#13 0x00007f876f9c6d87 llvm::parallel::detail::(anonymous namespace)::ThreadPoolExecutor::work() (/root/emsdk/upstream/bin/../lib/libLLVM-10svn.so+0x66dd87)
#14 0x00007f876f9c6c17 std::thread::_Impl<std::_Bind_simple<llvm::parallel::detail::(anonymous namespace)::ThreadPoolExecutor::ThreadPoolExecutor(unsigned int)::'lambda'() ()> >::_M_run() (/root/emsdk/upstream/bin/../lib/libLLVM-10svn.so+0x66dc17)
#15 0x00007f876f090ad3 std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() /root/gcc-6.1.0/build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:661:0
#16 0x00007f876f090ad3 std::__shared_ptr<std::thread::_Impl_base, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() /root/gcc-6.1.0/build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:928:0
#17 0x00007f876f090ad3 std::shared_ptr<std::thread::_Impl_base>::~shared_ptr() /root/gcc-6.1.0/build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr.h:93:0
#18 0x00007f876f090ad3 execute_native_thread_routine_compat /root/gcc-6.1.0/build/x86_64-pc-linux-gnu/libstdc++-v3/src/c++11/../../../../../libstdc++-v3/src/c++11/thread.cc:102:0
#19 0x00007f87728f8dd5 start_thread (/lib64/libpthread.so.0+0x7dd5)
#20 0x00007f876e7f102d clone (/lib64/libc.so.6+0xfe02d)

whether 1.38.42-upstream version fix the bug?

@sbc100
Copy link
Collaborator

sbc100 commented Aug 23, 2019

The version of llvm used in 1.38.42-upstream (edfaee08115376467d1c95573de991aebfdaeb42) looks like it doesn't include my fix from https://reviews.llvm.org/D66435 which landed at cf2b8722d4e3bc4f0c106a5724778d56074e1ec7.

The respective llvm revisions are:
edfaee08115376467d1c95573de991aebfdaeb42 -> 369268
cf2b8722d4e3bc4f0c106a5724778d56074e1ec7 -> 369423

@dschuff
Copy link
Member

dschuff commented Aug 23, 2019

If 1.38.42 doesn't have the revision you need, you could try manually replacing your clang with this one or wait for the next release.

@supernow
Copy link

Thanks a lot

chisuhua added a commit to chisuhua/lld that referenced this issue Sep 1, 2019
* Return early. NFC.

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@367200 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF] Fix finding locations in messages for undefined hidden symbols.

Previously, when `--vs-diagnostics` was used, the linker printed
something like

  hidden(undef.s): error: undefined hidden symbol: foo
  >>> referenced by undef.s:15

Differential Revision: https://reviews.llvm.org/D65499


git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@367515 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF][X86] Improve tests

* Add --no-show-raw-insn to llvm-objdump -d tests
* When linking an executable with %t.so, the path %t.so will be recorded
  in the DT_NEEDED entry if %t.so doesn't have DT_SONAME. .dynstr will
  have varying lengths on different systems. Add -soname so that the
  string in .dynstr is of fixed length to make tests more robust.
* Rename i386-tls-initial-exec-local.s to i386-tls-ie-local.s
* Refactor tls-initial-exec-local.s to x86-64-tls-ie-local.s

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@367533 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF] With --vs-diagnostics, print a separate message for each location of a duplicate symbol.

We extract and print the source location in the message header so that
Visual Studio is able to parse it and jump there. As duplicate symbols
are defined in several locations, it is more convenient to have separate
error messages, which allows a user to easily access all the locations.

Differential Revision: https://reviews.llvm.org/D65213


git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@367536 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF] Add -z separate-code and pad the last page of last PF_X PT_LOAD with traps only if -z separate-code is specified

This patch

1) adds -z separate-code and -z noseparate-code (default).
2) changes the condition that the last page of last PF_X PT_LOAD is
 padded with trap instructions.
 Current condition (after D33630): if there is no `SECTIONS` commands.
 After this change: if -z separate-code is specified.

-z separate-code was introduced to ld.bfd in 2018, to place the text
segment in its own pages. There is no overlap in pages between an
executable segment and a non-executable segment:

1) RX cannot load initial contents from R or RW(or non-SHF_ALLOC).
2) R and RW(or non-SHF_ALLOC) cannot load initial contents from RX.

lld's current status:

- Between R and RX: in `Writer<ELFT>::fixSectionAlignments()`, the start of a
  segment is always aligned to maxPageSize, so the initial contents loaded by R
  and RX do not overlap. I plan to allow overlaps in D64906 if -z noseparate-code
  is in effect.
- Between RX and RW(or non-SHF_ALLOC if RW doesn't exist):
  we currently unconditionally pad the last page to commonPageSize
  (defaults to 4096 on all targets we support).
  This patch will make it effective only if -z separate-code is specified.

-z separate-code is a dubious feature that intends to reduce the number
of ROP gadgets (which is actually ineffective because attackers can find
plenty of gadgets in the text segment, no need to find gadgets in
non-code regions).

With the overlapping PT_LOAD technique D64906, -z noseparate-code
removes two more alignments at segment boundaries than -z separate-code.
This saves at most defaultCommonPageSize*2 bytes, which are significant
on targets with large defaultCommonPageSize (AArch64/MIPS/PPC: 65536).

Issues/feedback on alignment at segment boundaries to help understand
the implication:

* binutils PR24490 (the situation on ld.bfd is worse because they have
  two R-- on both sides of R-E so more alignments.)

* In binutils, the 2018-02-27 commit "ld: Add --enable-separate-code" made -z separate-code the default on Linux.
  https://github.com/richfelker/musl-cross-make/commit/d969dea983a2cc54a1e0308a0cdeb6c3307e4bfa
  In musl-cross-make, binutils is configured with --disable-separate-code
  to address size regressions caused by -z separate-code. (lld actually has the same
  issue, which I plan to fix in a future patch. The ld.bfd x86 status is
  worse because they default to max-page-size=0x200000).

* https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=237676 people want
  smaller code size. This patch will remove one alignment boundary.

* Stef O'Rear: I'm opposed to any kind of page alignment at the
  text/rodata line (having a partial page of text aliased as rodata and
  vice versa has no demonstrable harm, and I actually care about small
  systems).

So, make -z noseparate-code the default.

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D64903

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@367537 91177308-0d34-0410-b5e6-96231b3b80d8

* [COFF] Fix wholearchive with thin archives

The Archive object created when loading an archive specified with
wholearchive got cleaned up immediately, when the owning std::unique_ptr
went out of scope, even if persisted StringRefs pointed to memory that
belonged to the archive, which no longer was mapped in memory.

This hasn't been an issue with regular (as opposed to thin) archives,
as references to the member objects has kept the mapping for the whole
archive file alive - but with thin archives, all such references point
to other files.

Add the std::unique_ptr to the arena allocator, to retain it as long
as necessary.

This fixes (the last issue raised in) PR42388.

Differential Revision: https://reviews.llvm.org/D65565

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@367599 91177308-0d34-0410-b5e6-96231b3b80d8

* Fix an unused variable warning.

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@367643 91177308-0d34-0410-b5e6-96231b3b80d8

* Improve raw_ostream so that you can "write" colors using operator<<

1. raw_ostream supports ANSI colors so that you can write messages to
the termina with colors. Previously, in order to change and reset
color, you had to call `changeColor` and `resetColor` functions,
respectively.

So, if you print out "error: " in red, for example, you had to do
something like this:

  OS.changeColor(raw_ostream::RED);
  OS << "error: ";
  OS.resetColor();

With this patch, you can write the same code as follows:

  OS << raw_ostream::RED << "error: " << raw_ostream::RESET;

2. Add a boolean flag to raw_ostream so that you can disable colored
output. If you disable colors, changeColor, operator<<(Color),
resetColor and other color-related functions have no effect.

Most LLVM tools automatically prints out messages using colors, and
you can disable it by passing a flag such as `--disable-colors`.
This new flag makes it easy to write code that works that way.

Differential Revision: https://reviews.llvm.org/D65564

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@367649 91177308-0d34-0410-b5e6-96231b3b80d8

* Add a comment for --vs-diagnostics.

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@367650 91177308-0d34-0410-b5e6-96231b3b80d8

* Add an assert() to catch possible regexp errors.

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@367651 91177308-0d34-0410-b5e6-96231b3b80d8

* Revert r367649: Improve raw_ostream so that you can "write" colors using operator<<

This reverts commit r367649 in an attempt to unbreak Windows bots.

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@367658 91177308-0d34-0410-b5e6-96231b3b80d8

* [COFF] Avoid loading objects for mingw autoimport, when a defined alias exists

This avoids a spurious and confusing log message in cases where
both e.g. "alias" and "__imp_alias" exist.

Differential Revision: https://reviews.llvm.org/D65598

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@367673 91177308-0d34-0410-b5e6-96231b3b80d8

* [COFF] Clarify a comment. NFC.

It's the __delayLoadHelper2 function that overwrites the jump table
slot, not this thunk.

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@367674 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF] Move R_*_IRELATIVE from .rel[a].plt to .rel[a].dyn unless --pack-dyn-relocs=android[+relr]

An R_*_IRELATIVE represents the address of a STT_GNU_IFUNC symbol
(redirected at runtime) which is non-preemptable and is not associated
with a canonical PLT (associated with a symbol with a section index of
SHN_UNDEF but a non-zero st_value).

.rel[a].plt [DT_JMPREL, DT_JMPREL+DT_JMPRELSZ) contains relocations that
can be lazily resolved. R_*_IRELATIVE are always eagerly resolved, so
conceptually they do not belong to .rela.plt. "iplt" is mostly a misnomer.

glibc powerpc and powerpc64 do not resolve R_*_IRELATIVE if they are in .rela.plt.

    // a.o - synthesized PLT call stub has an R_*_IRELATIVE
    void ifunc(); int main() { ifunc(); }
    // b.o
    static void real() {}
    asm (".type ifunc, %gnu_indirect_function");
    void *ifunc() { return &real; }

The lld-linked executable crashes. ld.bfd places R_*_IRELATIVE in
.rela.dyn and the executable works.

glibc i386, x86_64, and aarch64 have logic
(glibc/sysdeps/*/dl-machine.h:elf_machine_lazy_rel) to eagerly resolve
R_*_IRELATIVE in .rel[a].plt so the lld-linked executable works.

Move R_*_IRELATIVE from .rel[a].plt to .rel[a].dyn to fix the crashes on
glibc powerpc/powerpc64. This also helps simplifying ifunc
implementation in FreeBSD rtld-elf powerpc64.

If --pack-dyn-relocs=android[+relr] is specified, the Android packed
dynamic relocation format is used for .rela.dyn. We cannot name
in.relaIplt ".rela.dyn" because the output section will have mixed
formats. This can be improved in the future.

Reviewed By: pcc

Differential Revision: https://reviews.llvm.org/D65651

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@367745 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF][test] Delete redundant version-script-*.s tests

Delete version-script-missing.s: it is covered by version-script-noundef.s
Delete version-script-anonymous-local.s: it is covered by version-script-{glob,weak}.s etc
Delete version-script-no-warn{,2}.s: add --fatal-warnings to some version-script.s commands instead

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@367778 91177308-0d34-0410-b5e6-96231b3b80d8

* Rename F_{None,Text,Append} to OF_{None,Text,Append}. NFC

F_{None,Text,Append} are kept for compatibility since r334221.

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@367800 91177308-0d34-0410-b5e6-96231b3b80d8

* [COFF] Omit automatically imported symbols from the symbol table

These symbols actually point to the symbol's IAT entry, which
obviously is different from the symbol itself (which is imported
from a different module and doesn't exist in the current one).

Omitting this symbol helps gdb inspect automatically imported
symbols, see https://sourceware.org/bugzilla/show_bug.cgi?id=24574
for discussion on the matter.

Surprisingly, those extra symbols don't seem to be an issue for
gdb when the sources have been built with clang, only with gcc.
The actual logic in gdb that this depends on still is unknown, but
omitting these symbols from the symbol table is the right thing to
do in any case.

Differential Revision: https://reviews.llvm.org/D65727

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@367836 91177308-0d34-0410-b5e6-96231b3b80d8

* [MinGW] Add an lld specific option for requesting to delay load libraries

With GNU tools, delayload is handled completely differently. (One
creates a specific delayload import library using dlltool and then
links against it instead of the normal import library.)

Instead of requiring using -Xlink=-delayload:lib.dll, we can provide
an lld specific option for this.

Differential Revision: https://reviews.llvm.org/D65728

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@367837 91177308-0d34-0410-b5e6-96231b3b80d8

* Changing representation of .cv_def_range directives in Codeview debug info assembly format for better readability

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@367850 91177308-0d34-0410-b5e6-96231b3b80d8

* Revert "Changing representation of .cv_def_range directives in Codeview debug info assembly format for better readability"

This reverts commit a885afa9fa8cab3b34f1ddf3d21535f88b662881.

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@367861 91177308-0d34-0410-b5e6-96231b3b80d8

* Changing representation of .cv_def_range directives in Codeview debug info assembly format for better readability

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@367867 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF] Consistently prioritize non-* wildcards overs "*" in version scripts

We prioritize non-* wildcards overs VER_NDX_LOCAL/VER_NDX_GLOBAL "*".
This patch generalizes the rule to "*" of other versions and thus fixes PR40176.
I don't feel strongly about this GNU linkers' behavior but the
generalization simplifies code.

Delete `config->defaultSymbolVersion` which was used to special case
VER_NDX_LOCAL/VER_NDX_GLOBAL "*".

In `SymbolTable::scanVersionScript`, custom versions are handled the same
way as VER_NDX_LOCAL/VER_NDX_GLOBAL. So merge
`config->versionScript{Locals,Globals}` into `config->versionDefinitions`.
Overall this seems to simplify the code.

In `SymbolTable::assign{Exact,Wildcard}Versions`,
`sym->verdefIndex == config->defaultSymbolVersion` is changed to
`verdefIndex == UINT32_C(-1)`.
This allows us to give duplicate assignment diagnostics for
`{ global: foo; };` `V1 { global: foo; };`

In test/linkerscript/version-script.s:
  vs_index of an undefined symbol changes from 0 to 1. This doesn't matter (arguably 1 is better because the binding is STB_GLOBAL) because vs_index of an undefined symbol is ignored.

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D65716

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@367869 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF][test] Reorganize some tls-*.s tests

Some tls-*.s tests do not test generic TLS behavior but rather are x86 specific.
Rename them to i386-*.s or x86-64-*.s

Delete tls-static.s: covered by tls-opt.s
Delete tls-opt-no-plt.s: add --implicit-check-not=.plt to x86-64-tls-gdie.s to cover it

Rename tls-dynamic-i686.s to i386-tls-dynamic.s
Rename tls-i686.s to i386-tls-le.s
Rename tls-opt-i686.s to i386-tls-opt.s
Rename tls-opt-iele-i686-nopic.s to i386-tls-opt-iele-nopic.s

Rename tls-dynamic.s to x86-64-tls-dynamic.s . IE should be split off in the future.
Rename tls-error.s to x86-64-reloc-tpoff32-error.s
Rename tls-opt-gdie.s to x86-64-tls-gdie.s
Rename tls-opt-x86_64-noplt.s to x86-64-tls-opt-noplt.s
Rename tls-opt-local.s => x86-64-tls-ie-opt-local.s . It can be merged with x86-64-tls-ie-local.s in the future.

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@367877 91177308-0d34-0410-b5e6-96231b3b80d8

* [MachO] Update LLD to use 64-bit offsets with DataExtractor (3/5)

Differential Revision: https://reviews.llvm.org/D65639


git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368032 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF] Make binding (weak or non-weak) logic consistent for Undefined and SharedSymbol

This is a case missed by D64136. If %t1.o has a weak reference on foo,
and %t2.so has a non-weak reference on foo:

```
0. ld.lld %t1.o %t2.so          # ok; STB_WEAK; accepted since D64136
1. ld.lld %t2.so %t1.o          # undefined symbol: foo; STB_GLOBAL
2. gold %t1.o %t2.so            # ok; STB_WEAK
3. gold %t2.so %t1.o            # undefined reference to 'foo'; STB_GLOBAL
4. ld.bfd %t1.o %t2.so          # undefined reference to `foo'; STB_WEAK
5. ld.bfd %t2.so %t1.o          # undefined reference to `foo'; STB_WEAK
```

It can be argued that in both cases, the binding of the undefined foo
should be set to STB_WEAK, because the binding should not be affected by
referenced from shared objects.

--allow-shlib-undefined doesn't suppress errors (3,4,5), but -shared or
--noinhibit-exec allows ld.bfd/gold to produce a binary:

```
3. gold -shared %t2.so %t1.o    # ok; STB_GLOBAL
4. ld.bfd -shared %t2.so %t1.o  # ok; STB_WEAK
5. ld.bfd -shared %t1.o %t1.o   # ok; STB_WEAK
```

If %t2.so has DT_NEEDED entries, ld.bfd will load them (lld/gold don't
have the behavior). If one of the DSO defines foo and it is in the
link-time search path (e.g. DT_NEEDED entry is an absolute path, via
-rpath=, via -rpath-link=, etc),
`ld.bfd %t1.o %t2.so` and `ld.bfd %t1.o %t2.so` will not error.

In this patch, we make Undefined and SharedSymbol share the same binding
computing logic. Case 1 will be allowed:

```
0. ld.lld %t1.o %t2.so          # ok; STB_WEAK; accepted since D64136
1. ld.lld %t2.so %t1.o          # ok; STB_WEAK; changed by this patch
```

In the future, we can explore the option that turns both (0,1) into
errors if --no-allow-shlib-undefined (default when linking an
executable) is in action.

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D65584

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368038 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF][ARM] Fix /DISCARD/ of section with .ARM.exidx section

The combineEhSections runs, by design, before processSectionCommands so
that input exception sections like .ARM.exidx and .eh_frame are not assigned
to OutputSections. Unfortunately if /DISCARD/ removes InputSections that
have associated .ARM.exidx sections without discarding the .ARM.exidx
synthetic section then we will end up crashing when trying to sort the
InputSections in ascending address order.

We fix this by filtering out the sections that have been discarded prior
to processing the InputSections in finalizeContents().

fixes pr42890

Differential Revision: https://reviews.llvm.org/D65759


git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368041 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF][PPC] Don't relax ifunc toc-indirect accesses to toc-relative

Fixes PR42759.

```
// If ifunc is taken address in -fPIC code, it may have a toc entry
.section .toc,"aw",@progbits
  .quad ifunc

// ifunc may be defined as STT_GNU_IFUNC in another object file
.type ifunc, %gnu_indirect_function
```

If ifunc is non-preemptable (e.g. when linking an executable), the toc
entry will be relocated by R_PPC64_IRELATIVE.

R_*_IRELATIVE represents the symbolic value of a
non-preemptable ifunc (not associated with a canonical PLT) in a writable location. It has an unknown value at
link time, so we cannot apply toc-indirect to toc-relative relaxation.

Reviewed By: luporl, sfertile

Differential Revision: https://reviews.llvm.org/D65755

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368057 91177308-0d34-0410-b5e6-96231b3b80d8

* [WebAssembly] Fix null pointer in createInitTLSFunction

Summary:
`createSyntheticSymbols`, which creates `WasmSym::InitTLS`, is only called
when `!config->relocatable`, but this condition is not checked when calling
`createInitTLSFunction`.

This diff checks `!config->relocatable` before calling `createInitTLSFunction`.

Fixes https://github.com/emscripten-core/emscripten/issues/9155.

Reviewers: tlively, aheejin, kripken, sbc100

Subscribers: dschuff, jgravelle-google, sunfish, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D65785

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368078 91177308-0d34-0410-b5e6-96231b3b80d8

* Re-submit r367649: Improve raw_ostream so that you can "write" colors using operator<<

The original patch broke buildbots, perhaps because it changed the
default setting whether colors are enabled or not.

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368131 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF][X86] Add --no-show-raw-insn and -soname to some i386 tests

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368142 91177308-0d34-0410-b5e6-96231b3b80d8

* Simplify error message output. NFC.

Differential Revision: https://reviews.llvm.org/D65855

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368144 91177308-0d34-0410-b5e6-96231b3b80d8

* Handle /align option.

Differential Revision: https://reviews.llvm.org/D65736

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368145 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF] Fix splitting messages for duplicate symbols.

D65213 (rL367536) does not work for the case when a source file path
includes subdirectories.

Differential Revision: https://reviews.llvm.org/D65810


git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368153 91177308-0d34-0410-b5e6-96231b3b80d8

* Try to fix windows build bots after r368153.


git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368169 91177308-0d34-0410-b5e6-96231b3b80d8

* API update for change to LLVM's lib/DebugInfo/DWARF

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368190 91177308-0d34-0410-b5e6-96231b3b80d8

* Add a test demonstrating DWARF parse failures are not causing lld to exit non-zero

This bug was/is masking other issues - committing this to demonstrate
the problem/track fixing it.

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368220 91177308-0d34-0410-b5e6-96231b3b80d8

* gdb-index: Wire up str_offsets section to avoid incorrect error message about offsets_base

There's still a need for a deeper fix to the way libDebugInfoDWARF error
messages are propagated up to lld - if lld had exited non-zero on this
error message we would've found the issue sooner.

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368229 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF][AArch64] Support for movz, movk tprel relocations

This patch Implements the R_AARCH64_TLSLE_MOVW_TPREL_G*[_NC]. These are
logically the same calculation as the existing TLSLE relocations with
the result written back to mov[nz] and movk instructions. A typical code
sequence is:
movz x0, #:tprel_g2:foo    // bits [47:32] of R_TLS with overflow check
movk x0, #:tprel_g1_nc:foo // bits [31:16] of R_TLS with no overflow check
movk x0, #:tprel_g0_nc:foo // bits [15:0] of R_TLS with no overflow check

This type of code sequence is usually used with a large code model.

Differential Revision: https://reviews.llvm.org/D65882

Fixes: PR42853


git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368293 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF][AArch64] Delete two unused RUN lines from aarch64-movw-tprel.s after D65882

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368298 91177308-0d34-0410-b5e6-96231b3b80d8

* [lld][WebAssembly] Add optional symbols after input file handling

This allows undefined references in input files be resolved by the
optional symbols.  Previously we were doing this before input file
reading which means it was working only for command line symbols
references (i.e. -u or --export).

Also use addOptionalDataSymbol for __dso_handle and make all optional
symbols hidden by default.

Differential Revision: https://reviews.llvm.org/D65920

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368310 91177308-0d34-0410-b5e6-96231b3b80d8

* [lld][WebAssembly] Use createGlobalVariable helper function. NFC.

Differential Revision: https://reviews.llvm.org/D65911

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368325 91177308-0d34-0410-b5e6-96231b3b80d8

* [WebAssembly][lld] control __data_end export with config->shared

Summary:
Emscripten expects `__data_end` to show up in PIC code as long as it's not
linked with `--shared`.

Currently, Emscripten breaks with latest LLVM because `__data_end` is controlled
by `config->isPic` instead of `config->shared`.`

Reviewers: tlively, sbc100

Reviewed By: sbc100

Subscribers: dschuff, jgravelle-google, aheejin, sunfish, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D65980

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368361 91177308-0d34-0410-b5e6-96231b3b80d8

* [lld][WebAssembly] Don't create optional symbols when outputing an object file

Summary: This was a bug in rL368310.  I'm working on a test case now.

Differential Revision: https://reviews.llvm.org/D65985

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368369 91177308-0d34-0410-b5e6-96231b3b80d8

* ELF: Move sections referred to by __start_/__stop_ symbols into the main partition.

In the case where C identifier sections have SHF_LINK_ORDER they will most
likely be placed in the same partition as the section that they are associated
with. But unless this happens to be the main partition, this will cause them
to be excluded from the range covered by the __start_ and __stop_ symbols,
which may lead to incorrect program behaviour. So we need to move them
all into the main partition so that they will be covered by the __start_
and __stop_ symbols.

We may want to refine this approach later and allow different __start_/__stop_
symbol values for different partitions. This would only make sense for
relocations from SHT_NOTE sections since they are duplicated into each
partition.

Differential Revision: https://reviews.llvm.org/D65909

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368375 91177308-0d34-0410-b5e6-96231b3b80d8

* DebugInfo: Explicitly handle errors when parsing unit DIEs

This ensures these errors produce a non-zero exit and improves the
context (providing the name of the input object and section being
parsed).

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368378 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF] Expand regions for gaps due to explicit address

If the dot gets moved by an explicit section address, an empty gap between sections could be created. The encompassing region for the section being parsed needs to be expanded to include the gap.

Differential Revision: https://reviews.llvm.org/D65722

Patch by Gabriel Smith!

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368379 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF] For VS-style diagnostics, prefer printing full paths in the header.

The filename part in the message header is used by Visual Studio
to fill Error List so that a user can click on an item and jump
to the mentioned location. If we use only the name of a source file
and not the full path, Visual Studio might be unable to find the right
file or, even worse, show a wrong one.

Differential Revision: https://reviews.llvm.org/D65875

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368409 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF] Remove unnecessary assignment to `used` in replaceWithDefined

`Symbol::used` is used by Undefined and SharedSymbol to record if a
.symtab entry is needed. It is of no use for Defined.

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368533 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF] Remove redundant !isPreemptible in Symbol::computeBinding()

!isPreemptible was added in r343668 to fix PR39104: symbols redefined by
replaceWithDefined() might be incorrectly considered STB_LOCAL if a
version script specified `local: *;`.

After r367869 (`config->defaultSymbolVersion` was removed), we will
assign VER_NDX_LOCAL to only regular Defined and CommonSymbol, not
Defined created by replaceWithDefined() (because scanVersionScript() is
called before scanRelocations()). The !isPreemptible is thus redundant
and can be deleted.

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368535 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF] Remove redundant isDefined() in Symbol::computeBinding() and delete one redundant call site

After r367869, VER_NDX_LOCAL can only be assigned to Defined and
CommonSymbol.  CommonSymbol becomes Defined after replaceCommonSymbols(),
thus `versionId == VER_NDX_LOCAL` will imply `isDefined()`.

In maybeReportUndefined(), computeBinding() is called when the symbol is
unknown to be Undefined. computeBinding() != STB_LOCAL will always be
true.

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368536 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF] Remove unnecessary assignment to `isPreemptible` in replaceWithDefined()

After r368535, it is no longer used in the handling of VER_NDX_LOCAL.
Drop it.

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368550 91177308-0d34-0410-b5e6-96231b3b80d8

* [lld] Remove unnecessary "class Lazy"

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368644 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF][test] Add dynamic-list-preempt2.s

When producing a DSO, the isPreemptible property of a Defined with
default or protected visibility is affected by the --dynamic-list file,
but not by interposable symbols in other DSOs.

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368649 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF] Rename odd variable names "New" after r365730. NFC

New -> newSym or newFlags

Reviewed By: atanasyan

Differential Revision: https://reviews.llvm.org/D66127

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368651 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF] Simplify handling of exportDynamic and isPreemptible

In Writer::includeInDynSym(), exportDynamic is used by a Defined with
protected or default visibility, to record whether it is required to be
exported into .dynsym. It is set when any of the following conditions
hold:

1) There is an interposable symbol from a DSO (Undefined or SharedSymbol with default visibility)
2) If -shared or --export-dynamic is specified, any symbol in an object file/bitcode sets this property, unless suppressed by canBeOmittedFromSymbolTable().
3) --dynamic-list when producing an executable

4) protected symbol from a DSO preempted by copy relocation/canonical PLT when
  --ignore-{data,function}-address-equality is specified
5) ifunc is exported when -z ifunc-noplt is specified

Bullet points 4) and 5) are irrelevant in this patch.

Bullet 3) does not play well with 1) and 2). When -shared is specified,
exportDynamic of most symbols is true. This makes it incapable to record
--dynamic-list marked symbols. We thus have obscure:

    if (!config->shared)
      b->exportDynamic = true;
    else if (b->includeInDynsym())
      b->isPreemptible = true;

This patch adds another bit `Symbol::inDynamicList` to record
3). We can thus simplify handleDynamicList() by unifying the DSO and
  executable cases. It also allows us to simplify isPreemptible - now
the field is only used in finalizeSections() and later stages.

Reviewed By: peter.smith

Differential Revision: https://reviews.llvm.org/D66091

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368659 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF] Don't special case symbolic relocations with 0 addend to ifunc in writable locations

Currently the following 3 relocation types do not trigger the creation
of a canonical PLT (which changes STT_GNU_IFUNC to STT_FUNC and
redirects all references):

1) GOT-generating (`needsGot`)
2) PLT-generating (`needsPlt`)
3) R_ABS with 0 addend in a writable location. This is used for
  for ifunc function pointers in writable sections such as .data and .toc.

This patch deletes case 3) to simplify the R_*_IRELATIVE generating
logic added in D57371. Other advantages:

* It is guaranteed no more than 1 R_*_IRELATIVE is created for an ifunc.
* PPC64: no need to special case ifunc in toc-indirect to toc-relative relaxation. See D65755

The deleted elf::addIRelativeRelocs demonstrates that one-pass scan
through relocations makes several optimizations difficult. This is
something we can think about in the future.

Reviewed By: peter.smith

Differential Revision: https://reviews.llvm.org/D65995

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368661 91177308-0d34-0410-b5e6-96231b3b80d8

* [lld][test] Update test to print ELF note description data

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368710 91177308-0d34-0410-b5e6-96231b3b80d8

* [lld][WebAssembly] Allow linking of pic code into static binaries

Summary: See https://github.com/emscripten-core/emscripten/issues/9013

Subscribers: dschuff, jgravelle-google, aheejin, sunfish, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D65922

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368719 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF] Initialize 2 fields of Symbol in SymbolTable::insert

A new symbol is added to elf::symtab in 3 steps:

1) SymbolTable::insert creates a placeholder.
2) Symbol::mergeProperties
3) Symbol::replace

Fields referenced by steps 2) and 3) should be initialized in
SymbolTable::insert.  `traced` and `referenced` were missed previously.
This did not cause problems because compilers generated code that
initialized them (bit fields) to 0.

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D66130

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368784 91177308-0d34-0410-b5e6-96231b3b80d8

* [MinGW] Remove stray/inconsistent comment chars in test file. NFC.

Test directives don't need to be in comments in this file.

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368814 91177308-0d34-0410-b5e6-96231b3b80d8

* [MinGW] Restructure Options.td to use multiclass where sensible. NFC.

Differential Revision: https://reviews.llvm.org/D66065

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368815 91177308-0d34-0410-b5e6-96231b3b80d8

* [MinGW] Correct handling different forms of a few options

Support the equals form of the long --entry=<symbol> option,
add a test for the -e<symbol> form.

Add tests for single dash forms of -exclude-all-symbols and
-export-all-symbols.

Support single-dash forms of -out-implib and -output-def, support
the equals form of --output-def=<file>. (We previously had a test
to explicitly disallow -out-implib, but it turns out that GNU ld
actually does support it just fine, despite also matching the
-o<file> option.)

Disallow the double-dashed --u form, add a test for -u<symbol>.

Differential Revision: https://reviews.llvm.org/D66066

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368816 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF][test] Update silent-ignore.test

Some options are implemented now:

--no-warn-common : r263413
--allow-shlib-undefined : r352826

Some are ignored but were not reflected in this test.

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368837 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF] --gdb-index: fix odd variable name cUs after r365730 and replace lower_bound with partition_point. NFC

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368845 91177308-0d34-0410-b5e6-96231b3b80d8

* [LLD] Migrate llvm::make_unique to std::make_unique

Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.

Differential revision: https://reviews.llvm.org/D66259

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368936 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF][PPC] Improve error message for unknown relocations

Like rLLD354040.

Previously, for unrecognized relocation types, in -no-pie mode:

  foo.o: unrecognized reloc 256

In -pie/-shared mode:

  error: can't create dynamic relocation R_PPC_xxx against symbol: yyy in readonly segment

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368964 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF][AArch64] Improve error message for unknown relocations

Like rLLD354040.

Previously, for unrecognized relocation types, in -no-pie/-pie mode, we got something like:

  foo.o: unrecognized relocation ...

In -shared mode:

  error: can't create dynamic relocation ... against symbol: yyy in readonly segment

Delete the default case from AArch64::getRelExpr and add the error there.

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D66277

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@368983 91177308-0d34-0410-b5e6-96231b3b80d8

* Fix lld on GCC 5.1 after the C++14 move

Summary:
libstdc++ in GCC 5.1 has some bugs. The move to C++14 in D66195 triggered one
such bug caused by the new constexpr support in C++14, and the implementation
doing SFINAE wrong with the comparator to std::stable_sort.

Here's a small repro: https://godbolt.org/z/2QC3-n

The fix is to inline the lambdas directly into the llvm::stable_sort call
instead of erasing them through a std::function. The code is more readable as
well.

Reviewers: thakis, ruiu, espindola

Subscribers: emaste, arichardson, MaskRay, jkorous, dexonsmith, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D66306

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369023 91177308-0d34-0410-b5e6-96231b3b80d8

* [lld][Hexagon]Support HEX_32 when building shared objects

Differential Revision: https://reviews.llvm.org/D66105

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369121 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF][Hexagon] Replace R_HEXAGON_GOT with R_GOTPLT

R_GOTPLT is relative to .got.plt since D59594. Since R_HEXAGON_GOT
relocations always have 0 r_addend, they can use R_GOTPLT instead.

Reviewed By: sidneym

Differential Revision: https://reviews.llvm.org/D66274

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369128 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF][PPC] Fix getRelExpr for R_PPC64_REL16_HI

Fixes https://github.com/ClangBuiltLinux/linux/issues/640

R_PPC64_REL16_HI was incorrectly computed as an R_ABS relocation.
rLLD368964 made it a linker failure. Change it to use R_PC to fix the
failures.

Add ppc64-reloc-rel.s for these R_PPC64_REL* tests.

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369184 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF] Replace local variable hasExportDynamic with config->exportDynamic. NFC

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369187 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF][ARM] Add a test that maxes out the thunk convergence limit

Add a test that takes the maximum amount of passes permitted to converge.
This will make sure that any symbol defined in a linker script gets the
correct value and that any other convergence limit involving symbol address
doesn't restrict Thunk convergence.

Differential Revision: https://reviews.llvm.org/D66346



git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369246 91177308-0d34-0410-b5e6-96231b3b80d8

* [lld][Hexagon] Add GOTREL relocations.

Add GOTREL relocation support. (S + A - GOT)

Differential Revision: https://reviews.llvm.org/D66260

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369258 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF][Hexagon] Improve error message for unknown relocations

Like rLLD354040

Previously, for unknown relocation types, in -no-pie/-pie mode, we got something like:

    foo.o: unrecognized relocation ...

In -shared mode:

    error: can't create dynamic relocation ... against symbol: yyy in readonly segment

Delete the default case from Hexagon::getRelExpr and add the error there. We will get consistent error message like `error: unknown relocation (1024) against symbol foo`

Reviewed By: sidneym

Differential Revision: https://reviews.llvm.org/D66275

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369260 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF] Move (copy relocation/canonical PLT) before error checking

In processRelocAux(), we handle errors before copy relocation/canonical PLT.
This makes error checking a bit complex because we have to check for
conditions that will be allowed by copy relocation/canonical PLT.

Instead, move copy relocation/canonical PLT before error checking. This
simplifies the previous clumsy error checking code

`config->shared || (config->pie && expr == R_ABS && type != target->symbolicRel)`

to the simple `config->isPic`. Some diagnostics can be reported in
different ways. The code motion changes diagnostics for some contrived
test cases:

* copy-rel-pie-error.s -> copy-rel-pie2.s:
  It was rejected before but accepted now. ld.bfd also accepts the case.
* copy-errors.s: "cannot preempt symbol" changes to "symbol 'bar' has no type"
* got32{,x}-i386.s: the suggestion changes from "-fPIC or -Wl,-z,notext" to "-fPIE"
* x86-64-dyn-rel-error5.s: one diagnostic changes for -pie case

Reviewed By: peter.smith

Differential Revision: https://reviews.llvm.org/D66007

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369262 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF] Simplify processRelocAux and allow a corner-case error

After D66007/r369262, if the control flow reaches `if (sym.isUndefined())`, we know:

* The relocation is not a link-time constant => symbol is preemptable => Undefined or SharedSymbol
* Not an undef weak.
* -no-pie.
* The symbol type is neither STT_OBJECT nor STT_FUNC.

ld.lld --export-dynamic --unresolved-symbols=ignore-all %t.o can satisfy
these conditions. Delete the isUndefined() test so that we error
`symbol '...' has no type`, because we don't know the type to make the
decision to create copy relocation/canonical PLT.

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369271 91177308-0d34-0410-b5e6-96231b3b80d8

* [lld][WebAssembly] Honor --no-export-dynamic even with -shared

Differential Revision: https://reviews.llvm.org/D66359

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369276 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF][PPC] Allow PT_LOAD to have overlapping p_offset ranges

This change affects the non-linker script case (precisely, when the
`SECTIONS` command is not used). It deletes 3 alignments at PT_LOAD
boundaries for the default case: the size of a powerpc64 binary can be
decreased by at most 192kb. The technique can be ported to other
targets.

Let me demonstrate the idea with a maxPageSize=65536 example:

When assigning the address to the first output section of a new PT_LOAD,
if the end p_vaddr of the previous PT_LOAD is 0x10020, we advance to
the next multiple of maxPageSize: 0x20000. The new PT_LOAD will thus
have p_vaddr=0x20000. Because p_offset and p_vaddr are congruent modulo
maxPageSize, p_offset will be 0x20000, leaving a p_offset gap [0x10020,
0x20000) in the output.

Alternatively, if we advance to 0x20020, the new PT_LOAD will have
p_vaddr=0x20020. We can pick either 0x10020 or 0x20020 for p_offset!
Obviously 0x10020 is the choice because it leaves no gap. At runtime,
p_vaddr will be rounded down by pagesize (65536 if
pagesize=maxPageSize). This PT_LOAD will load additional initial
contents from p_offset ranges [0x10000,0x10020), which will also be
loaded by the previous PT_LOAD. This is fine if -z noseparate-code is in
effect or if we are not transiting between executable and non-executable
segments.

ld.bfd -z noseparate-code leverages this technique to keep output small.
This patch implements the technique in lld, which is mostly effective on
targets with large defaultMaxPageSize (AArch64/MIPS/PPC: 65536). The 3
removed alignments can save almost 3*65536 bytes.

Two places that rely on p_vaddr%pagesize = 0 have to be updated.

1) We used to round p_memsz(PT_GNU_RELRO) up to commonPageSize (defaults
  to 4096 on all targets). Now p_vaddr%commonPageSize may be non-zero.
  The updated formula takes account of that factor.
2) Our TP offsets formulae are only correct if p_vaddr%p_align = 0.
  Fix them. See the updated comments in InputSection.cpp for details.

  On targets that we enable the technique (only PPC64 now),
  we can potentially make `p_vaddr(PT_TLS)%p_align(PT_TLS) != 0`
  if `sh_addralign(.tdata) < sh_addralign(.tbss)`

  This exposes many problems in ld.so implementations, especially the
  offsets of dynamic TLS blocks. Known issues:

  FreeBSD 13.0-CURRENT rtld-elf (i386/amd64/powerpc/arm64)
  glibc (HEAD) i386 and x86_64 https://sourceware.org/bugzilla/show_bug.cgi?id=24606
  musl<=1.1.22 on TLS Variant I architectures (aarch64/powerpc64/...)

  So, force p_vaddr%p_align = 0 by rounding dot up to p_align(PT_TLS).

The technique will be enabled (with updated tests) for other targets in
subsequent patches.

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D64906

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369343 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF][AArch64] Allow PT_LOAD to have overlapping p_offset ranges

Ported the D64906 technique to AArch64. It deletes 3 alignments at
PT_LOAD boundaries for the default case: the size of an aarch64 binary
decreases by at most 192kb.

If `sh_addralign(.tdata) < sh_addralign(.tbss)`,
we can potentially make `p_vaddr(PT_TLS)%p_align(PT_TLS) != 0`.

ld.so that are known to have problems if p_vaddr%p_align!=0:

* musl<=1.1.22
* FreeBSD 13.0-CURRENT (and before) rtld-elf arm64

New test aarch64-tls-vaddr-align.s checks that our workaround makes p_vaddr%p_align = 0.

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D64930

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369344 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF][X86] Allow PT_LOAD to have overlapping p_offset ranges on EM_386

Ported the D64906 technique to EM_386.

If `sh_addralign(.tdata) < sh_addralign(.tbss)`,
we can potentially make `p_vaddr(PT_TLS)%p_align(PT_TLS) != 0`.

ld.so that are known to have problems if p_vaddr%p_align!=0:

* FreeBSD 13.0-CURRENT rtld-elf
* glibc https://sourceware.org/bugzilla/show_bug.cgi?id=24606

New test i386-tls-vaddr-align.s checks our workaround makes p_vaddr%p_align = 0.

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D65865

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369347 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF][PPC] Allow PT_LOAD to have overlapping p_offset ranges on EM_PPC

Ported the D64906 technique to EM_PPC.

Delete ppc-rela.s that is covered by ppc32-abs-pic.s

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369351 91177308-0d34-0410-b5e6-96231b3b80d8

* [COFF] Allow using custom .edata from input object files

This is used by Wine for manually crafting export tables.

If the input object contains .edata sections, GNU ld references them
in the export directory instead of synthesizing an export table using
either export directives or the normal auto export mechanism. (AFAIK,
historically, way way back, GNU ld didn't support synthesizing the
export table - one was supposed to generate it using dlltool and link
it in instead.)

If faced with --out-implib and --output-def, GNU ld still populates
those output files with the same export info as it would have generated
otherwise, disregarding the input .edata. As this isn't an intended
usage combination, I'm not adding checks for that in tests.

Differential Revision: https://reviews.llvm.org/D65903

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369358 91177308-0d34-0410-b5e6-96231b3b80d8

* [COFF] Require an explicit -implib option for creating implibs in mingw mode

GNU ld doesn't produce implibs unless explicitly requested.

Differential Revision: https://reviews.llvm.org/D66367

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369363 91177308-0d34-0410-b5e6-96231b3b80d8

* [WebAssembly][lld] Fix crash when applying relocations to debug sections

Debug sections are special in that they can contain relocations against
symbols that are not present in the final output (i.e. not live).
However it is also possible to have R_WASM_TABLE_INDEX relocations
against symbols that don't have a table index assigned (since they are
not address taken by actual code.

Fixes: https://github.com/emscripten-core/emscripten/issues/9023

Differential Revision: https://reviews.llvm.org/D66435

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369423 91177308-0d34-0410-b5e6-96231b3b80d8

* [COFF] Print the file name on errors writing the pdb file

This avoids confusing contextless error messages such as "No such file
or directory" if e.g. the pdb output file should be written to a
nonexistent directory. (This can happen with linkrepro scripts, at least
old ones.)

Differential Revision: https://reviews.llvm.org/D66466

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369425 91177308-0d34-0410-b5e6-96231b3b80d8

* [COFF] Check errorCount before committing the output file

This avoids producing an output file if errors appeared late in the
linking process (e.g. while fixing relocations, or as in the test,
while checking for multiple resources). If an output file is produced,
build tools might not retry building it on rebuilds, even if a previous
build failed due to the error return code.

Differential Revision: https://reviews.llvm.org/D66491

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369445 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF] More dynamic relocation packing

Currently, with Android dynamic relocation packing, only relative
relocations are grouped together. This patch implements similar
packing for non-relative relocations.

The implementation groups non-relative relocations with the same
r_info and r_addend, if using RELA. By requiring a minimum group
size of 3, this achieves smaller relocation sections. Building Android
for an ARM32 device, I see the total size of /system/lib decrease by
392 KB.

Grouping by r_info also allows the runtime dynamic linker to implement
an 1-entry cache to reduce the number of symbol lookup required. With
such 1-entry cache implemented on Android, I'm seeing 10% to 20%
reduction in total time spent in runtime linker for several executables
that I tested.

As a simple correctness check, I've also built x86_64 Android and booted
successfully.

Differential Revision: https://reviews.llvm.org/D66491
Patch by Vic Yang!

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369488 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF][test] Add CHECK lines omitted in r369488

Add append .o to some object file names

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369489 91177308-0d34-0410-b5e6-96231b3b80d8

* Revert D65242 "[ELF] More dynamic relocation packing"

This reverts r369488 and r369489. The change broke build bots:

http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap-ubsan/builds/14511
http://lab.llvm.org:8011/builders/lld-x86_64-freebsd/builds/34407

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369497 91177308-0d34-0410-b5e6-96231b3b80d8

* Reland D65242 "[ELF] More dynamic relocation packing""

This fixed a bug in r369488. When config->isRela is false, i->r_addend
is not initialized (see encodeDynamicReloc). So we should check
config->isRela before accessing r_addend:

- if (j - i < 3 || i->r_addend)
+ if (j - i < 3 || (config->isRela && i->r_addend != 0))

Original description:

Currently, with Android dynamic relocation packing, only relative
relocations are grouped together. This patch implements similar
packing for non-relative relocations.

The implementation groups non-relative relocations with the same
r_info and r_addend, if using RELA. By requiring a minimum group
size of 3, this achieves smaller relocation sections. Building Android
for an ARM32 device, I see the total size of /system/lib decrease by
392 KB.

Grouping by r_info also allows the runtime dynamic linker to implement
an 1-entry cache to reduce the number of symbol lookup required. With
such 1-entry cache implemented on Android, I'm seeing 10% to 20%
reduction in total time spent in runtime linker for several executables
that I tested.

As a simple correctness check, I've also built x86_64 Android and booted
successfully.

Differential Revision: https://reviews.llvm.org/D65242
Patch by Vic Yang

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369507 91177308-0d34-0410-b5e6-96231b3b80d8

* [LLD][ELF] - Simplify the bad-archive.s test case.

This removes the precompiled binary and improves the
check of the error reported.

Differential revision: https://reviews.llvm.org/D66523

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369516 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF][ARM] Simplify some llvm-objdump tests with both ARM/Thumb states

llvm-objdump can switch between ARM/Thumb states after D60927.

In a few lld tests, we run both

* llvm-objdump -d -triple=thumbv7a-none-linux-gnueabi %t
* llvm-objdump -d -triple=armv7a-none-linux-gnueabi %t

to test ARM/Thumb parts of the same file. In many cases we can just
run one command. There is a problem that prevents us from cleaning
more tests (e.g. test/ELF/arm-thumb-interwork-thunk.s):

In llvm-objdump, while we have ARM/Thumb (primary and secondary)
MCDisassembler and MCSubtargetInfo, we have just one MCInstrAnalysis
which is used to resolve the targets of calls in both ARM/Thumb parts.

    // ThumbMCInstrAnalysis evaluating ARM parts or ARMMCInstrAnalysis evaluating Thumb parts
    // will have incorrect offsets.
    // An example of llvm-objdump -d -triple=thumbv7a on ARM part:
    1304: 3d ff ff fa  blx     #-780                 # no <...>
    1308: 06 00 00 ea  b       #24 <arm_caller+0x24> # wrong target due to wrong offset

Reviewed By: peter.smith

Differential Revision: https://reviews.llvm.org/D66539

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369535 91177308-0d34-0410-b5e6-96231b3b80d8

* [lld-link] implement -lto-obj-path

Summary:
This adds the -lto-obj-path option to lld-link. This can be
used to specify a path at which to write a native object file for
the full LTO part when using LTO unit splitting.

Reviewers: ruiu, tejohnson, pcc, rnk

Reviewed By: ruiu, rnk

Subscribers: mehdi_amini, steven_wu, dexonsmith, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D65964

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369559 91177308-0d34-0410-b5e6-96231b3b80d8

* [COFF] Add libcall symbols to the link when LTO is being used

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369694 91177308-0d34-0410-b5e6-96231b3b80d8

* Fight a bit against global initializers. NFC.

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369695 91177308-0d34-0410-b5e6-96231b3b80d8

* Add a description about multiple linker scripts

Differential Revision: https://reviews.llvm.org/D66630

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369737 91177308-0d34-0410-b5e6-96231b3b80d8

* Explain --reproduce option

I think --reproduce is no longer a debug-only option but a useful
option that a common user may want to use. So, this patch updates
the description of the option in the manual page.

Differential Revision: https://reviews.llvm.org/D60557

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369740 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF] Mention contents of reproduce archive and add help description.

Building on D60557 mention the name of the linker generated contents of
the reproduce archive, response.txt and version.txt.

Also write a shorter description in the ld.lld --help that is closer to
the documentation.

Differential Revision: https://reviews.llvm.org/D66641



git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369762 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF] Align the first section of a PT_LOAD even if its type is SHT_NOBITS

Reported at https://reviews.llvm.org/D64930#1642223

If the only section of a PT_LOAD is a SHT_NOBITS section (e.g. .bss), we
may not align its sh_offset. p_offset of the PT_LOAD will be set to
sh_offset, and we will get p_offset!=p_vaddr (mod p_align).  If such
executable is mapped by the Linux kernel, it will segfault.

After D64906, this may happen the non-linker script case.

The linker script case has had this issue for a long time.
This was fixed by rL321657 (but the test linkerscript/nobits-offset.s
failed to test a SHT_NOBITS section), but broken by rL345154.

Reviewed By: peter.smith

Differential Revision: https://reviews.llvm.org/D66658

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369828 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF] Make member function Writer<ELFT>::removeEmptyPTLoad non-member. NFC

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369838 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF] Simplify with less_second. NFC

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369844 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF] Delete a redundant dyn_cast<InputSection>. NFC

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369868 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF] Error if --strip-all and --emit-relocs are used together

--strip-all suppresses the creation of in.symtab
This can cause a null pointer dereference in OutputSection::finalize()

  // --emit-relocs => copyRelocs is true
  if (!config->copyRelocs || (type != SHT_RELA && type != SHT_REL))
    return;
  ...
  link = in.symTab->getParent()->sectionIndex; // in.symTab is null

Let's just disallow the combination. In some cases the combination can
cause GNU linkers to fail:

* ld.bfd: final link failed: invalid operation
* gold: internal error in set_no_output_symtab_entry, at ../../gold/object.h:1814

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D66704

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369878 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF] Make LinkerScript::assignAddresses iterative

PR42990. For `SECTIONS { b = a; . = 0xff00 + (a >> 8); a = .; }`,
we currently set st_value(a)=0xff00 while st_value(b)=0xffff.

The following call tree demonstrates the problem:

```
link<ELF64LE>(Args);
  Script->declareSymbols(); // insert a and b as absolute Defined
  Writer<ELFT>().run();
    Script->processSectionCommands();
      addSymbol(cmd);       // a and b are re-inserted. LinkerScript::getSymbolValue
                            // is lazily called by subsequent evaluation
    finalizeSections();
      forEachRelSec(scanRelocations<ELFT>);
        processRelocAux     // another problem PR42506, not affected by this patch
      finalizeAddressDependentContent(); // loop executed once
        script->assignAddresses(); // a = 0, b = 0xff00
    script->assignAddresses(); // a = 0xff00, _end = 0xffff
```

We need another assignAddresses() to finalize the value of `a`.

This patch

1) modifies assignAddress() to track the original section/value of each
  symbol and return a symbol whose section/value has changed.
2) moves the post-finalizeSections assignAddress() inside the loop
  of finalizeAddressDependentContent() and makes it iterative.
  Symbol assignment may not converge so we make a few attempts before
  bailing out.

Note, assignAddresses() must be called at least twice. The penultimate
call finalized section addresses while the last finalized symbol values.
It is somewhat obscure and there was no comment.
linkerscript/addr-zero.test tests this.

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D66279

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369889 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF] EhFrameSection: postpone FDE liveness check to finalizeSections

EhFrameSection::addSection checks liveness of FDE early. This makes it
infeasible to move combineEhSections() before ICF.

Postpone the check to EhFrameSection::finalizeContents(). This is what
ARMExidxSyntheticSection does and it will make a subsequent patch D66717
simpler.

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D66727

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369890 91177308-0d34-0410-b5e6-96231b3b80d8

* Copy test data so tests don't traverse test directories.  NFC


git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@369984 91177308-0d34-0410-b5e6-96231b3b80d8

* [lld][WebAssembly] Store table base in config rather than passing it around. NFC.

I've got another change that makes more use of this value in other
places.

Differential Revision: https://reviews.llvm.org/D66777

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@370010 91177308-0d34-0410-b5e6-96231b3b80d8

* [lld][WebAssembly] Create optional symbols after handling --export/--undefined

Handling of --export/--undefined can pull in lazy symbols which in turn
can pull in referenced to optional symbols.  We need to delay the
creation of optional symbols until all possible references to them have
been created.

Differential Revision: https://reviews.llvm.org/D66768

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@370012 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF][ARM] Add --no-show-raw-insn and -soname to some ARM tests

Delete some insignificant addresses to make it simpler for layout
changes.

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@370048 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF][ARM] Allow PT_LOAD to have overlapping p_offset ranges on EM_ARM

Port the D64906 technique to ARM. It deletes 3 alignments at
PT_LOAD boundaries for the default case: the size of an arm binary
decreases by at most 12kb.

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D66749

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@370049 91177308-0d34-0410-b5e6-96231b3b80d8

* Change the X86 datalayout to add three address spaces for 32 bit signed,
32 bit unsigned, and 64 bit pointers.

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@370083 91177308-0d34-0410-b5e6-96231b3b80d8

* [lld][WebAssembly] Support for growable tables

Adds --growable-table flag to handle building wasm modules with tables
that can grow.

Wasm tables that we use to store function pointers. In order to add functions
to that table at runtime, we need to either preallocate space, or grow the table.
In order to specify a table with no maximum size, we need some flag to handle
that case, separately from a potential --max-table-size= flag.

Note that the number of elements in the table isn't knowable until link-time,
so it's unclear if we will want a --max-table-size= flag in the future.

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@370127 91177308-0d34-0410-b5e6-96231b3b80d8

* Revert "Change the X86 datalayout to add three address spaces for 32 bit signed,"

This reverts commit r370083 because it caused check-lld failures on
sanitizer-x86_64-linux-fast.

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@370142 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF][RISCV] Assign st_shndx of __global_pointer$ to 1 if .sdata does not exist

This essentially reverts the code change of D63132 and switches to a simpler approach.

In an executable/shared object, st_shndx of a symbol can be:

1) SHN_UNDEF: undefined symbol (or canonical PLT)
2) SHN_ABS: absolute symbol
3) any other value (usually a regular section index) represents a relative symbol.
  The actual value does not matter.

Many ld.so (musl, all archs except MIPS of FreeBSD rtld-elf) even treat 2) and 3)
the same. If .sdata does not exist, it does not matter what value/section
__global_pointer$ has, as long as it is relative (otherwise there will be a pedantic
lld error. See D63132). Just set the st_shndx arbitrarily to 1.

Dummy st_shndx=1 may be used by __rela_iplt_start, linker-script-defined symbols outside a section, __dso_handle, etc.

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D66798

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@370172 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF][AMDGPU][SPARC] Allow PT_LOAD to have overlapping p_offset ranges on EM_AMDGPU and EM_SPARCV9

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@370180 91177308-0d34-0410-b5e6-96231b3b80d8

* [mach-o] Extend LC_DATA_IN_CODE support to x86_64

Patch by LemonBoy

Differential Revision: https://reviews.llvm.org/D62185

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@370183 91177308-0d34-0410-b5e6-96231b3b80d8

* [ELF][RISCV] Allow PT_LOAD to have overlapping p_offset ranges on EM_RISCV

Port the D64906 technique to RISC-V. It deletes 3 alignments at
PT_LOAD boundaries for the default case: the size of a RISC-V binary
decreases by at most 12kb.

git-svn-id: https://llvm.org/svn/llvm-project/lld/trunk@370192 91177308-0d34-0410-b5e6-96231b3b80d8

* lld: Make a test not fail if "repro" is part of the build directory name

r268231 made it so that the name of the --reproduce archive
is no longer listed in the response file. Previously, with
"--reproduce …
earl pushed a commit to earl/llvm-mirror that referenced this issue Sep 4, 2019
Debug sections are special in that they can contain relocations against
symbols that are not present in the final output (i.e. not live).
However it is also possible to have R_WASM_TABLE_INDEX relocations
against symbols that don't have a table index assigned (since they are
not address taken by actual code.

Fixes: emscripten-core/emscripten#9023

Differential Revision: https://reviews.llvm.org/D66435

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@369423 91177308-0d34-0410-b5e6-96231b3b80d8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Development

No branches or pull requests

6 participants