SCEV Stack overflow during Thin-LTO link

[omartijn]

Bugzilla Link	49949
Version	unspecified
OS	Linux
CC	@dwblaikie,@erikolofsson,@smithp35,@yuanfang-chen

Extended Description

Linking a large application built as RelWithDebInfo and ThinLTO enabled leads to a stack overflow in the 12.0 RCs. This is a regression from 11.0, which works.

It eventually fails in ValueTracking.cpp, in the computeKnownBitsFromAssume function, while calling Known.getBitWidth(). I can print Known from lldb just fine, but calling the getBitWidth member function fails with an "address access protected" error. The stack trace at this point shows 13026 entries, which looks like a stack overflow.

The 25 top-most stack frames:

• thread #​15, name = 'ld.lld', stop reason = signal SIGSEGV: address access protected (fault address: 0x7fff6a9f4fe8)
  • frame #​0: 0x000000000510da71 ld.lldcomputeKnownBitsFromAssume(V=0x00007ffe4f728c20, Known=0x00007fff6a9f7760, Depth=5, Q=0x00007fff6a9f7148)::Query const&) at ValueTracking.cpp:662:29 frame #​1: 0x00000000050f9cf6 ld.lldcomputeKnownBits(V=0x00007ffe4f728c20, DemandedElts=0x00007fff6a9f6be0, Known=0x00007fff6a9f7760, Depth=5, Q=0x00007fff6a9f7148)::Query const&) at ValueTracking.cpp:1898:3
    frame #​2: 0x00000000050f921e ld.lldcomputeKnownBits(V=0x00007ffe4f728c20, Known=0x00007fff6a9f7760, Depth=5, Q=0x00007fff6a9f7148)::Query const&) at ValueTracking.cpp:217:3 frame #​3: 0x000000000510c6a9 ld.lldcomputeKnownBitsFromOperator(I=0x00007ffe4f728b38, DemandedElts=0x00007fff6a9f79a0, Known=0x00007fff6a9f87d8, Depth=2, Q=0x00007fff6a9f8c10)::Query const&) at
    deb http://apt.llvm.org/focal/ llvm-toolchain-focal-11 main
    ValueTracking.cpp:1479:9
    frame #​4: 0x00000000050f9c72 ld.lldcomputeKnownBits(V=0x00007ffe4f728b38, DemandedElts=0x00007fff6a9f79a0, Known=0x00007fff6a9f87d8, Depth=2, Q=0x00007fff6a9f8c10)::Query const&) at ValueTracking.cpp:1886:5 frame #​5: 0x00000000050f921e ld.lldcomputeKnownBits(V=0x00007ffe4f728b38, Known=0x00007fff6a9f87d8, Depth=2, Q=0x00007fff6a9f8c10)::Query const&) at ValueTracking.cpp:217:3
    frame #​6: 0x000000000510bab2 ld.lldcomputeKnownBitsFromOperator(I=0x00007ffe4f728c20, DemandedElts=0x00007fff6a9f8ad0, Known=0x00007fff6a9f87d8, Depth=1, Q=0x00007fff6a9f8c10)::Query const&) at ValueTracking.cpp:1279:5 frame #​7: 0x00000000050f9c72 ld.lldcomputeKnownBits(V=0x00007ffe4f728c20, DemandedElts=0x00007fff6a9f8ad0, Known=0x00007fff6a9f87d8, Depth=1, Q=0x00007fff6a9f8c10)::Query const&) at ValueTracking.cpp:1886:5
    frame #​8: 0x0000000005109ef1 ld.lldComputeNumSignBitsImpl(V=0x00007ffe4f728c20, DemandedElts=0x00007fff6a9f8ad0, Depth=1, Q=0x00007fff6a9f8c10)::Query const&) at ValueTracking.cpp:2978:3 frame #​9: 0x0000000005108e7b ld.lldComputeNumSignBits(V=0x00007ffe4f728c20, DemandedElts=0x00007fff6a9f8ad0, Depth=1, Q=0x00007fff6a9f8c10)::Query const&) at ValueTracking.cpp:2652:21
    frame #​10: 0x00000000050fbdb4 ld.lldComputeNumSignBits(V=0x00007ffe4f728c20, Depth=1, Q=0x00007fff6a9f8c10)::Query const&) at ValueTracking.cpp:389:10 frame #​11: 0x0000000005109bb1 ld.lldComputeNumSignBitsImpl(V=0x00007ffe4f728b38, DemandedElts=0x00007fff6a9f8e90, Depth=0, Q=0x00007fff6a9f8f08)::Query const&) at ValueTracking.cpp:2901:18
    frame #​12: 0x0000000005108e7b ld.lldComputeNumSignBits(V=0x00007ffe4f728b38, DemandedElts=0x00007fff6a9f8e90, Depth=0, Q=0x00007fff6a9f8f08)::Query const&) at ValueTracking.cpp:2652:21 frame #​13: 0x00000000050fbdb4 ld.lldComputeNumSignBits(V=0x00007ffe4f728b38, Depth=0, Q=0x00007fff6a9f8f08)::Query const&) at ValueTracking.cpp:389:10
    frame #​14: 0x00000000050fbcff ld.lldllvm::ComputeNumSignBits(V=0x00007ffe4f728b38, DL=0x00007fff49aafad0, Depth=0, AC=0x00007ffe4dc03958, CxtI=0x0000000000000000, DT=0x00007ffe4d49e948, UseInstrInfo=true) at ValueTracking.cpp:396:10 frame #​15: 0x0000000005020edc ld.lldllvm::ScalarEvolution::getRangeRef(this=0x00007fff49bc9d48, S=0x00007ffe531cbcb0, SignHint=HINT_RANGE_SIGNED) at ScalarEvolution.cpp:5854:21
    frame #​16: 0x0000000003e2ed61 ld.lldllvm::ScalarEvolution::getSignedRange(this=0x00007fff49bc9d48, S=0x00007ffe531cbcb0) at ScalarEvolution.h:868:12 frame #​17: 0x0000000005014e4f ld.lldStrengthenNoWrapFlags(SE=0x00007fff49bc9d48, Type=scAddExpr, Ops=const llvm::ArrayRef<const llvm::SCEV *> @ 0x00007fff6a9f9e60, Flags=FlagNUW) at ScalarEvolution.cpp:2262:34
    frame #​18: 0x0000000005013631 ld.lldllvm::ScalarEvolution::getAddExpr(this=0x00007fff6a9fa7d8, Ops=const llvm::ArrayRef<const llvm::SCEV *> @ 0x00007fff6a9f9ea0)::$_0::operator()(llvm::ArrayRef<llvm::SCEV const*>) const at ScalarEvolution.cpp:2324:12 frame #&#8203;19: 0x000000000500b015 ld.lldllvm::ScalarEvolution::getAddExpr(this=0x00007fff49bc9d48, Ops=0x00007fff6a9fa890, OrigFlags=FlagNUW, Depth=0) at ScalarEvolution.cpp:2674:34
    frame #​20: 0x00000000020c3747 ld.lldllvm::ScalarEvolution::getAddExpr(this=0x00007fff49bc9d48, LHS=0x00007ffe531cbcb0, RHS=0x00007ffe4d158920, Flags=FlagNUW, Depth=0) at ScalarEvolution.h:527:12 frame #​21: 0x00000000050153c7 ld.lldllvm::ScalarEvolution::getGEPExpr(this=0x00007fff49bc9d48, GEP=0x00007ffe4f728c20, IndexExprs=0x00007fff6a9faa08) at ScalarEvolution.cpp:3469:10
    frame #​22: 0x000000000501ebcd ld.lldllvm::ScalarEvolution::createNodeForGEP(this=0x00007fff49bc9d48, GEP=0x00007ffe4f728c20) at ScalarEvolution.cpp:5521:10 frame #​23: 0x0000000005019642 ld.lldllvm::ScalarEvolution::createSCEV(this=0x00007fff49bc9d48, V=0x00007ffe4f728c20) at ScalarEvolution.cpp:6733:12
    frame #​24: 0x00000000050154b1 ld.lld`llvm::ScalarEvolution::getSCEV(this=0x00007fff49bc9d48, V=0x00007ffe4f728c20) at ScalarEvolution.cpp:3873:9

This was build from fa0971b

[yuanfang-chen]

Could you please describe how to reproduce the issue?

[omartijn]

I've been trying to create a small reproducible example, but I've not succeeded in this. So far, it only fails while linking one executable in our project. The linker command looks something like this:

"/usr/bin/ld.lld" --hash-style=both --build-id --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o ../../RelWithDebInfo/ /usr/bin/../lib/gcc/x86_64-linux-gnu/8/../../../x86_64-linux-gnu/crt1.o /usr/bin/../lib/gcc/x86_64-linux-gnu/8/../../../x86_64-linux-gnu/crti.o /usr/bin/../lib/gcc/x86_64-linux-gnu/8/crtbegin.o -L/usr/bin/../lib/gcc/x86_64-linux-gnu/8 -L/usr/bin/../lib/gcc/x86_64-linux-gnu/8/../../../x86_64-linux-gnu -L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu -L/usr/bin/../lib/gcc/x86_64-linux-gnu/8/../../.. -L/usr/lib/llvm-12/bin/../lib -L/lib -L/usr/lib -plugin-opt=mcpu=x86-64 -plugin-opt=O2 -plugin-opt=thinlto <56 object files> -rpath <10 static libs> -lXinerama -lXi -lXrandr -lXcursor -lXcomposite -lxkbcommon -lwayland-cursor -lwayland-egl -lwayland-client -lepoxy -ldl -lGL -lpthread -lm -lEGL -lXdamage -lXfixes -lX11-xcb -lxcb-glx -lxcb-dri2 -lXxf86vm -ldrm -lpangoft2-1.0 -lharfbuzz -lgraphite2 -lfribidi -lresolv -lgmodule-2.0 -lmount -lblkid -lselinux -lsepol -lz -lpixman-1 -lfontconfig -luuid -lexpat -lpng16 -lxcb-shm -lxcb-render -lXrender -lXext -lX11 -lxcb -lXau -lXdmcp -lffi -lpcre -latk-bridge-2.0 -latspi -ldbus-1 -lsystemd -lXinerama -lXi -lXrandr -lXcursor -lXcomposite -lxkbcommon -lwayland-cursor -lwayland-egl -lwayland-client -lepoxy -ldl -lGL -lpthread -lm -lEGL -lXdamage -lXfixes -lX11-xcb -lxcb-glx -lxcb-dri2 -lXxf86vm -ldrm -lpangoft2-1.0 -lharfbuzz -lgraphite2 -lfribidi -lresolv -lgmodule-2.0 -lmount -lblkid -lselinux -lsepol -lz -lpixman-1 -lfontconfig -luuid -lexpat -lpng16 -lxcb-shm -lxcb-render -lXrender -lXext -lX11 -lxcb -lXau -lXdmcp -lffi -lpcre -latk-bridge-2.0 -latspi -ldbus-1 -lsystemd /usr/lib/x86_64-linux-gnu/libasound.so -lwebkit2gtk-4.0 -lsoup-2.4 -ljavascriptcoregtk-4.0 -lsoup-2.4 -ljavascriptcoregtk-4.0 /usr/lib/x86_64-linux-gnu/libGLEW.so /usr/lib/x86_64-linux-gnu/libcurl.so <2 static libs> -latomic -ldns_sd -lgtk-3 -lgdk-3 -lpangocairo-1.0 -lpango-1.0 -latk-1.0 -lcairo-gobject -lcairo -lgdk_pixbuf-2.0 -lgio-2.0 -lgobject-2.0 -lglib-2.0 /usr/lib/x86_64-linux-gnu/libxcb-randr.so /usr/lib/x86_64-linux-gnu/libxcb.so ../../../libs/libjpeg-turbo/lib/linux/libturbojpeg.a ../../libs/magma/mevi-gfx/libs/libfreetype.a ../../libs/libGTest.a ../../../libs/ffmpeg/bin/linux/libavcodec.so ../../../libs/ffmpeg/bin/linux/libavformat.so ../../../libs/ffmpeg/bin/linux/libavutil.so ../../../libs/ffmpeg/bin/linux/libswscale.so ../../libs/libEASTL.a ../../libs/libLIBTIFF.a ../../libs/libSnappy.a ../../libs/libDXTR.a ../../libs/libLZ4.a ../../libs/libLZF.a ../../libs/libycocg.a ../../libs/libBlackmagicDecklink.a ../../libs/libOscPack.a ../../libs/libAppleProRes.a <5 static libs> ../../../libs/newtekndi/linux/lib/x86_64-linux-gnu/libndi.so.4.6.1 ../../libs/libartnet/libArtnet.so <3 static libs> /usr/lib/x86_64-linux-gnu/libvulkan.so /usr/lib/x86_64-linux-gnu/libxcb.so /usr/lib/x86_64-linux-gnu/libxcb-shm.so /usr/lib/x86_64-linux-gnu/libxcb-image.so /usr/lib/x86_64-linux-gnu/libxcb-randr.so <1 static lib> ../../libs/magma/mevi-gfx/libs/libeastl.a <7 static libs> ../../libs/libltc/libLTC.so -lstdc++ -lm -lgcc_s -lgcc -lpthread -lc -lgcc_s -lgcc /usr/bin/../lib/gcc/x86_64-linux-gnu/8/crtend.o /usr/bin/../lib/gcc/x86_64-linux-gnu/8/../../../x86_64-linux-gnu/crtn.o

The individual object files are all compiled like this:

/usr/bin/c++ -Werror=return-stack-address -Werror=array-bounds -Werror=return-type -Werror=sentinel -Werror=null-conversion -Werror=unsequenced -Werror=undefined-inline -Werror=abstract-final-class -Werror=inconsistent-missing-override -Wno-deprecated-declarations -Wno-nonportable-include-path -Wno-switch -Wno-format -Wno-integer-overflow -Wno-tautological-undefined-compare -Wno-format-security -Wno-potentially-evaluated-expression -Wno-objc-missing-super-calls -Wno-literal-conversion -Wno-error=unused-private-field -O2 -g -DNDEBUG -flto=thin -fPIE -std=gnu++17 -MD -MT -MF -o -c

I'll continue to see if I can reproduce this in a simpler setup.

[omartijn]

I've tried to simplify it as much as possible. The end-result is two static libraries and a single object file that, when linked together, result in a stack-overflow. They're too big to upload, unfortunately, so I've uploaded them elsewhere. The files are these:

http://resources.martijnotto.nl/files/default/libHTEngine.a
http://resources.martijnotto.nl/files/default/libJUCE.a
http://resources.martijnotto.nl/files/default/SingleTrackTask.cpp.o

This fails when linking like this:

/usr/bin/ld.lld --hash-style=both --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o a.out /usr/lib/x86_64-linux-gnu/crt1.o /usr/lib/x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/8/crtbegin.o -plugin-opt=mcpu=x86-64 -plugin-opt=O2 -plugin-opt=thinlto SingleTrackTask.cpp.o libHTEngine.a libJUCE.a

For some reason, not including either one of the two static libraries, or the object file, makes the linker bail out as expected (missing symbol 'main').

[erikolofsson]

I have the same problem. See the stack trace below. It seems to include the same ScalarEvolution code.

When I increased the stack size in ThreadPool.cpp to 8 MB it completes successfully, so it's a case of using a lot of stack, not an infinite recursion.

I used https://github.com/metarutaiga/HackingSTL/blob/master/thread) to increase stack size.

• frame #​0: 0x0000000112e15321 libLTO.dylibcomputeKnownBitsFromAssume(V=0x00000002e788bce0, Known=0x00007000027b9658, Depth=5, Q=0x00007000027b96b0)::Query const&) at ValueTracking.cpp:641:32 [opt] frame #​1: 0x0000000112dfc9b3 libLTO.dylibcomputeKnownBits(V=, DemandedElts=0x00007000027b9510, Known=0x00007000027b9658, Depth=5, Q=0x00007000027b96b0)::Query const&) at ValueTracking.cpp:1919:3 [opt]
  frame #​2: 0x0000000112dfbf38 libLTO.dylibcomputeKnownBits(V=0x00000002e788bce0, Known=0x00007000027b9658, Depth=5, Q=0x00007000027b96b0)::Query const&) at ValueTracking.cpp:209:3 [opt] frame #​3: 0x0000000112e13b23 libLTO.dylibcomputeKnownBitsFromOperator(I=, DemandedElts=, Known=0x00007000027b98a0, Depth=0, Q=)::Query const&) at ValueTracking.cpp:1493:9 [opt]
  frame #​4: 0x0000000112dfc971 libLTO.dylibcomputeKnownBits(V=<unavailable>, DemandedElts=0x00007000027b97d0, Known=0x00007000027b98a0, Depth=0, Q=0x00007000027b9810)::Query const&) at ValueTracking.cpp:1907:5 [opt] frame #&#8203;5: 0x0000000112dfbf38 libLTO.dylibcomputeKnownBits(V=0x00000002f41a6758, Known=0x00007000027b98a0, Depth=0, Q=0x00007000027b9810)::Query const&) at ValueTracking.cpp:209:3 [opt]
  frame #​6: 0x0000000112dfcc1e libLTO.dylibllvm::computeKnownBits(llvm::Value const*, llvm::DataLayout const&, unsigned int, llvm::AssumptionCache*, llvm::Instruction const*, llvm::DominatorTree const*, llvm::OptimizationRemarkEmitter*, bool) [inlined] computeKnownBits(V=0x00000002f41a6758, Depth=0, Q=0x00007000027b9810)::Query const&) at ValueTracking.cpp:1779:3 [opt] frame #​7: 0x0000000112dfcba8 libLTO.dylibllvm::computeKnownBits(V=0x00000002f41a6758, DL=0x0000000237c24220, Depth=0, AC=, CxtI=, DT=0x0000000332944160, ORE=0x0000000000000000, UseInstrInfo=) at ValueTracking.cpp:242 [opt]
  frame #​8: 0x0000000112d82799 libLTO.dylibllvm::ScalarEvolution::GetMinTrailingZerosImpl(this=0x000000035f846750, S=0x000000036f06d800) at ScalarEvolution.cpp:5615:23 [opt] frame #​9: 0x0000000112d6c75e libLTO.dylibllvm::ScalarEvolution::GetMinTrailingZeros(this=0x000000035f846750, S=) at ScalarEvolution.cpp:5628:21 [opt]
  frame #​10: 0x0000000112d83ded libLTO.dylibllvm::ScalarEvolution::getRangeRef(this=0x000000035f846750, S=0x000000036f06d800, SignHint=HINT_RANGE_SIGNED) at ScalarEvolution.cpp:5798:17 [opt] frame #​11: 0x0000000112d86c01 libLTO.dylibllvm::ScalarEvolution::getRangeRef(this=, S=, SignHint=HINT_RANGE_SIGNED) at ScalarEvolution.cpp:6026:26 [opt]
  frame #​12: 0x0000000112d86c01 libLTO.dylibllvm::ScalarEvolution::getRangeRef(this=<unavailable>, S=<unavailable>, SignHint=HINT_RANGE_SIGNED) at ScalarEvolution.cpp:6026:26 [opt] ... frame #&#8203;442: 0x0000000112d86c01 libLTO.dylibllvm::ScalarEvolution::getRangeRef(this=, S=, SignHint=HINT_RANGE_SIGNED) at ScalarEvolution.cpp:6026:26 [opt]
  frame #​443: 0x0000000112d75b58 libLTO.dylibStrengthenNoWrapFlags(llvm::ScalarEvolution*, llvm::SCEVTypes, llvm::ArrayRef<llvm::SCEV const*>, llvm::SCEV::NoWrapFlags) [inlined] llvm::ScalarEvolution::getSignedRange(this=0x000000035f846750, S=<unavailable>) at ScalarEvolution.h:868:12 [opt] frame #&#8203;444: 0x0000000112d75b4b libLTO.dylibStrengthenNoWrapFlags(SE=0x000000035f846750, Type=, Ops=const llvm::ArrayRef<const llvm::SCEV > @ 0x00007fa9ea8a2110, Flags=) at ScalarEvolution.cpp:2292 [opt]
  frame #​445: 0x0000000112d69bab libLTO.dylibllvm::ScalarEvolution::getAddExpr(llvm::SmallVectorImpl<llvm::SCEV const*>&, llvm::SCEV::NoWrapFlags, unsigned int) [inlined] llvm::ScalarEvolution::getAddExpr(this=<unavailable>, Ops=const llvm::ArrayRef<const llvm::SCEV *> @ 0x00007fa9ead824e0)::$_0::operator()(llvm::ArrayRef<llvm::SCEV const*>) const at ScalarEvolution.cpp:2354:12 [opt] frame #&#8203;446: 0x0000000112d69b91 libLTO.dylibllvm::ScalarEvolution::getAddExpr(this=, Ops=, OrigFlags=FlagAnyWrap, Depth=) at ScalarEvolution.cpp:2704 [opt]
  frame #​447: 0x0000000112d696d9 libLTO.dylibllvm::ScalarEvolution::getAddExpr(this=<unavailable>, Ops=<unavailable>, OrigFlags=FlagAnyWrap, Depth=<unavailable>) at ScalarEvolution.cpp:2639:22 [opt] frame #&#8203;448: 0x0000000112d76256 libLTO.dylibllvm::ScalarEvolution::getGEPExpr(llvm::GEPOperator, llvm::SmallVectorImpl<llvm::SCEV const*> const&) [inlined] llvm::ScalarEvolution::getAddExpr(this=0x000000035f846750, LHS=, RHS=, Flags=FlagAnyWrap, Depth=0) at ScalarEvolution.h:528:12 [opt]
  frame #​449: 0x0000000112d76215 libLTO.dylibllvm::ScalarEvolution::getGEPExpr(this=<unavailable>, GEP=0x00000002f7633f40, IndexExprs=<unavailable>) at ScalarEvolution.cpp:3499 [opt] frame #&#8203;450: 0x0000000112d82652 libLTO.dylibllvm::ScalarEvolution::createNodeForGEP(this=0x000000035f846750, GEP=0x00000002f7633f40) at ScalarEvolution.cpp:5542:10 [opt]
  frame #​451: 0x0000000112d7a4d2 libLTO.dylibllvm::ScalarEvolution::createSCEV(this=<unavailable>, V=<unavailable>) at ScalarEvolution.cpp:6862:12 [opt] frame #&#8203;452: 0x0000000112d7633a libLTO.dylibllvm::ScalarEvolution::getSCEV(this=0x000000035f846750, V=0x00000002f7633f40) at ScalarEvolution.cpp:3898:9 [opt]
  frame #​453: 0x0000000112d90059 libLTO.dylibllvm::ScalarEvolution::computeExitLimitFromICmp(this=0x000000035f846750, L=0x0000000240f931c8, ExitCond=0x00000002f7634060, ExitIfTrue=true, ControlsExit=<unavailable>, AllowPredicates=<unavailable>) at ScalarEvolution.cpp:7805:21 [opt] frame #&#8203;454: 0x0000000112d8f7eb libLTO.dylibllvm::ScalarEvolution::computeExitLimitFromCondImpl(this=, Cache=0x00007000028376f0, L=0x0000000240f931c8, ExitCond=, ExitIfTrue=true, ControlsExit=, AllowPredicates=) at ScalarEvolution.cpp:7667:9 [opt]
  frame #​455: 0x0000000112d8f3ad libLTO.dylibllvm::ScalarEvolution::computeExitLimitFromCondCached(this=0x000000035f846750, Cache=0x00007000028376f0, L=0x0000000240f931c8, ExitCond=0x00000002f7634060, ExitIfTrue=<unavailable>, ControlsExit=<unavailable>, AllowPredicates=<unavailable>) at ScalarEvolution.cpp:7649:18 [opt] frame #&#8203;456: 0x0000000112d8ed98 libLTO.dylibllvm::ScalarEvolution::computeExitLimit(llvm::Loop const*, llvm::BasicBlock*, bool) [inlined] llvm::ScalarEvolution::computeExitLimitFromCond(this=0x000000035f846750, L=0x0000000240f931c8, ExitCond=, ExitIfTrue=, AllowPredicates=false) at ScalarEvolution.cpp:7605:10 [opt]
  frame #​457: 0x0000000112d8ed21 libLTO.dylibllvm::ScalarEvolution::computeExitLimit(this=0x000000035f846750, L=0x0000000240f931c8, ExitingBlock=<unavailable>, AllowPredicates=false) at ScalarEvolution.cpp:7579 [opt] frame #&#8203;458: 0x0000000112d8b8f7 libLTO.dylibllvm::ScalarEvolution::computeBackedgeTakenCount(this=, L=, AllowPredicates=false) at ScalarEvolution.cpp:7510:20 [opt]
  frame #​459: 0x0000000112d8a3c9 libLTO.dylibllvm::ScalarEvolution::getBackedgeTakenInfo(this=0x000000035f846750, L=0x0000000240f931c8) at ScalarEvolution.cpp:7100:30 [opt] frame #​460: 0x0000000112da4a72 libLTO.dylibllvm::ScalarEvolution::hasLoopInvariantBackedgeTakenCount(llvm::Loop const*) [inlined] llvm::ScalarEvolution::getBackedgeTakenCount(this=0x000000035f846750, L=0x0000000240f931c8, Kind=Exact) at ScalarEvolution.cpp:7045:12 [opt]
  frame #​461: 0x0000000112da4a6d libLTO.dylibllvm::ScalarEvolution::hasLoopInvariantBackedgeTakenCount(this=0x000000035f846750, L=0x0000000240f931c8) at ScalarEvolution.cpp:12236 [opt] frame #​462: 0x000000011245269e libLTO.dylib(anonymous namespace)::LoopIdiomRecognize::runOnLoop(this=0x0000700002838010, L=0x0000000240f931c8) at LoopIdiomRecognize.cpp:379:13 [opt]
  frame #​463: 0x0000000112454655 libLTO.dylib(anonymous namespace)::LoopIdiomRecognizeLegacyPass::runOnLoop(this=<unavailable>, L=0x0000000240f931c8, LPM=<unavailable>) at LoopIdiomRecognize.cpp:296:16 [opt] frame #&#8203;464: 0x0000000112d00586 libLTO.dylibllvm::LPPassManager::runOnFunction(this=, F=) at LoopPass.cpp:198:27 [opt]
  frame #​465: 0x00000001115c9112 libLTO.dylibllvm::FPPassManager::runOnFunction(this=<unavailable>, F=<unavailable>) at LegacyPassManager.cpp:1439:27 [opt] frame #&#8203;466: 0x0000000112bf72b4 libLTO.dylib(anonymous namespace)::CGPassManager::runOnModule(llvm::Module&) at CallGraphSCCPass.cpp:180:25 [opt]
  frame #​467: 0x0000000112bf71f8 libLTO.dylib(anonymous namespace)::CGPassManager::runOnModule(llvm::Module&) at CallGraphSCCPass.cpp:478 [opt] frame #​468: 0x0000000112bf6ff4 libLTO.dylib(anonymous namespace)::CGPassManager::runOnModule(this=, M=) at CallGraphSCCPass.cpp:543 [opt]
  frame #​469: 0x00000001115c973e libLTO.dylibllvm::legacy::PassManagerImpl::run(llvm::Module&) at LegacyPassManager.cpp:1554:27 [opt] frame #​470: 0x00000001115c9498 libLTO.dylibllvm::legacy::PassManagerImpl::run(this=0x00000001e884f000, M=) at LegacyPassManager.cpp:542 [opt]
  frame #​471: 0x0000000111a6f746 libLTO.dylib(anonymous namespace)::optimizeModule(TheModule=0x0000000237c24110, TM=0x00000001e5d4a400, OptLevel=3, Freestanding=<unavailable>, Index=0x0000000232cea090) at ThinLTOCodeGenerator.cpp:262:6 [opt] frame #&#8203;472: 0x0000000111a754db libLTO.dylibstd::__1::__function::__func<std::__1::__bind<llvm::ThinLTOCodeGenerator::run()::$_3, int&>, std::__1::allocator<std::__1::__bind<llvm::ThinLTOCodeGenerator::run()::$_3, int&> >, void ()>::operator()() [inlined] (anonymous namespace)::ProcessThinLTOModule(TheModule=0x0000000237c24110, Index=0x0000000232cea090, ModuleMap=0x00007ffeefbfae60, TM=0x00000001e5d4a400, ImportList=, ExportList=0x00000002375fe148, GUIDPreservedSymbols=0x00007ffeefbfaf20, DefinedGlobals=, CacheOptions=, SaveTempsDir=(Data = 0x232329ab9 "", Length = 0), OptLevel=3, count=) at ThinLTOCodeGenerator.cpp:464:3 [opt]
  frame #​473: 0x0000000111a753c0 libLTO.dylibstd::__1::__function::__func<std::__1::__bind<llvm::ThinLTOCodeGenerator::run()::$_3, int&>, std::__1::allocator<std::__1::__bind<llvm::ThinLTOCodeGenerator::run()::$_3, int&> >, void ()>::operator()() [inlined] llvm::ThinLTOCodeGenerator::run(this=0x0000000238408808, count=<unavailable>)::$_3::operator()(int) const at ThinLTOCodeGenerator.cpp:1131 [opt] frame #&#8203;474: 0x0000000111a745a0 libLTO.dylibstd::__1::__function::__func<std::__1::__bind<llvm::ThinLTOCodeGenerator::run()::$_3, int&>, std::__1::allocator<std::__1::__bind<llvm::ThinLTOCodeGenerator::run()::$_3, int&> >, void ()>::operator()() [inlined] decltype(__f=0x0000000238408808)::$_3&>(fp)(std::__1::forward<int&>(fp0))) std::__1::__invoke<llvm::ThinLTOCodeGenerator::run()::$_3&, int&>(llvm::ThinLTOCodeGenerator::run()::$_3&, int&) at type_traits:3545 [opt]
  frame #​475: 0x0000000111a745a0 libLTO.dylibstd::__1::__function::__func<std::__1::__bind<llvm::ThinLTOCodeGenerator::run()::$_3, int&>, std::__1::allocator<std::__1::__bind<llvm::ThinLTOCodeGenerator::run()::$_3, int&> >, void ()>::operator()() [inlined] std::__1::__bind_return<llvm::ThinLTOCodeGenerator::run()::$_3, std::__1::tuple<int>, std::__1::tuple<>, __is_valid_bind_return<llvm::ThinLTOCodeGenerator::run()::$_3, std::__1::tuple<int>, std::__1::tuple<> >::value>::type std::__1::__apply_functor<llvm::ThinLTOCodeGenerator::run(__f=0x0000000238408808, __args=<unavailable>)::$_3, std::__1::tuple<int>, 0ul, std::__1::tuple<> >(llvm::ThinLTOCodeGenerator::run()::$_3&, std::__1::tuple<int>&, std::__1::__tuple_indices<0ul>, std::__1::tuple<>&&) at functional:2845 [opt] frame #&#8203;476: 0x0000000111a745a0 libLTO.dylibstd::__1::__function::__func<std::__1::__bind<llvm::ThinLTOCodeGenerator::run()::$_3, int&>, std::__1::allocator<std::__1::__bind<llvm::ThinLTOCodeGenerator::run()::$_3, int&> >, void ()>::operator()() [inlined] std::__1::__bind_return<llvm::ThinLTOCodeGenerator::run()::$_3, std::__1::tuple, std::__1::tuple<>, __is_valid_bind_return<llvm::ThinLTOCodeGenerator::run()::$_3, std::__1::tuple, std::__1::tuple<> >::value>::type std::__1::__bind<llvm::ThinLTOCodeGenerator::run(this=0x0000000238408808)::$_3, int&>::operator()<>() at functional:2878 [opt]
  frame #​477: 0x0000000111a74595 libLTO.dylibstd::__1::__function::__func<std::__1::__bind<llvm::ThinLTOCodeGenerator::run()::$_3, int&>, std::__1::allocator<std::__1::__bind<llvm::ThinLTOCodeGenerator::run()::$_3, int&> >, void ()>::operator()() [inlined] decltype(__f=0x0000000238408808)::$_3, int&>&>(fp)()) std::__1::__invoke<std::__1::__bind<llvm::ThinLTOCodeGenerator::run()::$_3, int&>&>(std::__1::__bind<llvm::ThinLTOCodeGenerator::run()::$_3, int&>&) at type_traits:3545 [opt] frame #&#8203;478: 0x0000000111a74595 libLTO.dylibstd::__1::__function::__func<std::__1::__bind<llvm::ThinLTOCodeGenerator::run()::$_3, int&>, std::__1::allocator<std::__1::__bind<llvm::ThinLTOCodeGenerator::run()::$_3, int&> >, void ()>::operator()() [inlined] void std::__1::__invoke_void_return_wrapper::__call<std::__1::__bind<llvm::ThinLTOCodeGenerator::run(__args=0x0000000238408808)::$_3, int&>&>(std::__1::__bind<llvm::ThinLTOCodeGenerator::run()::$_3, int&>&) at __functional_base:348 [opt]
  frame #​479: 0x0000000111a74595 libLTO.dylibstd::__1::__function::__func<std::__1::__bind<llvm::ThinLTOCodeGenerator::run()::$_3, int&>, std::__1::allocator<std::__1::__bind<llvm::ThinLTOCodeGenerator::run()::$_3, int&> >, void ()>::operator()() [inlined] std::__1::__function::__alloc_func<std::__1::__bind<llvm::ThinLTOCodeGenerator::run()::$_3, int&>, std::__1::allocator<std::__1::__bind<llvm::ThinLTOCodeGenerator::run()::$_3, int&> >, void ()>::operator(this=0x0000000238408808)() at functional:1546 [opt] frame #&#8203;480: 0x0000000111a74595 libLTO.dylibstd::__1::__function::__func<std::__1::__bind<llvm::ThinLTOCodeGenerator::run()::$_3, int&>, std::__1::allocator<std::__1::__bind<llvm::ThinLTOCodeGenerator::run()::$_3, int&> >, void ()>::operator(this=0x0000000238408800)() at functional:1720 [opt]
  frame #​481: 0x0000000111ba3a5e libLTO.dylibvoid* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, llvm::ThreadPool::ThreadPool(llvm::ThreadPoolStrategy)::$_0> >(void*) [inlined] std::__1::__packaged_task_function<void ()>::operator()() const at future:1993:12 [opt] frame #&#8203;482: 0x0000000111ba3a58 libLTO.dylibvoid* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_deletestd::__1::__thread_struct >, llvm::ThreadPool::ThreadPool(llvm::ThreadPoolStrategy)::$_0> >(void*) [inlined] std::__1::packaged_task<void ()>::operator(this=0x0000700002838f40)() at future:2213 [opt]
  frame #​483: 0x0000000111ba3a1c libLTO.dylibvoid* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, llvm::ThreadPool::ThreadPool(llvm::ThreadPoolStrategy)::$_0> >(void*) at ThreadPool.cpp:179 [opt] frame #&#8203;484: 0x0000000111ba37bd libLTO.dylibvoid* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_deletestd::__1::__thread_struct >, llvm::ThreadPool::ThreadPool(llvm::ThreadPoolStrategy)::$_0> >(void*) [inlined] decltype(__f=0x0000000237ad3b28)::$_0>(fp)()) std::__1::__invokellvm::ThreadPool::ThreadPool(llvm::ThreadPoolStrategy)::$_0(llvm::ThreadPool::ThreadPool(llvm::ThreadPoolStrategy)::$_0&&) at type_traits:3545 [opt]
  frame #​485: 0x0000000111ba37bd libLTO.dylibvoid* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, llvm::ThreadPool::ThreadPool(llvm::ThreadPoolStrategy)::$_0> >(void*) [inlined] void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, llvm::ThreadPool::ThreadPool(llvm::ThreadPoolStrategy)::$_0>(__t=size=2)::$_0>&, std::__1::__tuple_indices<>) at thread:273 [opt] frame #&#8203;486: 0x0000000111ba37bd libLTO.dylibvoid* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_deletestd::__1::__thread_struct >, llvm::ThreadPool::ThreadPool(llvm::ThreadPoolStrategy)::$_0> >(__vp=0x0000000237ad3b20) at thread:284 [opt]
  frame #​487: 0x00007fff20677954 libsystem_pthread.dylib_pthread_start + 224 frame #​488: 0x00007fff206734a7 libsystem_pthread.dylibthread_start + 15

[omartijn]

linux-stacksize-workaround.patch
I wasn't able to get lld to link using the HackingSTL link you pasted, but I was able to get the stacksize increased using the attached patch. It only works on linux I think (since it uses non-posix extensions to pthreads).

For me, it only worked when I increased the stack size to 16MB. I'm not sure what the real solution to this is, though. Just increasing the stack size for everybody feels like an ugly hack.

[fhahn]

This might be the same stack overflow as in #44546 & #32731.

@omartijn could you try if applying the patch from https://reviews.llvm.org/D114650 fixes the issue?

[omartijn]

Applied the patch on current main. Didn't work. I reported the stack trace on Phabricator since I guess the person writing the patch is the right one to look at it.

[fhahn]

Thanks. Would it be possible to extract the whole module before running the SLPVectorizer and sharing the module (possibly with just the bits required for the crashing function using llvm-extract)? A hacky way to do that would be to just add F.getParent()->dump() to llvm::SLPVectorizerPass::run

[omartijn]

Hmm, I only now saw that you're the same user who made the patch. I tried to do your "hacky way" (I really don't know too much about the compiler internals), and while that compiled it doesn't link:

/usr/bin/ld: ../../lib/libLLVMVectorize.a(SLPVectorizer.cpp.o): in function `llvm::SLPVectorizerPass::runImpl(llvm::Function&, llvm::ScalarEvolution*, llvm::TargetTransformInfo*, llvm::TargetLibraryInfo*, llvm::AAResults*, llvm::LoopInfo*, llvm::DominatorTree*, llvm::AssumptionCache*, llvm::DemandedBits*, llvm::OptimizationRemarkEmitter*)':
SLPVectorizer.cpp:(.text._ZN4llvm17SLPVectorizerPass7runImplERNS_8FunctionEPNS_15ScalarEvolutionEPNS_19TargetTransformInfoEPNS_17TargetLibraryInfoEPNS_9AAResultsEPNS_8LoopInfoEPNS_13DominatorTreeEPNS_15AssumptionCacheEPNS_12DemandedBitsEPNS_25OptimizationRemarkEmitterE+0x28): undefined reference to `llvm::Module::dump() const'

I don't really understand why. Module::dump() seems to be implemented in llvm/lib/IR/AsmWriter.cpp, which is part of the LLVMCore component library - assuming that add_llvm_component_library does what I expect (something like add_library).

The LLVMVectorize library already links the Core component. I tried adding LLVMCore under LINK_COMPONENTS, but that's clearly not the way to go.

[fhahn]

I don't really understand why.

Module::dump unfortunately isn't available in a release build. You need to build with assertions enabled by passing -DLLVM_ENABLE_ASSERTIONS=On

[omartijn]

Ah, that work to get the linker linked.

When trying to actually use the generated linker I cancelled when the file I was dumping to reached 3.6 GB. I don't think that's a useful way of solving this.

[fhahn]

Yeah it will create a lot of output, the IR dump is likely very big. Dumping in SLPVectorizerPass::run means it will dump the whole module for each function in the module, until it reaches the function that causes the crash. If you know the name of the function in which LLVM crashes, you could only dump when SLPVectorizerPass::run excuses for the function (if (F.getName() == "my crashing fn name")).

Being able to provide single IR module that can be used to reproduce the crash makes it much easier for people to investigate the issue :)