Recent regression produces crash in clang

[dcb314]

eggwrapbox-e1f21d.c.gz
eggwrapbox-e1f21d.sh.gz

I struggled to reproduce the bug here, so I hand it over to folks with more idea.

[nikic]

Can't reproduce a crash on current main. What's the backtrace you got?

[dcb314]

 #0 0x0000000001e1660b llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/home/dcb38/llvm/results.20240131/bin/clang+0x1e1660b)
 #1 0x0000000001e14444 llvm::sys::CleanupOnSignal(unsigned long) (/home/dcb38/llvm/results.20240131/bin/clang+0x1e14444)
 #2 0x0000000001d59090 CrashRecoverySignalHandler(int) CrashRecoveryContext.cpp:0:0
 #3 0x00007f7ee985fbb0 __restore_rt (/lib64/libc.so.6+0x3dbb0)
 #4 0x00000000053177bc clang::Lexer::LexStringLiteral(clang::Token&, char const*, clang::tok::TokenKind) (/home/dcb38/llvm/results.20240131/bin/clang+0x53177bc)
 #5 0x000000000531885f clang::Lexer::LexTokenInternal(clang::Token&, bool) (/home/dcb38/llvm/results.20240131/bin/clang+0x531885f)
 #6 0x000000000298e049 clang::TextDiagnostic::emitSnippetAndCaret(clang::FullSourceLoc, clang::DiagnosticsEngine::Level, llvm::SmallVectorImpl<clang::CharSourceRange>&, llvm::ArrayRef<clang::FixItHint>) (/home/dcb38/llvm/results.20240131/bin/clang+0x298e049)
 #7 0x0000000002976e9a clang::DiagnosticRenderer::emitDiagnostic(clang::FullSourceLoc, clang::DiagnosticsEngine::Level, llvm::StringRef, llvm::ArrayRef<clang::CharSourceRange>, llvm::ArrayRef<clang::FixItHint>, llvm::PointerUnion<clang::Diagnostic const*, clang::StoredDiagnostic const*>) (/home/dcb38/llvm/results.20240131/bin/clang+0x2976e9a)
 #8 0x00000000029779fb clang::DiagnosticRenderer::emitSingleMacroExpansion(clang::FullSourceLoc, clang::DiagnosticsEngine::Level, llvm::ArrayRef<clang::CharSourceRange>) (/home/dcb38/llvm/results.20240131/bin/clang+0x29779fb)

[llvmbot]

@llvm/issue-subscribers-clang-frontend

Author: None (dcb314)

[eggwrapbox-e1f21d.c.gz](https://github.com/llvm/llvm-project/files/14110352/eggwrapbox-e1f21d.c.gz) [eggwrapbox-e1f21d.sh.gz](https://github.com/llvm/llvm-project/files/14110377/eggwrapbox-e1f21d.sh.gz)

I struggled to reproduce the bug here, so I hand it over to folks with more idea.

[tbaederr]

Sounds like my fault.

[tbaederr]

I can't reproduce either though, the output I get is:

../src/widgets/eggwrapbox.c:322:3: warning: 'g_type_class_add_private' is deprecated [-Wdeprecated-declarations]
  322 |   g_type_class_add_private (class, sizeof (EggWrapBoxPrivate));
      |   ^
/usr/include/glib-2.0/gobject/gtype.h:1382:1: note: 'g_type_class_add_private' has been explicitly marked deprecated here
 1382 | GOBJECT_DEPRECATED_IN_2_58
      | ^
/usr/include/glib-2.0/gobject/gobject-visibility.h:581:36: note: expanded from macro 'GOBJECT_DEPRECATED_IN_2_58'
  581 | #define GOBJECT_DEPRECATED_IN_2_58 GOBJECT_DEPRECATED
      |                                    ^
/usr/include/glib-2.0/gobject/gobject-visibility.h:30:28: note: expanded from macro 'GOBJECT_DEPRECATED'
   30 | #define GOBJECT_DEPRECATED G_DEPRECATED _GOBJECT_EXTERN
      |                            ^
/usr/include/glib-2.0/glib/gmacros.h:1262:37: note: expanded from macro 'G_DEPRECATED'
 1262 | #define G_DEPRECATED __attribute__((__deprecated__))
      |                                     ^
../src/widgets/eggwrapbox.c:331:5: warning: Deprecated pre-processor symbol: replace with "G_ADD_PRIVATE" [-W#pragma-messages]
  331 |     G_TYPE_INSTANCE_GET_PRIVATE (box, EGG_TYPE_WRAP_BOX, EggWrapBoxPrivate);
      |     ^
/usr/include/glib-2.0/gobject/gtype.h:686:145: note: expanded from macro 'G_TYPE_INSTANCE_GET_PRIVATE'
  686 | #define G_TYPE_INSTANCE_GET_PRIVATE(instance, g_type, c_type)   ((c_type*) g_type_instance_get_private ((GTypeInstance*) (instance), (g_type))) GOBJECT_DEPRECATED_MACRO_IN_2_58_FOR(G_ADD_PRIVATE)
      |                                                                                                                                                 ^
/usr/include/glib-2.0/gobject/gobject-visibility.h:584:49: note: expanded from macro 'GOBJECT_DEPRECATED_MACRO_IN_2_58_FOR'
  584 | #define GOBJECT_DEPRECATED_MACRO_IN_2_58_FOR(f) GLIB_DEPRECATED_MACRO_FOR (f)
      |                                                 ^
/usr/include/glib-2.0/glib/gmacros.h:1299:3: note: expanded from macro 'GLIB_DEPRECATED_MACRO_FOR'
 1299 |   _GLIB_GNUC_DO_PRAGMA(GCC warning G_STRINGIFY (Deprecated pre-processor symbol: replace with #f))
      |   ^
/usr/include/glib-2.0/glib/gmacros.h:1296:33: note: expanded from macro '_GLIB_GNUC_DO_PRAGMA'
 1296 | #define _GLIB_GNUC_DO_PRAGMA(x) _Pragma(G_STRINGIFY (x))
      |                                 ^
<scratch space>:47:6: note: expanded from here
   47 |  GCC warning "Deprecated pre-processor symbol: replace with \"G_ADD_PRIVATE\""
      |      ^
2 warnings generated.

Do you have an in-development version of any of the dependencies installed?

[dcb314]

Do you have an in-development version of any of the dependencies installed?

I am not sure I understand your question. I have enough installed to start the compilation,
but I don't think that was the question you were asking.

Here is what looks like a duplicate:

test-gom-sorting-44de12.sh.gz
test-gom-sorting-44de12.c.gz

Stack backtrace is

 #0 0x0000000001e1660b llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/home/dcb38/llvm/results.20240131/bin/clang+0x1e1660b)
 #1 0x0000000001e14444 llvm::sys::CleanupOnSignal(unsigned long) (/home/dcb38/llvm/results.20240131/bin/clang+0x1e14444)
 #2 0x0000000001d59090 CrashRecoverySignalHandler(int) CrashRecoveryContext.cpp:0:0
 #3 0x00007f90ee37ebb0 __restore_rt (/lib64/libc.so.6+0x3dbb0)
 #4 0x0000000005310fbc clang::Lexer::SkipWhitespace(clang::Token&, char const*, bool&) (/home/dcb38/llvm/results.20240131/bin/clang+0x5310fbc)
 #5 0x0000000005317bee clang::Lexer::LexTokenInternal(clang::Token&, bool) (/home/dcb38/llvm/results.20240131/bin/clang+0x5317bee)
 #6 0x000000000298e049 clang::TextDiagnostic::emitSnippetAndCaret(clang::FullSourceLoc, clang::DiagnosticsEngine::Level, llvm::SmallVectorImpl<clang::CharSourceRange>&, llvm::ArrayRef<clang::FixItHint>) (/home/dcb38/llvm/results.20240131/bin/clang+0x298e049)
 #7 0x0000000002976e9a clang::DiagnosticRenderer::emitDiagnostic(clang::FullSourceLoc, clang::DiagnosticsEngine::Level, llvm::StringRef, llvm::ArrayRef<clang::CharSourceRange>, llvm::ArrayRef<clang::FixItHint>, llvm::PointerUnion<clang::Diagnostic const*, clang::StoredDiagnostic const*>) (/home/dcb38/llvm/results.20240131/bin/clang+0x2976e9a)
 #8 0x00000000029779fb clang::DiagnosticRenderer::emitSingleMacroExpansion(clang::FullSourceLoc, clang::DiagnosticsEngine::Level, llvm::ArrayRef<clang::CharSourceRange>) (/home/dcb38/llvm/results.20240131/bin/clang+0x29779fb)

[tbaederr]

Do you have an in-development version of any of the dependencies installed?

I am not sure I understand your question. I have enough installed to start the compilation, but I don't think that was the question you were asking.

What version of glib and other dependencies your test case uses do you have installed?

Here is what looks like a duplicate:

test-gom-sorting-44de12.sh.gz test-gom-sorting-44de12.c.gz

No luck here either. :(

[dcb314]

What version of glib and other dependencies your test case uses do you have installed?

glib2 is at version 2.76.6, but I don't see how that helps.

No luck here either. :(

I am happy to write this bug report off as can't reproduce ;-|

[Endilll]

I compiled Clang at b5c0b67 in release configuration with assertions enabled, and can't reproduce the crash using either of the examples.

[Endilll]

@dcb314 Can you try to reduce your reproducers using creduce or cvise?

[dcb314]

@dcb314 Can you try to reduce your reproducers using creduce or cvise?

Sadly no. See original comment. I struggle to reproduce it here.
It works fine for me at -O1, -O2, -O3, -O3 -march=znver3.

I suspect some other flag from the reproducer is required, but that is much later
in the process than the lexer, where this bug seems to be.

[tbaederr]

I suspect some other flag from the reproducer is required, but that is much later
in the process than the lexer, where this bug seems to be.

The crash happens when emitting a diagnostic, where we no re-lex part of the file.

[dcb314]

The crash happens when emitting a diagnostic, where we no re-lex part of the file.

Thanks for the extra detail.

I tried -g -O3 -march=native -Wall -Wextra and git revision
65066c0 coped fine.

I even added -Winvalid-pch and -Werror=format-security, from the reproducer
and still it compiled ok.

I am still happy to write this one off as can't reproduce.

[bgra8]

@dcb314 are you getting this crash with some open source libraries?

If that is the case can you share what libraries I'd need to configure and build and any other specific steps required to repro the crash?

[dcb314]

@dcb314 are you getting this crash with some open source libraries?

Yes, packages almanah & gom so far. From Fedora rawhide.

If that is the case can you share what libraries I'd need to configure and build and any other specific
steps required to repro the crash?

Not a feasible idea. I have a rather contrived development environment here.
Basically compile Fedora rawhide with today's version of clang (with -g -O3 -march=znver3
switched on).

I have rebuilt today's clang with assertions enabled and now get this stack backtrace:

Stack dump:

0.  Program arguments: /home/dcb38/llvm/results.20240202/bin/clang -g -O3 -march=native -Wall -Isrc/almanah.p -Isrc -I../src -I. -I.. -I../src/events -I../src/event-factories -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -I/usr/include/sysprof-4 -I/usr/include/gtk-3.0 -I/usr/include/pango-1.0 -I/usr/include/harfbuzz -I/usr/include/freetype2 -I/usr/include/libpng16 -I/usr/include/libmount -I/usr/include/blkid -I/usr/include/fribidi -I/usr/include/libxml2 -I/usr/include/cairo -I/usr/include/pixman-1 -I/usr/include/gdk-pixbuf-2.0 -I/usr/include/gio-unix-2.0 -I/usr/include/cloudproviders -I/usr/include/atk-1.0 -I/usr/include/at-spi2-atk/2.0 -I/usr/include/at-spi-2.0 -I/usr/include/dbus-1.0 -I/usr/lib64/dbus-1.0/include -I/usr/include/gtksourceview-3.0 -I/usr/include/gcr-4 -I/usr/include/p11-kit-1 -I/usr/include/gck-2 -I/usr/include/libcryptui -I/usr/include/libassuan2 -I/usr/include/evolution-data-server -I/usr/include/libsecret-1 -I/usr/include/libsoup-3.0 -I/usr/include/json-glib-1.0 -I/usr/include/gtkspell-3.0 -I/usr/include/enchant-2 -fdiagnostics-color=always -DNDEBUG -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -ffat-lto-objects -fexceptions -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -fasynchronous-unwind-tables -fstack-clash-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -pthread -MD -MQ src/almanah.p/widgets_eggwrapbox.c.o -MF src/almanah.p/widgets_eggwrapbox.c.o.d -o src/almanah.p/widgets_eggwrapbox.c.o -c ../src/widgets/eggwrapbox.c
1.  ../src/widgets/eggwrapbox.c:331:5 <Spelling=/usr/include/glib-2.0/glib/gmacros.h:1296:56>: current parser token ')'
2.  ../src/widgets/eggwrapbox.c:327:1: parsing function body 'egg_wrap_box_init'
3.  ../src/widgets/eggwrapbox.c:327:1: in compound statement ('{}')
 #0 0x0000000001faeb3b llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/home/dcb38/llvm/results.20240202/bin/clang+0x1faeb3b)
 #1 0x0000000001fac0b4 llvm::sys::CleanupOnSignal(unsigned long) (/home/dcb38/llvm/results.20240202/bin/clang+0x1fac0b4)
 #2 0x0000000001ef0c40 CrashRecoverySignalHandler(int) CrashRecoveryContext.cpp:0:0
 #3 0x00007f9d5bc5fbb0 __restore_rt (/lib64/libc.so.6+0x3dbb0)
 #4 0x00007f9d5bcb0884 __pthread_kill_implementation (/lib64/libc.so.6+0x8e884)
 #5 0x00007f9d5bc5fafe gsignal (/lib64/libc.so.6+0x3dafe)
 #6 0x00007f9d5bc4887f abort (/lib64/libc.so.6+0x2687f)
 #7 0x00007f9d5bc4879b _nl_load_domain.cold (/lib64/libc.so.6+0x2679b)
 #8 0x00007f9d5bc58187 (/lib64/libc.so.6+0x36187)
 #9 0x0000000002bb92fe clang::TextDiagnostic::emitSnippetAndCaret(clang::FullSourceLoc, clang::DiagnosticsEngine::Level, llvm::SmallVectorImpl<clang::CharSourceRange>&, llvm::ArrayRef<clang::FixItHint>) (/home/dcb38/llvm/results.20240202/bin/clang+0x2bb92fe)

So the bug is reproducible here.

I don't have enough compute power here to build clang with debug switched on (-g),
so we can see which line is crashing ;-<

My clang build script is the following:

cmake -S /home/dcb38/llvm/trunk/llvm -B /home/dcb38/llvm/working
-G "Unix Makefiles"
-DLLVM_ENABLE_PROJECTS="clang"
-DLLVM_TARGETS_TO_BUILD="host"
-DLLVM_ENABLE_ASSERTIONS=ON
-DLLVM_PARALLEL_LINK_JOBS=1
-DCMAKE_INSTALL_PREFIX=/home/dcb38/llvm/results.$DATE
-DCMAKE_C_COMPILER="/home/dcb38/gcc/results/bin/gcc"
-DCMAKE_C_FLAGS="-O1"
-DCMAKE_CXX_COMPILER="/home/dcb38/gcc/results/bin/g++"
-DCMAKE_CXX_FLAGS="-O1"
-DCMAKE_BUILD_TYPE=Release

If anyone can remove unneeded features from that script, but still build clang,
I would be very interested.

Perhaps there is a debug flag (other than -g) that is really lightweight ?.

[bgra8]

@dcb314 can you please sync clang after a986f5e and try with the new version?

That should fix your crash!

[dcb314]

My clang build script is the following:

-DLLVM_PARALLEL_LINK_JOBS=1

Despite having this, there are still 6+ links running in parallel on a make with -j 16.
WTF ? I don't think that flag works ?

Perhaps there is a debug flag (other than -g) that is really lightweight ?.

-g on its own is -g2. I am trying again with -g1, but even with 16 Gig RAM, there
is a lot of swapping going on.

I also checked the number of binaries that my build script produces.

bin $ pwd
/home/dcb38/llvm/results/bin
bin $ ls | wc -l
103

I am somewhat sceptical that all 103 binaries are the minimal set
to get clang and clang++ to work.

bin $ ls -C | more
amdgpu-arch llvm-cov llvm-opt-report
analyze-build llvm-c-test llvm-otool
bugpoint llvm-cvtres llvm-pdbutil
c-index-test llvm-cxxdump llvm-profdata
clang llvm-cxxfilt llvm-profgen
clang++ llvm-cxxmap llvm-ranlib
clang-19 llvm-debuginfo-analyzer llvm-rc
clang-check llvm-debuginfod llvm-readelf
clang-cl llvm-debuginfod-find llvm-readobj
clang-cpp llvm-diff llvm-readtapi
clang-extdef-mapping llvm-dis llvm-reduce
clang-format llvm-dlltool llvm-remarkutil
clang-linker-wrapper llvm-dwarfdump llvm-rtdyld
clang-offload-bundler llvm-dwarfutil llvm-sim
clang-offload-packager llvm-dwp llvm-size
clang-refactor llvm-exegesis llvm-split
clang-rename llvm-extract llvm-stress
clang-repl llvm-gsymutil llvm-strings
clang-scan-deps llvm-ifs llvm-strip
clang-tblgen llvm-install-name-tool llvm-symbolizer
diagtool llvm-jitlink llvm-tblgen
dsymutil llvm-lib llvm-tli-checker
git-clang-format llvm-libtool-darwin llvm-undname
hmaptool llvm-link llvm-windres
intercept-build llvm-lipo llvm-xray
llc llvm-lto nvptx-arch
lli llvm-lto2 opt
llvm-addr2line llvm-mc sancov
llvm-ar llvm-mca sanstats
llvm-as llvm-ml scan-build
llvm-bcanalyzer llvm-modextract scan-build-py
llvm-bitcode-strip llvm-mt scan-view
llvm-cat llvm-nm verify-uselistorder

Now, how to not build all the ones I don't need ?

[dcb314]

@dcb314 can you please sync clang after a986f5e and try with the new version?

That should fix your crash!

Indeed it does. Thanks for that !