Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

clang-tidy asserts and fails to process valid preprocessed source when in C++20 mode only due to ExpandModularHeadersPPCallbacks #47910

Closed
jblazquez mannequin opened this issue Dec 21, 2020 · 3 comments
Closed

clang-tidy asserts and fails to process valid preprocessed source when in C++20 mode only due to ExpandModularHeadersPPCallbacks #47910

jblazquez mannequin opened this issue Dec 21, 2020 · 3 comments
Labels
bugzilla Issues migrated from bugzilla c++20 clang-tidy confirmed Verified by a second party crash Prefer [crash-on-valid] or [crash-on-invalid]

Comments

@jblazquez
Copy link
Mannequin

jblazquez mannequin commented Dec 21, 2020 •
edited by Endilll
Loading

Bugzilla Link 48566
Version unspecified
OS All

Extended Description

Using the latest commit as of writing (3fa2d37), clang-tidy fails to process the following preprocessed source:

	# 1 "main.cpp"
	# 1 "usr/include/sys/cdefs.h" 1
	# 1 "usr/include/android/api-level.h" 1
	# 47 "usr/include/android/api-level.h"
	extern "C" {
	# 170 "usr/include/android/api-level.h"
	#if 1
	# 1 "usr/include/bits/get_device_api_level_inlines.h" 1
	# 30 "usr/include/bits/get_device_api_level_inlines.h"
	# 32 "usr/include/bits/get_device_api_level_inlines.h"
	# 33 "usr/include/bits/get_device_api_level_inlines.h"
	# 34 "usr/include/bits/get_device_api_level_inlines.h"
	# 51 "usr/include/bits/get_device_api_level_inlines.h"
	#endif
	# 177 "usr/include/android/api-level.h" 2
	# 191 "usr/include/android/api-level.h"
	}
	# 363 "usr/include/sys/cdefs.h" 2
	# 1 "main.cpp" 2

The following command line was used:

	clang-tidy main.cpp -- -std=c++20

In Release, clang-tidy outputs this error:

	1 error generated.
	Error while processing main.cpp.
	main.cpp:19:16: error: invalid line marker flag '2': cannot pop empty include stack [clang-diagnostic-error]
	# 1 "main.cpp" 2
	               ^
	Found compiler error(s).

In Debug, clang-tidy crashes with this assert:

	clang-tidy: /tmp/llvm-project/clang/lib/Basic/SourceManager.cpp:210: void clang::LineTableInfo::AddLineNote(clang::FileID, unsigned int, unsigned int, int, unsigned int, clang::SrcMgr::CharacteristicKind): Assertion `(Entries.empty() || Entries.back().FileOffset < Offset) && "Adding line entries out of order!"' failed.
	PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace.
	Stack dump:
	0.      Program arguments: ./clang-tidy /tmp/main.cpp -- -std=c++20
	1.      main.cpp:0:3: current parser token '1'
	 #&#8203;0 0x000055cffd08ed58 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) /tmp/llvm-project/llvm/lib/Support/Unix/Signals.inc:563:0
	 #&#8203;1 0x000055cffd08ee0f PrintStackTraceSignalHandler(void*) /tmp/llvm-project/llvm/lib/Support/Unix/Signals.inc:630:0
	 #&#8203;2 0x000055cffd08caf3 llvm::sys::RunSignalHandlers() /tmp/llvm-project/llvm/lib/Support/Signals.cpp:71:0
	 #&#8203;3 0x000055cffd08e6da SignalHandler(int) /tmp/llvm-project/llvm/lib/Support/Unix/Signals.inc:405:0
	 #&#8203;4 0x00007fb179480980 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x12980)
	 #&#8203;5 0x00007fb178335fb7 gsignal /build/glibc-S7xCS9/glibc-2.27/signal/../sysdeps/unix/sysv/linux/raise.c:51:0
	 #&#8203;6 0x00007fb178337921 abort /build/glibc-S7xCS9/glibc-2.27/stdlib/abort.c:81:0
	 #&#8203;7 0x00007fb17832748a __assert_fail_base /build/glibc-S7xCS9/glibc-2.27/assert/assert.c:89:0
	 #&#8203;8 0x00007fb178327502 (/lib/x86_64-linux-gnu/libc.so.6+0x30502)
	 #&#8203;9 0x000055cffcb5f09d clang::LineTableInfo::AddLineNote(clang::FileID, unsigned int, unsigned int, int, unsigned int, clang::SrcMgr::CharacteristicKind) /tmp/llvm-project/clang/lib/Basic/SourceManager.cpp:209:0
	llvm/llvm-project#382 0x000055cffcb5f46d clang::SourceManager::AddLineNote(clang::SourceLocation, unsigned int, int, bool, bool, clang::SrcMgr::CharacteristicKind) /tmp/llvm-project/clang/lib/Basic/SourceManager.cpp:290:0
	llvm/llvm-project#383 0x000055cffca8a137 clang::Preprocessor::HandleDigitDirective(clang::Token&) /tmp/llvm-project/clang/lib/Lex/PPDirectives.cpp:1363:0
	llvm/llvm-project#384 0x000055cffca88b1a clang::Preprocessor::HandleDirective(clang::Token&) /tmp/llvm-project/clang/lib/Lex/PPDirectives.cpp:977:0
	llvm/llvm-project#385 0x000055cffca4c554 clang::Lexer::LexTokenInternal(clang::Token&, bool) /tmp/llvm-project/clang/lib/Lex/Lexer.cpp:3975:0
	llvm/llvm-project#386 0x000055cffca498d8 clang::Lexer::Lex(clang::Token&) /tmp/llvm-project/clang/lib/Lex/Lexer.cpp:3188:0
	llvm/llvm-project#387 0x000055cffcacc1fa clang::Preprocessor::Lex(clang::Token&) /tmp/llvm-project/clang/lib/Lex/Preprocessor.cpp:898:0
	llvm/llvm-project#388 0x000055cffaded894 clang::Parser::ConsumeToken() /tmp/llvm-project/clang/include/clang/Parse/Parser.h:483:0
	llvm/llvm-project#389 0x000055cffade2f55 clang::Parser::Initialize() /tmp/llvm-project/clang/lib/Parse/Parser.cpp:562:0
	llvm/llvm-project#390 0x000055cffaddedb6 clang::ParseAST(clang::Sema&, bool, bool) /tmp/llvm-project/clang/lib/Parse/ParseAST.cpp:156:0
	llvm/llvm-project#391 0x000055cffab19495 clang::ASTFrontendAction::ExecuteAction() /tmp/llvm-project/clang/lib/Frontend/FrontendAction.cpp:1056:0
	llvm/llvm-project#392 0x000055cffab18d5e clang::FrontendAction::Execute() /tmp/llvm-project/clang/lib/Frontend/FrontendAction.cpp:953:0
	llvm/llvm-project#393 0x000055cffaa7ab38 clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) /tmp/llvm-project/clang/lib/Frontend/CompilerInstance.cpp:991:0
	llvm/llvm-project#394 0x000055cff9ed747d clang::tooling::FrontendActionFactory::runInvocation(std::shared_ptr<clang::CompilerInvocation>, clang::FileManager*, std::shared_ptr<clang::PCHContainerOperations>, clang::DiagnosticConsumer*) /tmp/llvm-project/clang/lib/Tooling/Tooling.cpp:396:0
	llvm/llvm-project#395 0x000055cff9e5f9a3 runInvocation /tmp/llvm-project/clang-tools-extra/clang-tidy/ClangTidy.cpp:556:0
	llvm/llvm-project#396 0x000055cff9e5f9a3 clang::tidy::runClangTidy(clang::tidy::ClangTidyContext&, clang::tooling::CompilationDatabase const&, llvm::ArrayRef<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, llvm::IntrusiveRefCntPtr<llvm::vfs::OverlayFileSystem>, bool, llvm::StringRef)::ActionFactory::runInvocation(std::shared_ptr<clang::CompilerInvocation>, clang::FileManager*, std::shared_ptr<clang::PCHContainerOperations>, clang::DiagnosticConsumer*) (./clang-tidy+0x11e19a3)
	llvm/llvm-project#397 0x000055cff9ed72ad clang::tooling::ToolInvocation::runInvocation(char const*, clang::driver::Compilation*, std::shared_ptr<clang::CompilerInvocation>, std::shared_ptr<clang::PCHContainerOperations>) /tmp/llvm-project/clang/lib/Tooling/Tooling.cpp:372:0
	llvm/llvm-project#398 0x000055cff9ed70c4 clang::tooling::ToolInvocation::run() /tmp/llvm-project/clang/lib/Tooling/Tooling.cpp:357:0
	llvm/llvm-project#399 0x000055cff9ed884d clang::tooling::ClangTool::run(clang::tooling::ToolAction*) /tmp/llvm-project/clang/lib/Tooling/Tooling.cpp:549:0
	llvm/llvm-project#400 0x000055cff9e5fda1 clang::tidy::runClangTidy(clang::tidy::ClangTidyContext&, clang::tooling::CompilationDatabase const&, llvm::ArrayRef<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, llvm::IntrusiveRefCntPtr<llvm::vfs::OverlayFileSystem>, bool, llvm::StringRef) /tmp/llvm-project/clang-tools-extra/clang-tidy/ClangTidy.cpp:577:0
	llvm/llvm-project#401 0x000055cff927ce27 clang::tidy::clangTidyMain(int, char const**) /tmp/llvm-project/clang-tools-extra/clang-tidy/tool/ClangTidyMain.cpp:486:0
	llvm/llvm-project#402 0x000055cff927b1ea main /tmp/llvm-project/clang-tools-extra/clang-tidy/tool/ClangTidyToolMain.cpp:21:0
	llvm/llvm-project#403 0x00007fb178318bf7 __libc_start_main /build/glibc-S7xCS9/glibc-2.27/csu/../csu/libc-start.c:344:0
	llvm/llvm-project#404 0x000055cff927b0ea _start (./clang-tidy+0x5fd0ea)
	Aborted

This only occurs with -std=c++20. It does not occur with -std=c++17.

If I disable the use of ExpandModularHeadersPPCallbacks here (

llvm-project/clang-tools-extra/clang-tidy/ClangTidy.cpp

Lines 414 to 419 in 3fa2d37

if (Context.getLangOpts().Modules && OverlayFS != nullptr) {
auto ModuleExpander = std::make_unique<ExpandModularHeadersPPCallbacks>(
&Compiler, OverlayFS);
ModuleExpanderPP = ModuleExpander->getPreprocessor();
PP->addPPCallbacks(std::move(ModuleExpander));
}
) then clang-tidy succeeds even in C++20 mode.
@llvmbot llvmbot transferred this issue from llvm/llvm-bugzilla-archive Dec 11, 2021
@avudnez
Copy link

avudnez commented Jan 30, 2024 •
edited
Loading

I am hitting this too (17.0.1), after #62405.

This is making analyzing Unreal Engine (for example) source code even more difficult, and I will need to resort to ugly hacks to work around this.
Maybe @PiotrZSL can provide some information?

@Endilll Endilll added c++20 confirmed Verified by a second party labels Jan 31, 2024
@llvmbot
Copy link
Member

llvmbot commented Jan 31, 2024

@llvm/issue-subscribers-c-20

Author: None (41fa9ab2-4e9a-46d3-8102-9366090d3552)

| | | | --- | --- | | Bugzilla Link | [48566](https://llvm.org/bz48566) | | Version | unspecified | | OS | All |

Extended Description

Using the latest commit as of writing (3fa2d37), clang-tidy fails to process the following preprocessed source:

	# 1 "main.cpp"
	# 1 "usr/include/sys/cdefs.h" 1
	# 1 "usr/include/android/api-level.h" 1
	# 47 "usr/include/android/api-level.h"
	extern "C" {
	# 170 "usr/include/android/api-level.h"
	#if 1
	# 1 "usr/include/bits/get_device_api_level_inlines.h" 1
	# 30 "usr/include/bits/get_device_api_level_inlines.h"
	# 32 "usr/include/bits/get_device_api_level_inlines.h"
	# 33 "usr/include/bits/get_device_api_level_inlines.h"
	# 34 "usr/include/bits/get_device_api_level_inlines.h"
	# 51 "usr/include/bits/get_device_api_level_inlines.h"
	#endif
	# 177 "usr/include/android/api-level.h" 2
	# 191 "usr/include/android/api-level.h"
	}
	# 363 "usr/include/sys/cdefs.h" 2
	# 1 "main.cpp" 2

The following command line was used:

	clang-tidy main.cpp -- -std=c++20

In Release, clang-tidy outputs this error:

	1 error generated.
	Error while processing main.cpp.
	main.cpp:19:16: error: invalid line marker flag '2': cannot pop empty include stack [clang-diagnostic-error]
	# 1 "main.cpp" 2
	               ^
	Found compiler error(s).

In Debug, clang-tidy crashes with this assert:

	clang-tidy: /tmp/llvm-project/clang/lib/Basic/SourceManager.cpp:210: void clang::LineTableInfo::AddLineNote(clang::FileID, unsigned int, unsigned int, int, unsigned int, clang::SrcMgr::CharacteristicKind): Assertion `(Entries.empty() || Entries.back().FileOffset &lt; Offset) &amp;&amp; "Adding line entries out of order!"' failed.
	PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace.
	Stack dump:
	0.      Program arguments: ./clang-tidy /tmp/main.cpp -- -std=c++20
	1.      main.cpp:0:3: current parser token '1'
	 #&amp;#<!-- -->8203;0 0x000055cffd08ed58 llvm::sys::PrintStackTrace(llvm::raw_ostream&amp;, int) /tmp/llvm-project/llvm/lib/Support/Unix/Signals.inc:563:0
	 #&amp;#<!-- -->8203;1 0x000055cffd08ee0f PrintStackTraceSignalHandler(void*) /tmp/llvm-project/llvm/lib/Support/Unix/Signals.inc:630:0
	 #&amp;#<!-- -->8203;2 0x000055cffd08caf3 llvm::sys::RunSignalHandlers() /tmp/llvm-project/llvm/lib/Support/Signals.cpp:71:0
	 #&amp;#<!-- -->8203;3 0x000055cffd08e6da SignalHandler(int) /tmp/llvm-project/llvm/lib/Support/Unix/Signals.inc:405:0
	 #&amp;#<!-- -->8203;4 0x00007fb179480980 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x12980)
	 #&amp;#<!-- -->8203;5 0x00007fb178335fb7 gsignal /build/glibc-S7xCS9/glibc-2.27/signal/../sysdeps/unix/sysv/linux/raise.c:51:0
	 #&amp;#<!-- -->8203;6 0x00007fb178337921 abort /build/glibc-S7xCS9/glibc-2.27/stdlib/abort.c:81:0
	 #&amp;#<!-- -->8203;7 0x00007fb17832748a __assert_fail_base /build/glibc-S7xCS9/glibc-2.27/assert/assert.c:89:0
	 #&amp;#<!-- -->8203;8 0x00007fb178327502 (/lib/x86_64-linux-gnu/libc.so.6+0x30502)
	 #&amp;#<!-- -->8203;9 0x000055cffcb5f09d clang::LineTableInfo::AddLineNote(clang::FileID, unsigned int, unsigned int, int, unsigned int, clang::SrcMgr::CharacteristicKind) /tmp/llvm-project/clang/lib/Basic/SourceManager.cpp:209:0
	llvm/llvm-project#<!-- -->382 0x000055cffcb5f46d clang::SourceManager::AddLineNote(clang::SourceLocation, unsigned int, int, bool, bool, clang::SrcMgr::CharacteristicKind) /tmp/llvm-project/clang/lib/Basic/SourceManager.cpp:290:0
	llvm/llvm-project#<!-- -->383 0x000055cffca8a137 clang::Preprocessor::HandleDigitDirective(clang::Token&amp;) /tmp/llvm-project/clang/lib/Lex/PPDirectives.cpp:1363:0
	llvm/llvm-project#<!-- -->384 0x000055cffca88b1a clang::Preprocessor::HandleDirective(clang::Token&amp;) /tmp/llvm-project/clang/lib/Lex/PPDirectives.cpp:977:0
	llvm/llvm-project#<!-- -->385 0x000055cffca4c554 clang::Lexer::LexTokenInternal(clang::Token&amp;, bool) /tmp/llvm-project/clang/lib/Lex/Lexer.cpp:3975:0
	llvm/llvm-project#<!-- -->386 0x000055cffca498d8 clang::Lexer::Lex(clang::Token&amp;) /tmp/llvm-project/clang/lib/Lex/Lexer.cpp:3188:0
	llvm/llvm-project#<!-- -->387 0x000055cffcacc1fa clang::Preprocessor::Lex(clang::Token&amp;) /tmp/llvm-project/clang/lib/Lex/Preprocessor.cpp:898:0
	llvm/llvm-project#<!-- -->388 0x000055cffaded894 clang::Parser::ConsumeToken() /tmp/llvm-project/clang/include/clang/Parse/Parser.h:483:0
	llvm/llvm-project#<!-- -->389 0x000055cffade2f55 clang::Parser::Initialize() /tmp/llvm-project/clang/lib/Parse/Parser.cpp:562:0
	llvm/llvm-project#<!-- -->390 0x000055cffaddedb6 clang::ParseAST(clang::Sema&amp;, bool, bool) /tmp/llvm-project/clang/lib/Parse/ParseAST.cpp:156:0
	llvm/llvm-project#<!-- -->391 0x000055cffab19495 clang::ASTFrontendAction::ExecuteAction() /tmp/llvm-project/clang/lib/Frontend/FrontendAction.cpp:1056:0
	llvm/llvm-project#<!-- -->392 0x000055cffab18d5e clang::FrontendAction::Execute() /tmp/llvm-project/clang/lib/Frontend/FrontendAction.cpp:953:0
	llvm/llvm-project#<!-- -->393 0x000055cffaa7ab38 clang::CompilerInstance::ExecuteAction(clang::FrontendAction&amp;) /tmp/llvm-project/clang/lib/Frontend/CompilerInstance.cpp:991:0
	llvm/llvm-project#<!-- -->394 0x000055cff9ed747d clang::tooling::FrontendActionFactory::runInvocation(std::shared_ptr&lt;clang::CompilerInvocation&gt;, clang::FileManager*, std::shared_ptr&lt;clang::PCHContainerOperations&gt;, clang::DiagnosticConsumer*) /tmp/llvm-project/clang/lib/Tooling/Tooling.cpp:396:0
	llvm/llvm-project#<!-- -->395 0x000055cff9e5f9a3 runInvocation /tmp/llvm-project/clang-tools-extra/clang-tidy/ClangTidy.cpp:556:0
	llvm/llvm-project#<!-- -->396 0x000055cff9e5f9a3 clang::tidy::runClangTidy(clang::tidy::ClangTidyContext&amp;, clang::tooling::CompilationDatabase const&amp;, llvm::ArrayRef&lt;std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt; &gt;, llvm::IntrusiveRefCntPtr&lt;llvm::vfs::OverlayFileSystem&gt;, bool, llvm::StringRef)::ActionFactory::runInvocation(std::shared_ptr&lt;clang::CompilerInvocation&gt;, clang::FileManager*, std::shared_ptr&lt;clang::PCHContainerOperations&gt;, clang::DiagnosticConsumer*) (./clang-tidy+0x11e19a3)
	llvm/llvm-project#<!-- -->397 0x000055cff9ed72ad clang::tooling::ToolInvocation::runInvocation(char const*, clang::driver::Compilation*, std::shared_ptr&lt;clang::CompilerInvocation&gt;, std::shared_ptr&lt;clang::PCHContainerOperations&gt;) /tmp/llvm-project/clang/lib/Tooling/Tooling.cpp:372:0
	llvm/llvm-project#<!-- -->398 0x000055cff9ed70c4 clang::tooling::ToolInvocation::run() /tmp/llvm-project/clang/lib/Tooling/Tooling.cpp:357:0
	llvm/llvm-project#<!-- -->399 0x000055cff9ed884d clang::tooling::ClangTool::run(clang::tooling::ToolAction*) /tmp/llvm-project/clang/lib/Tooling/Tooling.cpp:549:0
	llvm/llvm-project#<!-- -->400 0x000055cff9e5fda1 clang::tidy::runClangTidy(clang::tidy::ClangTidyContext&amp;, clang::tooling::CompilationDatabase const&amp;, llvm::ArrayRef&lt;std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt; &gt;, llvm::IntrusiveRefCntPtr&lt;llvm::vfs::OverlayFileSystem&gt;, bool, llvm::StringRef) /tmp/llvm-project/clang-tools-extra/clang-tidy/ClangTidy.cpp:577:0
	llvm/llvm-project#<!-- -->401 0x000055cff927ce27 clang::tidy::clangTidyMain(int, char const**) /tmp/llvm-project/clang-tools-extra/clang-tidy/tool/ClangTidyMain.cpp:486:0
	llvm/llvm-project#<!-- -->402 0x000055cff927b1ea main /tmp/llvm-project/clang-tools-extra/clang-tidy/tool/ClangTidyToolMain.cpp:21:0
	llvm/llvm-project#<!-- -->403 0x00007fb178318bf7 __libc_start_main /build/glibc-S7xCS9/glibc-2.27/csu/../csu/libc-start.c:344:0
	llvm/llvm-project#<!-- -->404 0x000055cff927b0ea _start (./clang-tidy+0x5fd0ea)
	Aborted

This only occurs with -std=c++20. It does not occur with -std=c++17.

If I disable the use of ExpandModularHeadersPPCallbacks here (

llvm-project/clang-tools-extra/clang-tidy/ClangTidy.cpp

Lines 414 to 419 in 3fa2d37

if (Context.getLangOpts().Modules && OverlayFS != nullptr) {
auto ModuleExpander = std::make_unique<ExpandModularHeadersPPCallbacks>(
&Compiler, OverlayFS);
ModuleExpanderPP = ModuleExpander->getPreprocessor();
PP->addPPCallbacks(std::move(ModuleExpander));
}
) then clang-tidy succeeds even in C++20 mode.

@avudnez
Copy link

avudnez commented Mar 20, 2025

It was already fixed since b530eee by @PiotrZSL. Thanks!

@EugeneZelenko EugeneZelenko added the crash Prefer [crash-on-valid] or [crash-on-invalid] label Mar 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bugzilla Issues migrated from bugzilla c++20 clang-tidy confirmed Verified by a second party crash Prefer [crash-on-valid] or [crash-on-invalid]
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@EugeneZelenko @Endilll @llvmbot @avudnez