dead loop in clang::dataflow::WatchedLiteralsSolverImpl::solve(long long) when running bugprone-unchecked-optional-access tidy-check #69369

andy-yx-chen · 2023-10-17T18:50:10Z

Reproduce step:

build https://github.com/ccache/ccache with compile_commands.json
with 17.02 clangd check src/ccache.cpp

Stack trace

 thread #1, queue = 'com.apple.main-thread', stop reason = instruction step over
  * frame #0: 0x00000001063b1a44 clangd`clang::dataflow::WatchedLiteralsSolverImpl::solve(long long) && + 228
    frame #1: 0x00000001063b17a4 clangd`clang::dataflow::WatchedLiteralsSolver::solve(llvm::ArrayRef<clang::dataflow::Formula const*>) + 64
    frame #2: 0x000000010639bc84 clangd`clang::dataflow::DataflowAnalysisContext::isUnsatisfiable(llvm::SetVector<clang::dataflow::Formula const*, std::__1::vector<clang::dataflow::Formula const*, std::__1::allocator<clang::dataflow::Formula const*>>, llvm::DenseSet<clang::dataflow::Formula const*, llvm::DenseMapInfo<clang::dataflow::Formula const*, void>>, 0u>) + 180
    frame #3: 0x000000010639b8d8 clangd`clang::dataflow::DataflowAnalysisContext::flowConditionImplies(clang::dataflow::Atom, clang::dataflow::Formula const&) + 196
    frame #4: 0x000000010638853c clangd`clang::dataflow::UncheckedOptionalAccessModel::compare(clang::QualType, clang::dataflow::Value const&, clang::dataflow::Environment const&, clang::dataflow::Value const&, clang::dataflow::Environment const&) + 280
    frame #5: 0x0000000106388978 clangd`clang::dataflow::UncheckedOptionalAccessModel::widen(clang::QualType, clang::dataflow::Value&, clang::dataflow::Environment const&, clang::dataflow::Value&, clang::dataflow::Environment&) + 44
    frame #6: 0x00000001063a1588 clangd`clang::dataflow::Environment::widen(clang::dataflow::Environment const&, clang::dataflow::Environment::ValueModel&) + 404
    frame #7: 0x00000001063aee8c clangd`clang::dataflow::runTypeErasedDataflowAnalysis(clang::dataflow::ControlFlowContext const&, clang::dataflow::TypeErasedDataflowAnalysis&, clang::dataflow::Environment const&, std::__1::function<void (clang::CFGElement const&, clang::dataflow::TypeErasedDataflowAnalysisState const&)>) + 1340
    frame #8: 0x000000010635f5b4 clangd`llvm::Expected<std::__1::vector<std::__1::optional<clang::dataflow::DataflowAnalysisState<clang::dataflow::UncheckedOptionalAccessModel::Lattice>>, std::__1::allocator<std::__1::optional<clang::dataflow::DataflowAnalysisState<clang::dataflow::UncheckedOptionalAccessModel::Lattice>>>>> clang::dataflow::runDataflowAnalysis<clang::dataflow::UncheckedOptionalAccessModel>(clang::dataflow::ControlFlowContext const&, clang::dataflow::UncheckedOptionalAccessModel&, clang::dataflow::Environment const&, std::__1::function<void (clang::CFGElement const&, clang::dataflow::DataflowAnalysisState<clang::dataflow::UncheckedOptionalAccessModel::Lattice> const&)>) + 216
    frame #9: 0x000000010635f13c clangd`clang::tidy::bugprone::UncheckedOptionalAccessCheck::check(clang::ast_matchers::MatchFinder::MatchResult const&) + 464
    frame #10: 0x0000000106c42058 clangd`clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor::MatchVisitor::visitMatch(clang::ast_matchers::BoundNodes const&) + 132
    frame #11: 0x0000000106c6d1a0 clangd`clang::ast_matchers::internal::BoundNodesTreeBuilder::visitMatches(clang::ast_matchers::internal::BoundNodesTreeBuilder::Visitor*) + 256
    frame #12: 0x0000000106c419ac clangd`clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor::matchWithFilter(clang::DynTypedNode const&) + 1104
    frame #13: 0x0000000106c433fc clangd`clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor::TraverseDecl(clang::Decl*) + 276
    frame #14: 0x0000000106c46728 clangd`clang::RecursiveASTVisitor<clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor>::TraverseDecl(clang::Decl*) + 13004
    frame #15: 0x0000000106c43408 clangd`clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor::TraverseDecl(clang::Decl*) + 288
    frame #16: 0x0000000106c226bc clangd`clang::ast_matchers::MatchFinder::matchAST(clang::ASTContext&) + 748
    frame #17: 0x0000000105fca154 clangd`clang::clangd::ParsedAST::build(llvm::StringRef, clang::clangd::ParseInputs const&, std::__1::unique_ptr<clang::CompilerInvocation, std::__1::default_delete<clang::CompilerInvocation>>, llvm::ArrayRef<clang::clangd::Diag>, std::__1::shared_ptr<clang::clangd::PreambleData const>) + 7508
    frame #18: 0x0000000105e63994 clangd`clang::clangd::check(llvm::StringRef, clang::clangd::ThreadsafeFS const&, clang::clangd::ClangdLSPServer::Options const&) + 7376
    frame #19: 0x0000000105e53294 clangd`clang::clangd::clangdMain(int, char**) + 8664
    frame #20: 0x000000018659bf28 dyld`start + 2236

The text was updated successfully, but these errors were encountered:

llvmbot · 2023-10-17T18:53:04Z

@llvm/issue-subscribers-clangd

Author: Andy Chen (andy-yx-chen)

Reproduce step: 1. build https://github.com/ccache/ccache with compile_commands.json 2. with 17.02 clangd check src/ccache.cpp

Stack trace

 thread #<!-- -->1, queue = 'com.apple.main-thread', stop reason = instruction step over
  * frame #<!-- -->0: 0x00000001063b1a44 clangd`clang::dataflow::WatchedLiteralsSolverImpl::solve(long long) &amp;&amp; + 228
    frame #<!-- -->1: 0x00000001063b17a4 clangd`clang::dataflow::WatchedLiteralsSolver::solve(llvm::ArrayRef&lt;clang::dataflow::Formula const*&gt;) + 64
    frame #<!-- -->2: 0x000000010639bc84 clangd`clang::dataflow::DataflowAnalysisContext::isUnsatisfiable(llvm::SetVector&lt;clang::dataflow::Formula const*, std::__1::vector&lt;clang::dataflow::Formula const*, std::__1::allocator&lt;clang::dataflow::Formula const*&gt;&gt;, llvm::DenseSet&lt;clang::dataflow::Formula const*, llvm::DenseMapInfo&lt;clang::dataflow::Formula const*, void&gt;&gt;, 0u&gt;) + 180
    frame #<!-- -->3: 0x000000010639b8d8 clangd`clang::dataflow::DataflowAnalysisContext::flowConditionImplies(clang::dataflow::Atom, clang::dataflow::Formula const&amp;) + 196
    frame #<!-- -->4: 0x000000010638853c clangd`clang::dataflow::UncheckedOptionalAccessModel::compare(clang::QualType, clang::dataflow::Value const&amp;, clang::dataflow::Environment const&amp;, clang::dataflow::Value const&amp;, clang::dataflow::Environment const&amp;) + 280
    frame #<!-- -->5: 0x0000000106388978 clangd`clang::dataflow::UncheckedOptionalAccessModel::widen(clang::QualType, clang::dataflow::Value&amp;, clang::dataflow::Environment const&amp;, clang::dataflow::Value&amp;, clang::dataflow::Environment&amp;) + 44
    frame #<!-- -->6: 0x00000001063a1588 clangd`clang::dataflow::Environment::widen(clang::dataflow::Environment const&amp;, clang::dataflow::Environment::ValueModel&amp;) + 404
    frame #<!-- -->7: 0x00000001063aee8c clangd`clang::dataflow::runTypeErasedDataflowAnalysis(clang::dataflow::ControlFlowContext const&amp;, clang::dataflow::TypeErasedDataflowAnalysis&amp;, clang::dataflow::Environment const&amp;, std::__1::function&lt;void (clang::CFGElement const&amp;, clang::dataflow::TypeErasedDataflowAnalysisState const&amp;)&gt;) + 1340
    frame #<!-- -->8: 0x000000010635f5b4 clangd`llvm::Expected&lt;std::__1::vector&lt;std::__1::optional&lt;clang::dataflow::DataflowAnalysisState&lt;clang::dataflow::UncheckedOptionalAccessModel::Lattice&gt;&gt;, std::__1::allocator&lt;std::__1::optional&lt;clang::dataflow::DataflowAnalysisState&lt;clang::dataflow::UncheckedOptionalAccessModel::Lattice&gt;&gt;&gt;&gt;&gt; clang::dataflow::runDataflowAnalysis&lt;clang::dataflow::UncheckedOptionalAccessModel&gt;(clang::dataflow::ControlFlowContext const&amp;, clang::dataflow::UncheckedOptionalAccessModel&amp;, clang::dataflow::Environment const&amp;, std::__1::function&lt;void (clang::CFGElement const&amp;, clang::dataflow::DataflowAnalysisState&lt;clang::dataflow::UncheckedOptionalAccessModel::Lattice&gt; const&amp;)&gt;) + 216
    frame #<!-- -->9: 0x000000010635f13c clangd`clang::tidy::bugprone::UncheckedOptionalAccessCheck::check(clang::ast_matchers::MatchFinder::MatchResult const&amp;) + 464
    frame #<!-- -->10: 0x0000000106c42058 clangd`clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor::MatchVisitor::visitMatch(clang::ast_matchers::BoundNodes const&amp;) + 132
    frame #<!-- -->11: 0x0000000106c6d1a0 clangd`clang::ast_matchers::internal::BoundNodesTreeBuilder::visitMatches(clang::ast_matchers::internal::BoundNodesTreeBuilder::Visitor*) + 256
    frame #<!-- -->12: 0x0000000106c419ac clangd`clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor::matchWithFilter(clang::DynTypedNode const&amp;) + 1104
    frame #<!-- -->13: 0x0000000106c433fc clangd`clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor::TraverseDecl(clang::Decl*) + 276
    frame #<!-- -->14: 0x0000000106c46728 clangd`clang::RecursiveASTVisitor&lt;clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor&gt;::TraverseDecl(clang::Decl*) + 13004
    frame #<!-- -->15: 0x0000000106c43408 clangd`clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor::TraverseDecl(clang::Decl*) + 288
    frame #<!-- -->16: 0x0000000106c226bc clangd`clang::ast_matchers::MatchFinder::matchAST(clang::ASTContext&amp;) + 748
    frame #<!-- -->17: 0x0000000105fca154 clangd`clang::clangd::ParsedAST::build(llvm::StringRef, clang::clangd::ParseInputs const&amp;, std::__1::unique_ptr&lt;clang::CompilerInvocation, std::__1::default_delete&lt;clang::CompilerInvocation&gt;&gt;, llvm::ArrayRef&lt;clang::clangd::Diag&gt;, std::__1::shared_ptr&lt;clang::clangd::PreambleData const&gt;) + 7508
    frame #<!-- -->18: 0x0000000105e63994 clangd`clang::clangd::check(llvm::StringRef, clang::clangd::ThreadsafeFS const&amp;, clang::clangd::ClangdLSPServer::Options const&amp;) + 7376
    frame #<!-- -->19: 0x0000000105e53294 clangd`clang::clangd::clangdMain(int, char**) + 8664
    frame #<!-- -->20: 0x000000018659bf28 dyld`start + 2236

llvmbot · 2023-10-18T06:05:53Z

@llvm/issue-subscribers-clang-tidy

Author: Andy Chen (andy-yx-chen)

Reproduce step: 1. build https://github.com/ccache/ccache with compile_commands.json 2. with 17.02 clangd check src/ccache.cpp

Stack trace

 thread #<!-- -->1, queue = 'com.apple.main-thread', stop reason = instruction step over
  * frame #<!-- -->0: 0x00000001063b1a44 clangd`clang::dataflow::WatchedLiteralsSolverImpl::solve(long long) &amp;&amp; + 228
    frame #<!-- -->1: 0x00000001063b17a4 clangd`clang::dataflow::WatchedLiteralsSolver::solve(llvm::ArrayRef&lt;clang::dataflow::Formula const*&gt;) + 64
    frame #<!-- -->2: 0x000000010639bc84 clangd`clang::dataflow::DataflowAnalysisContext::isUnsatisfiable(llvm::SetVector&lt;clang::dataflow::Formula const*, std::__1::vector&lt;clang::dataflow::Formula const*, std::__1::allocator&lt;clang::dataflow::Formula const*&gt;&gt;, llvm::DenseSet&lt;clang::dataflow::Formula const*, llvm::DenseMapInfo&lt;clang::dataflow::Formula const*, void&gt;&gt;, 0u&gt;) + 180
    frame #<!-- -->3: 0x000000010639b8d8 clangd`clang::dataflow::DataflowAnalysisContext::flowConditionImplies(clang::dataflow::Atom, clang::dataflow::Formula const&amp;) + 196
    frame #<!-- -->4: 0x000000010638853c clangd`clang::dataflow::UncheckedOptionalAccessModel::compare(clang::QualType, clang::dataflow::Value const&amp;, clang::dataflow::Environment const&amp;, clang::dataflow::Value const&amp;, clang::dataflow::Environment const&amp;) + 280
    frame #<!-- -->5: 0x0000000106388978 clangd`clang::dataflow::UncheckedOptionalAccessModel::widen(clang::QualType, clang::dataflow::Value&amp;, clang::dataflow::Environment const&amp;, clang::dataflow::Value&amp;, clang::dataflow::Environment&amp;) + 44
    frame #<!-- -->6: 0x00000001063a1588 clangd`clang::dataflow::Environment::widen(clang::dataflow::Environment const&amp;, clang::dataflow::Environment::ValueModel&amp;) + 404
    frame #<!-- -->7: 0x00000001063aee8c clangd`clang::dataflow::runTypeErasedDataflowAnalysis(clang::dataflow::ControlFlowContext const&amp;, clang::dataflow::TypeErasedDataflowAnalysis&amp;, clang::dataflow::Environment const&amp;, std::__1::function&lt;void (clang::CFGElement const&amp;, clang::dataflow::TypeErasedDataflowAnalysisState const&amp;)&gt;) + 1340
    frame #<!-- -->8: 0x000000010635f5b4 clangd`llvm::Expected&lt;std::__1::vector&lt;std::__1::optional&lt;clang::dataflow::DataflowAnalysisState&lt;clang::dataflow::UncheckedOptionalAccessModel::Lattice&gt;&gt;, std::__1::allocator&lt;std::__1::optional&lt;clang::dataflow::DataflowAnalysisState&lt;clang::dataflow::UncheckedOptionalAccessModel::Lattice&gt;&gt;&gt;&gt;&gt; clang::dataflow::runDataflowAnalysis&lt;clang::dataflow::UncheckedOptionalAccessModel&gt;(clang::dataflow::ControlFlowContext const&amp;, clang::dataflow::UncheckedOptionalAccessModel&amp;, clang::dataflow::Environment const&amp;, std::__1::function&lt;void (clang::CFGElement const&amp;, clang::dataflow::DataflowAnalysisState&lt;clang::dataflow::UncheckedOptionalAccessModel::Lattice&gt; const&amp;)&gt;) + 216
    frame #<!-- -->9: 0x000000010635f13c clangd`clang::tidy::bugprone::UncheckedOptionalAccessCheck::check(clang::ast_matchers::MatchFinder::MatchResult const&amp;) + 464
    frame #<!-- -->10: 0x0000000106c42058 clangd`clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor::MatchVisitor::visitMatch(clang::ast_matchers::BoundNodes const&amp;) + 132
    frame #<!-- -->11: 0x0000000106c6d1a0 clangd`clang::ast_matchers::internal::BoundNodesTreeBuilder::visitMatches(clang::ast_matchers::internal::BoundNodesTreeBuilder::Visitor*) + 256
    frame #<!-- -->12: 0x0000000106c419ac clangd`clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor::matchWithFilter(clang::DynTypedNode const&amp;) + 1104
    frame #<!-- -->13: 0x0000000106c433fc clangd`clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor::TraverseDecl(clang::Decl*) + 276
    frame #<!-- -->14: 0x0000000106c46728 clangd`clang::RecursiveASTVisitor&lt;clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor&gt;::TraverseDecl(clang::Decl*) + 13004
    frame #<!-- -->15: 0x0000000106c43408 clangd`clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor::TraverseDecl(clang::Decl*) + 288
    frame #<!-- -->16: 0x0000000106c226bc clangd`clang::ast_matchers::MatchFinder::matchAST(clang::ASTContext&amp;) + 748
    frame #<!-- -->17: 0x0000000105fca154 clangd`clang::clangd::ParsedAST::build(llvm::StringRef, clang::clangd::ParseInputs const&amp;, std::__1::unique_ptr&lt;clang::CompilerInvocation, std::__1::default_delete&lt;clang::CompilerInvocation&gt;&gt;, llvm::ArrayRef&lt;clang::clangd::Diag&gt;, std::__1::shared_ptr&lt;clang::clangd::PreambleData const&gt;) + 7508
    frame #<!-- -->18: 0x0000000105e63994 clangd`clang::clangd::check(llvm::StringRef, clang::clangd::ThreadsafeFS const&amp;, clang::clangd::ClangdLSPServer::Options const&amp;) + 7376
    frame #<!-- -->19: 0x0000000105e53294 clangd`clang::clangd::clangdMain(int, char**) + 8664
    frame #<!-- -->20: 0x000000018659bf28 dyld`start + 2236

HighCommander4 · 2023-10-18T06:07:16Z

Added clang-tidy label because the hang is inside the bugprone-unchecked-optional-access clang-tidy check.

There are existing issues on file for hangs in that checker (#62085, #55530).

As a clangd user, a workaround is to disable the checker for clangd by adding:

Diagnostics:
  ClangTidy:
    Remove: bugprone-unchecked-optional-access

to .clangd.

Fixes llvm#69369.

andy-yx-chen · 2023-10-18T17:29:43Z

Added clang-tidy label because the hang is inside the bugprone-unchecked-optional-access clang-tidy check.

There are existing issues on file for hangs in that checker (#62085, #55530).

As a clangd user, a workaround is to disable the checker for clangd by adding:
Diagnostics:
  ClangTidy:
    Remove: bugprone-unchecked-optional-access
to .clangd.

this works for me, thanks!

Fixes llvm/llvm-project#69369. Fixes clangd/clangd#1700. (cherry picked from commit e63ab13c82e78f65baca48d5b5e4f6ea8d55dbc7)

EugeneZelenko · 2023-10-19T14:59:42Z

Root cause was not fixed yet.

kadircet · 2023-10-20T10:32:16Z

@EugeneZelenko sorry for the noise here, i marked this one as fixed because symptoms in clangd are "fixed" (what a broad meaning that word has..) and because there are other issues mentioned in #69369 (comment) for tracking the problems in the tidy-check.

I've renamed the issue at least to not mention clangd, feel free to keep/close it.

EugeneZelenko · 2023-10-25T14:13:35Z

Still no fix for root cause.

jcsxky · 2023-10-26T06:06:45Z

Still no fix for root cause.

Root case is the constraint on SAT solver(I think). Currently, satisfiability algorithm used in FlowSensitiveAnalysis is based on DPLL framework and can't solve large formula which contains a few thousand variable.

HighCommander4 · 2023-10-26T22:50:04Z

Root case is the constraint on SAT solver(I think). Currently, satisfiability algorithm used in FlowSensitiveAnalysis is based on DPLL framework and can't solve large formula which contains a few thousand variable.

Can we make it so that hitting the limits of the solver results in a false negative (no diagnostic issued) rather than a hang?

jcsxky · 2023-10-27T02:15:46Z

Root case is the constraint on SAT solver(I think). Currently, satisfiability algorithm used in FlowSensitiveAnalysis is based on DPLL framework and can't solve large formula which contains a few thousand variable.

Can we make it so that hitting the limits of the solver results in a false negative (no diagnostic issued) rather than a hang?

Actually, UncheckedOptionalAccessCheck use default iteration number(1'000'000'000) as the iteration limit, change it like this

if (llvm::Expected<llvm::SmallVector<SourceLocation>> Locs =
          dataflow::diagnoseFunction<UncheckedOptionalAccessModel,
                                     SourceLocation>(*FuncDecl, *Result.Context,
                                                     Diagnoser, /*# of iteration= */ 1000))
    for (const SourceLocation &Loc : *Locs)
      diag(Loc, "unchecked access to optional value");
  else
    llvm::consumeError(Locs.takeError());

would avoid hanging and return timeout(reach limitation), but this will produce false positive(not very sure) based on current implementation.

… to hang (#115124) Let's see if it helps #114913 The issues on llvm are at llvm/llvm-project#55530 and llvm/llvm-project#69369. In my CI test, I saw the following process hanged: ``` /pytorch/pytorch/.lintbin/clang-tidy -p=/pytorch/pytorch/build --extra-arg -I/usr/lib/llvm-11/include/openmp --extra-arg -I/opt/conda/envs/py_3.9/include/python3.9 --extra-arg -I/pytorch/pytorch/third_party/pybind11/include --extra-arg -I/usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11 --extra-arg -I/usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/x86_64-linux-gnu/c++/11 --extra-arg -I/usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/backward --extra-arg -I/usr/lib/llvm-14/lib/clang/14.0.0/include --extra-arg -I/usr/local/include --extra-arg -I/usr/include/x86_64-linux-gnu --extra-arg -I/usr/include /pytorch/pytorch/torch/csrc/autograd/python_nested_functions_manual.cpp ``` and the core dump matches the description found in llvm/llvm-project#69369 showing the stuck in `clang::tidy::bugprone::UncheckedOptionalAccessCheck::check`: ``` #0 0x00000000030c7420 in clang::dataflow::WatchedLiteralsSolverImpl::updateWatchedLiterals() () #1 0x00000000030c6c2a in clang::dataflow::WatchedLiteralsSolverImpl::solve() && () #2 0x00000000030c6572 in clang::dataflow::WatchedLiteralsSolver::solve(llvm::DenseSet<clang::dataflow::BoolValue*, llvm::DenseMapInfo<clang::dataflow::BoolValue*, void> >) () #3 0x00000000030b3bd3 in clang::dataflow::DataflowAnalysisContext::querySolver(llvm::DenseSet<clang::dataflow::BoolValue*, llvm::DenseMapInfo<clang::dataflow::BoolValue*, void> >) () #4 0x00000000030b3ca5 in clang::dataflow::DataflowAnalysisContext::flowConditionImplies(clang::dataflow::AtomicBoolValue&, clang::dataflow::BoolValue&) () #5 0x00000000030b1213 in clang::dataflow::(anonymous namespace)::diagnoseUnwrapCall(clang::Expr const*, clang::Expr const*, clang::dataflow::Environment const&) () #6 0x00000000030b1357 in std::_Function_handler<std::vector<clang::SourceLocation, std::allocator<clang::SourceLocation> > (clang::CallExpr const*, clang::ast_matchers::MatchFinder::MatchResult const&, clang::dataflow::Environment const&), clang::dataflow::(anonymous namespace)::buildDiagnoseMatchSwitch(clang::dataflow::UncheckedOptionalAccessModelOptions const&)::$_7>::_M_invoke(std::_Any_data const&, clang::CallExpr const*&&, clang::ast_matchers::MatchFinder::MatchResult const&, clang::dataflow::Environment const&) () #7 0x00000000030b1292 in std::_Function_handler<std::vector<clang::SourceLocation, std::allocator<clang::SourceLocation> > (clang::Stmt const*, clang::ast_matchers::MatchFinder::MatchResult const&, clang::dataflow::Environment const&), clang::dataflow::MatchSwitchBuilder<clang::dataflow::Environment const, std::vector<clang::SourceLocation, std::allocator<clang::SourceLocation> > >::CaseOf<clang::CallExpr>(clang::ast_matchers::internal::Matcher<clang::Stmt>, std::function<std::vector<clang::SourceLocation, std::allocator<clang::SourceLocation> > (clang::CallExpr const*, clang::ast_matchers::MatchFinder::MatchResult const&, clang::dataflow::Environment const&)>) &&::{lambda(clang::Stmt const*, clang::ast_matchers::MatchFinder::MatchResult const&, clang::dataflow::Environment const&)#1}>::_M_invoke(std::_Any_data const&, clang::Stmt const*&&, clang::ast_matchers::MatchFinder::MatchResult const&, clang::dataflow::Environment const&) () #8 0x00000000030b1995 in clang::dataflow::MatchSwitchBuilder<clang::dataflow::Environment const, std::vector<clang::SourceLocation, std::allocator<clang::SourceLocation> > >::Build() &&::{lambda(clang::Stmt const&, clang::ASTContext&, clang::dataflow::Environment const&)#1}::operator()(clang::Stmt const&, clang::ASTContext&, clang::dataflow::Environment const&) const () #9 0x00000000030b170c in std::_Function_handler<std::vector<clang::SourceLocation, std::allocator<clang::SourceLocation> > (clang::Stmt const&, clang::ASTContext&, clang::dataflow::Environment const&), clang::dataflow::MatchSwitchBuilder<clang::dataflow::Environment const, std::vector<clang::SourceLocation, std::allocator<clang::SourceLocation> > >::Build() &&::{lambda(clang::Stmt const&, clang::ASTContext&, clang::dataflow::Environment const&)#1}>::_M_invoke(std::_Any_data const&, clang::Stmt const&, clang::ASTContext&, clang::dataflow::Environment const&) () #10 0x00000000030a7c27 in clang::dataflow::UncheckedOptionalAccessDiagnoser::diagnose(clang::ASTContext&, clang::Stmt const*, clang::dataflow::Environment const&) () #11 0x0000000002931286 in std::_Function_handler<void (clang::Stmt const*, clang::dataflow::DataflowAnalysisState<clang::dataflow::NoopLattice> const&), clang::tidy::bugprone::analyzeFunction(clang::FunctionDecl const&, clang::ASTContext&)::$_0>::_M_invoke(std::_Any_data const&, clang::Stmt const*&&, clang::dataflow::DataflowAnalysisState<clang::dataflow::NoopLattice> const&) () #12 0x0000000002930b41 in clang::dataflow::runDataflowAnalysis<clang::dataflow::UncheckedOptionalAccessModel>(clang::dataflow::ControlFlowContext const&, clang::dataflow::UncheckedOptionalAccessModel&, clang::dataflow::Environment const&, std::function<void (clang::Stmt const*, clang::dataflow::DataflowAnalysisState<clang::dataflow::UncheckedOptionalAccessModel::Lattice> const&)>)::{lambda(clang::Stmt const*, clang::dataflow::TypeErasedDataflowAnalysisState const&)#1}::operator()(clang::Stmt const*, clang::dataflow::TypeErasedDataflowAnalysisState const&) const () #13 0x00000000030c18cc in std::_Function_handler<void (clang::CFGStmt const&, clang::dataflow::TypeErasedDataflowAnalysisState const&), clang::dataflow::runTypeErasedDataflowAnalysis(clang::dataflow::ControlFlowContext const&, clang::dataflow::TypeErasedDataflowAnalysis&, clang::dataflow::Environment const&, std::function<void (clang::Stmt const*, clang::dataflow::TypeErasedDataflowAnalysisState const&)>)::$_1>::_M_invoke(std::_Any_data const&, clang::CFGStmt const&, clang::dataflow::TypeErasedDataflowAnalysisState const&) () #14 0x00000000030bf069 in clang::dataflow::transferBlock(clang::dataflow::ControlFlowContext const&, std::vector<llvm::Optional<clang::dataflow::TypeErasedDataflowAnalysisState>, std::allocator<llvm::Optional<clang::dataflow::TypeErasedDataflowAnalysisState> > >&, clang::CFGBlock const&, clang::dataflow::Environment const&, clang::dataflow::TypeErasedDataflowAnalysis&, std::function<void (clang::CFGStmt const&, clang::dataflow::TypeErasedDataflowAnalysisState const&)>) () #15 0x00000000030bfaa5 in clang::dataflow::runTypeErasedDataflowAnalysis(clang::dataflow::ControlFlowContext const&, clang::dataflow::TypeErasedDataflowAnalysis&, clang::dataflow::Environment const&, std::function<void (clang::Stmt const*, clang::dataflow::TypeErasedDataflowAnalysisState const&)>) () #16 0x00000000029301b3 in llvm::Expected<std::vector<llvm::Optional<clang::dataflow::DataflowAnalysisState<clang::dataflow::UncheckedOptionalAccessModel::Lattice> >, std::allocator<llvm::Optional<clang::dataflow::DataflowAnalysisState<clang::dataflow::UncheckedOptionalAccessModel::Lattice> > > > > clang::dataflow::runDataflowAnalysis<clang::dataflow::UncheckedOptionalAccessModel>(clang::dataflow::ControlFlowContext const&, clang::dataflow::UncheckedOptionalAccessModel&, clang::dataflow::Environment const&, std::function<void (clang::Stmt const*, clang::dataflow::DataflowAnalysisState<clang::dataflow::UncheckedOptionalAccessModel::Lattice> const&)>) () #17 0x000000000292fbe8 in clang::tidy::bugprone::UncheckedOptionalAccessCheck::check(clang::ast_matchers::MatchFinder::MatchResult const&) () #18 0x00000000022e1572 in clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor::MatchVisitor::visitMatch(clang::ast_matchers::BoundNodes const&) () #19 0x0000000002797a1c in clang::ast_matchers::internal::BoundNodesTreeBuilder::visitMatches(clang::ast_matchers::internal::BoundNodesTreeBuilder::Visitor*) () #20 0x00000000022e0dc6 in clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor::matchWithFilter(clang::DynTypedNode const&) () #21 0x00000000022e3b57 in clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor::TraverseDecl(clang::Decl*) () #22 0x00000000022e4c0c in clang::RecursiveASTVisitor<clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor>::TraverseDecl(clang::Decl*) () #23 0x00000000022e3b62 in clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor::TraverseDecl(clang::Decl*) () #24 0x00000000022e4c0c in clang::RecursiveASTVisitor<clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor>::TraverseDecl(clang::Decl*) () #25 0x00000000022e3b62 in clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor::TraverseDecl(clang::Decl*) () #26 0x00000000022e4c0c in clang::RecursiveASTVisitor<clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor>::TraverseDecl(clang::Decl*) () #27 0x00000000022e3b62 in clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor::TraverseDecl(clang::Decl*) () #28 0x00000000022e4c0c in clang::RecursiveASTVisitor<clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor>::TraverseDecl(clang::Decl*) () #29 0x00000000022e3b62 in clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor::TraverseDecl(clang::Decl*) () #30 0x00000000022e8791 in clang::RecursiveASTVisitor<clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor>::TraverseDecl(clang::Decl*) () #31 0x00000000022e3b62 in clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor::TraverseDecl(clang::Decl*) () #32 0x00000000022c017a in clang::ast_matchers::MatchFinder::matchAST(clang::ASTContext&) () #33 0x000000000370ad3c in clang::MultiplexConsumer::HandleTranslationUnit(clang::ASTContext&) () #34 0x00000000038ed4bb in clang::ParseAST(clang::Sema&, bool, bool) () #35 0x000000000369eda7 in clang::FrontendAction::Execute() () #36 0x000000000360d3f6 in clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) () #37 0x00000000027c475c in clang::tooling::FrontendActionFactory::runInvocation(std::shared_ptr<clang::CompilerInvocation>, clang::FileManager*, std::shared_ptr<clang::PCHContainerOperations>, clang::DiagnosticConsumer*) () #38 0x00000000022ad486 in clang::tidy::runClangTidy(clang::tidy::ClangTidyContext&, clang::tooling::CompilationDatabase const&, llvm::ArrayRef<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, llvm::IntrusiveRefCntPtr<llvm::vfs::OverlayFileSystem>, bool, bool, llvm::StringRef)::ActionFactory::runInvocation(std::shared_ptr<clang::CompilerInvocation>, clang::FileManager*, std::shared_ptr<clang::PCHContainerOperations>, clang::DiagnosticConsumer*) () #39 0x00000000027c44c6 in clang::tooling::ToolInvocation::runInvocation(char const*, clang::driver::Compilation*, std::shared_ptr<clang::CompilerInvocation>, std::shared_ptr<clang::PCHContainerOperations>) () #40 0x00000000027c360b in clang::tooling::ToolInvocation::run() () #41 0x00000000027c5bb1 in clang::tooling::ClangTool::run(clang::tooling::ToolAction*) () #42 0x00000000022a90c7 in clang::tidy::runClangTidy(clang::tidy::ClangTidyContext&, clang::tooling::CompilationDatabase const&, llvm::ArrayRef<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, llvm::IntrusiveRefCntPtr<llvm::vfs::OverlayFileSystem>, bool, bool, llvm::StringRef) () #43 0x0000000001ebc7f2 in clang::tidy::clangTidyMain(int, char const**) () #44 0x0000000004c54ba0 in __libc_start_main () #45 0x0000000001eb76ae in _start () ``` Another note is that clang-tidy is CPU-bound. So we could consider running lintrunner job on 4xlarge if needed. Pull Request resolved: #115124 Approved by: https://github.com/kit1980, https://github.com/Skylion007, https://github.com/malfet

… to hang (pytorch#115124) Let's see if it helps pytorch#114913 The issues on llvm are at llvm/llvm-project#55530 and llvm/llvm-project#69369. In my CI test, I saw the following process hanged: ``` /pytorch/pytorch/.lintbin/clang-tidy -p=/pytorch/pytorch/build --extra-arg -I/usr/lib/llvm-11/include/openmp --extra-arg -I/opt/conda/envs/py_3.9/include/python3.9 --extra-arg -I/pytorch/pytorch/third_party/pybind11/include --extra-arg -I/usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11 --extra-arg -I/usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/x86_64-linux-gnu/c++/11 --extra-arg -I/usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/backward --extra-arg -I/usr/lib/llvm-14/lib/clang/14.0.0/include --extra-arg -I/usr/local/include --extra-arg -I/usr/include/x86_64-linux-gnu --extra-arg -I/usr/include /pytorch/pytorch/torch/csrc/autograd/python_nested_functions_manual.cpp ``` and the core dump matches the description found in llvm/llvm-project#69369 showing the stuck in `clang::tidy::bugprone::UncheckedOptionalAccessCheck::check`: ``` #0 0x00000000030c7420 in clang::dataflow::WatchedLiteralsSolverImpl::updateWatchedLiterals() () pytorch#1 0x00000000030c6c2a in clang::dataflow::WatchedLiteralsSolverImpl::solve() && () pytorch#2 0x00000000030c6572 in clang::dataflow::WatchedLiteralsSolver::solve(llvm::DenseSet<clang::dataflow::BoolValue*, llvm::DenseMapInfo<clang::dataflow::BoolValue*, void> >) () pytorch#3 0x00000000030b3bd3 in clang::dataflow::DataflowAnalysisContext::querySolver(llvm::DenseSet<clang::dataflow::BoolValue*, llvm::DenseMapInfo<clang::dataflow::BoolValue*, void> >) () pytorch#4 0x00000000030b3ca5 in clang::dataflow::DataflowAnalysisContext::flowConditionImplies(clang::dataflow::AtomicBoolValue&, clang::dataflow::BoolValue&) () pytorch#5 0x00000000030b1213 in clang::dataflow::(anonymous namespace)::diagnoseUnwrapCall(clang::Expr const*, clang::Expr const*, clang::dataflow::Environment const&) () pytorch#6 0x00000000030b1357 in std::_Function_handler<std::vector<clang::SourceLocation, std::allocator<clang::SourceLocation> > (clang::CallExpr const*, clang::ast_matchers::MatchFinder::MatchResult const&, clang::dataflow::Environment const&), clang::dataflow::(anonymous namespace)::buildDiagnoseMatchSwitch(clang::dataflow::UncheckedOptionalAccessModelOptions const&)::$_7>::_M_invoke(std::_Any_data const&, clang::CallExpr const*&&, clang::ast_matchers::MatchFinder::MatchResult const&, clang::dataflow::Environment const&) () pytorch#7 0x00000000030b1292 in std::_Function_handler<std::vector<clang::SourceLocation, std::allocator<clang::SourceLocation> > (clang::Stmt const*, clang::ast_matchers::MatchFinder::MatchResult const&, clang::dataflow::Environment const&), clang::dataflow::MatchSwitchBuilder<clang::dataflow::Environment const, std::vector<clang::SourceLocation, std::allocator<clang::SourceLocation> > >::CaseOf<clang::CallExpr>(clang::ast_matchers::internal::Matcher<clang::Stmt>, std::function<std::vector<clang::SourceLocation, std::allocator<clang::SourceLocation> > (clang::CallExpr const*, clang::ast_matchers::MatchFinder::MatchResult const&, clang::dataflow::Environment const&)>) &&::{lambda(clang::Stmt const*, clang::ast_matchers::MatchFinder::MatchResult const&, clang::dataflow::Environment const&)pytorch#1}>::_M_invoke(std::_Any_data const&, clang::Stmt const*&&, clang::ast_matchers::MatchFinder::MatchResult const&, clang::dataflow::Environment const&) () pytorch#8 0x00000000030b1995 in clang::dataflow::MatchSwitchBuilder<clang::dataflow::Environment const, std::vector<clang::SourceLocation, std::allocator<clang::SourceLocation> > >::Build() &&::{lambda(clang::Stmt const&, clang::ASTContext&, clang::dataflow::Environment const&)pytorch#1}::operator()(clang::Stmt const&, clang::ASTContext&, clang::dataflow::Environment const&) const () pytorch#9 0x00000000030b170c in std::_Function_handler<std::vector<clang::SourceLocation, std::allocator<clang::SourceLocation> > (clang::Stmt const&, clang::ASTContext&, clang::dataflow::Environment const&), clang::dataflow::MatchSwitchBuilder<clang::dataflow::Environment const, std::vector<clang::SourceLocation, std::allocator<clang::SourceLocation> > >::Build() &&::{lambda(clang::Stmt const&, clang::ASTContext&, clang::dataflow::Environment const&)pytorch#1}>::_M_invoke(std::_Any_data const&, clang::Stmt const&, clang::ASTContext&, clang::dataflow::Environment const&) () pytorch#10 0x00000000030a7c27 in clang::dataflow::UncheckedOptionalAccessDiagnoser::diagnose(clang::ASTContext&, clang::Stmt const*, clang::dataflow::Environment const&) () pytorch#11 0x0000000002931286 in std::_Function_handler<void (clang::Stmt const*, clang::dataflow::DataflowAnalysisState<clang::dataflow::NoopLattice> const&), clang::tidy::bugprone::analyzeFunction(clang::FunctionDecl const&, clang::ASTContext&)::$_0>::_M_invoke(std::_Any_data const&, clang::Stmt const*&&, clang::dataflow::DataflowAnalysisState<clang::dataflow::NoopLattice> const&) () pytorch#12 0x0000000002930b41 in clang::dataflow::runDataflowAnalysis<clang::dataflow::UncheckedOptionalAccessModel>(clang::dataflow::ControlFlowContext const&, clang::dataflow::UncheckedOptionalAccessModel&, clang::dataflow::Environment const&, std::function<void (clang::Stmt const*, clang::dataflow::DataflowAnalysisState<clang::dataflow::UncheckedOptionalAccessModel::Lattice> const&)>)::{lambda(clang::Stmt const*, clang::dataflow::TypeErasedDataflowAnalysisState const&)pytorch#1}::operator()(clang::Stmt const*, clang::dataflow::TypeErasedDataflowAnalysisState const&) const () pytorch#13 0x00000000030c18cc in std::_Function_handler<void (clang::CFGStmt const&, clang::dataflow::TypeErasedDataflowAnalysisState const&), clang::dataflow::runTypeErasedDataflowAnalysis(clang::dataflow::ControlFlowContext const&, clang::dataflow::TypeErasedDataflowAnalysis&, clang::dataflow::Environment const&, std::function<void (clang::Stmt const*, clang::dataflow::TypeErasedDataflowAnalysisState const&)>)::$_1>::_M_invoke(std::_Any_data const&, clang::CFGStmt const&, clang::dataflow::TypeErasedDataflowAnalysisState const&) () pytorch#14 0x00000000030bf069 in clang::dataflow::transferBlock(clang::dataflow::ControlFlowContext const&, std::vector<llvm::Optional<clang::dataflow::TypeErasedDataflowAnalysisState>, std::allocator<llvm::Optional<clang::dataflow::TypeErasedDataflowAnalysisState> > >&, clang::CFGBlock const&, clang::dataflow::Environment const&, clang::dataflow::TypeErasedDataflowAnalysis&, std::function<void (clang::CFGStmt const&, clang::dataflow::TypeErasedDataflowAnalysisState const&)>) () pytorch#15 0x00000000030bfaa5 in clang::dataflow::runTypeErasedDataflowAnalysis(clang::dataflow::ControlFlowContext const&, clang::dataflow::TypeErasedDataflowAnalysis&, clang::dataflow::Environment const&, std::function<void (clang::Stmt const*, clang::dataflow::TypeErasedDataflowAnalysisState const&)>) () pytorch#16 0x00000000029301b3 in llvm::Expected<std::vector<llvm::Optional<clang::dataflow::DataflowAnalysisState<clang::dataflow::UncheckedOptionalAccessModel::Lattice> >, std::allocator<llvm::Optional<clang::dataflow::DataflowAnalysisState<clang::dataflow::UncheckedOptionalAccessModel::Lattice> > > > > clang::dataflow::runDataflowAnalysis<clang::dataflow::UncheckedOptionalAccessModel>(clang::dataflow::ControlFlowContext const&, clang::dataflow::UncheckedOptionalAccessModel&, clang::dataflow::Environment const&, std::function<void (clang::Stmt const*, clang::dataflow::DataflowAnalysisState<clang::dataflow::UncheckedOptionalAccessModel::Lattice> const&)>) () pytorch#17 0x000000000292fbe8 in clang::tidy::bugprone::UncheckedOptionalAccessCheck::check(clang::ast_matchers::MatchFinder::MatchResult const&) () pytorch#18 0x00000000022e1572 in clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor::MatchVisitor::visitMatch(clang::ast_matchers::BoundNodes const&) () pytorch#19 0x0000000002797a1c in clang::ast_matchers::internal::BoundNodesTreeBuilder::visitMatches(clang::ast_matchers::internal::BoundNodesTreeBuilder::Visitor*) () pytorch#20 0x00000000022e0dc6 in clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor::matchWithFilter(clang::DynTypedNode const&) () pytorch#21 0x00000000022e3b57 in clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor::TraverseDecl(clang::Decl*) () pytorch#22 0x00000000022e4c0c in clang::RecursiveASTVisitor<clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor>::TraverseDecl(clang::Decl*) () pytorch#23 0x00000000022e3b62 in clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor::TraverseDecl(clang::Decl*) () pytorch#24 0x00000000022e4c0c in clang::RecursiveASTVisitor<clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor>::TraverseDecl(clang::Decl*) () pytorch#25 0x00000000022e3b62 in clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor::TraverseDecl(clang::Decl*) () pytorch#26 0x00000000022e4c0c in clang::RecursiveASTVisitor<clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor>::TraverseDecl(clang::Decl*) () pytorch#27 0x00000000022e3b62 in clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor::TraverseDecl(clang::Decl*) () pytorch#28 0x00000000022e4c0c in clang::RecursiveASTVisitor<clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor>::TraverseDecl(clang::Decl*) () pytorch#29 0x00000000022e3b62 in clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor::TraverseDecl(clang::Decl*) () pytorch#30 0x00000000022e8791 in clang::RecursiveASTVisitor<clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor>::TraverseDecl(clang::Decl*) () pytorch#31 0x00000000022e3b62 in clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor::TraverseDecl(clang::Decl*) () pytorch#32 0x00000000022c017a in clang::ast_matchers::MatchFinder::matchAST(clang::ASTContext&) () pytorch#33 0x000000000370ad3c in clang::MultiplexConsumer::HandleTranslationUnit(clang::ASTContext&) () pytorch#34 0x00000000038ed4bb in clang::ParseAST(clang::Sema&, bool, bool) () pytorch#35 0x000000000369eda7 in clang::FrontendAction::Execute() () pytorch#36 0x000000000360d3f6 in clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) () pytorch#37 0x00000000027c475c in clang::tooling::FrontendActionFactory::runInvocation(std::shared_ptr<clang::CompilerInvocation>, clang::FileManager*, std::shared_ptr<clang::PCHContainerOperations>, clang::DiagnosticConsumer*) () pytorch#38 0x00000000022ad486 in clang::tidy::runClangTidy(clang::tidy::ClangTidyContext&, clang::tooling::CompilationDatabase const&, llvm::ArrayRef<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, llvm::IntrusiveRefCntPtr<llvm::vfs::OverlayFileSystem>, bool, bool, llvm::StringRef)::ActionFactory::runInvocation(std::shared_ptr<clang::CompilerInvocation>, clang::FileManager*, std::shared_ptr<clang::PCHContainerOperations>, clang::DiagnosticConsumer*) () pytorch#39 0x00000000027c44c6 in clang::tooling::ToolInvocation::runInvocation(char const*, clang::driver::Compilation*, std::shared_ptr<clang::CompilerInvocation>, std::shared_ptr<clang::PCHContainerOperations>) () pytorch#40 0x00000000027c360b in clang::tooling::ToolInvocation::run() () pytorch#41 0x00000000027c5bb1 in clang::tooling::ClangTool::run(clang::tooling::ToolAction*) () pytorch#42 0x00000000022a90c7 in clang::tidy::runClangTidy(clang::tidy::ClangTidyContext&, clang::tooling::CompilationDatabase const&, llvm::ArrayRef<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, llvm::IntrusiveRefCntPtr<llvm::vfs::OverlayFileSystem>, bool, bool, llvm::StringRef) () pytorch#43 0x0000000001ebc7f2 in clang::tidy::clangTidyMain(int, char const**) () pytorch#44 0x0000000004c54ba0 in __libc_start_main () pytorch#45 0x0000000001eb76ae in _start () ``` Another note is that clang-tidy is CPU-bound. So we could consider running lintrunner job on 4xlarge if needed. Pull Request resolved: pytorch#115124 Approved by: https://github.com/kit1980, https://github.com/Skylion007, https://github.com/malfet

… to hang (pytorch#115124) Let's see if it helps pytorch#114913 The issues on llvm are at llvm/llvm-project#55530 and llvm/llvm-project#69369. In my CI test, I saw the following process hanged: ``` /pytorch/pytorch/.lintbin/clang-tidy -p=/pytorch/pytorch/build --extra-arg -I/usr/lib/llvm-11/include/openmp --extra-arg -I/opt/conda/envs/py_3.9/include/python3.9 --extra-arg -I/pytorch/pytorch/third_party/pybind11/include --extra-arg -I/usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11 --extra-arg -I/usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/x86_64-linux-gnu/c++/11 --extra-arg -I/usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/backward --extra-arg -I/usr/lib/llvm-14/lib/clang/14.0.0/include --extra-arg -I/usr/local/include --extra-arg -I/usr/include/x86_64-linux-gnu --extra-arg -I/usr/include /pytorch/pytorch/torch/csrc/autograd/python_nested_functions_manual.cpp ``` and the core dump matches the description found in llvm/llvm-project#69369 showing the stuck in `clang::tidy::bugprone::UncheckedOptionalAccessCheck::check`: ``` #0 0x00000000030c7420 in clang::dataflow::WatchedLiteralsSolverImpl::updateWatchedLiterals() () #1 0x00000000030c6c2a in clang::dataflow::WatchedLiteralsSolverImpl::solve() && () #2 0x00000000030c6572 in clang::dataflow::WatchedLiteralsSolver::solve(llvm::DenseSet<clang::dataflow::BoolValue*, llvm::DenseMapInfo<clang::dataflow::BoolValue*, void> >) () #3 0x00000000030b3bd3 in clang::dataflow::DataflowAnalysisContext::querySolver(llvm::DenseSet<clang::dataflow::BoolValue*, llvm::DenseMapInfo<clang::dataflow::BoolValue*, void> >) () pytorch#4 0x00000000030b3ca5 in clang::dataflow::DataflowAnalysisContext::flowConditionImplies(clang::dataflow::AtomicBoolValue&, clang::dataflow::BoolValue&) () pytorch#5 0x00000000030b1213 in clang::dataflow::(anonymous namespace)::diagnoseUnwrapCall(clang::Expr const*, clang::Expr const*, clang::dataflow::Environment const&) () pytorch#6 0x00000000030b1357 in std::_Function_handler<std::vector<clang::SourceLocation, std::allocator<clang::SourceLocation> > (clang::CallExpr const*, clang::ast_matchers::MatchFinder::MatchResult const&, clang::dataflow::Environment const&), clang::dataflow::(anonymous namespace)::buildDiagnoseMatchSwitch(clang::dataflow::UncheckedOptionalAccessModelOptions const&)::$_7>::_M_invoke(std::_Any_data const&, clang::CallExpr const*&&, clang::ast_matchers::MatchFinder::MatchResult const&, clang::dataflow::Environment const&) () pytorch#7 0x00000000030b1292 in std::_Function_handler<std::vector<clang::SourceLocation, std::allocator<clang::SourceLocation> > (clang::Stmt const*, clang::ast_matchers::MatchFinder::MatchResult const&, clang::dataflow::Environment const&), clang::dataflow::MatchSwitchBuilder<clang::dataflow::Environment const, std::vector<clang::SourceLocation, std::allocator<clang::SourceLocation> > >::CaseOf<clang::CallExpr>(clang::ast_matchers::internal::Matcher<clang::Stmt>, std::function<std::vector<clang::SourceLocation, std::allocator<clang::SourceLocation> > (clang::CallExpr const*, clang::ast_matchers::MatchFinder::MatchResult const&, clang::dataflow::Environment const&)>) &&::{lambda(clang::Stmt const*, clang::ast_matchers::MatchFinder::MatchResult const&, clang::dataflow::Environment const&)#1}>::_M_invoke(std::_Any_data const&, clang::Stmt const*&&, clang::ast_matchers::MatchFinder::MatchResult const&, clang::dataflow::Environment const&) () pytorch#8 0x00000000030b1995 in clang::dataflow::MatchSwitchBuilder<clang::dataflow::Environment const, std::vector<clang::SourceLocation, std::allocator<clang::SourceLocation> > >::Build() &&::{lambda(clang::Stmt const&, clang::ASTContext&, clang::dataflow::Environment const&)#1}::operator()(clang::Stmt const&, clang::ASTContext&, clang::dataflow::Environment const&) const () pytorch#9 0x00000000030b170c in std::_Function_handler<std::vector<clang::SourceLocation, std::allocator<clang::SourceLocation> > (clang::Stmt const&, clang::ASTContext&, clang::dataflow::Environment const&), clang::dataflow::MatchSwitchBuilder<clang::dataflow::Environment const, std::vector<clang::SourceLocation, std::allocator<clang::SourceLocation> > >::Build() &&::{lambda(clang::Stmt const&, clang::ASTContext&, clang::dataflow::Environment const&)#1}>::_M_invoke(std::_Any_data const&, clang::Stmt const&, clang::ASTContext&, clang::dataflow::Environment const&) () pytorch#10 0x00000000030a7c27 in clang::dataflow::UncheckedOptionalAccessDiagnoser::diagnose(clang::ASTContext&, clang::Stmt const*, clang::dataflow::Environment const&) () pytorch#11 0x0000000002931286 in std::_Function_handler<void (clang::Stmt const*, clang::dataflow::DataflowAnalysisState<clang::dataflow::NoopLattice> const&), clang::tidy::bugprone::analyzeFunction(clang::FunctionDecl const&, clang::ASTContext&)::$_0>::_M_invoke(std::_Any_data const&, clang::Stmt const*&&, clang::dataflow::DataflowAnalysisState<clang::dataflow::NoopLattice> const&) () pytorch#12 0x0000000002930b41 in clang::dataflow::runDataflowAnalysis<clang::dataflow::UncheckedOptionalAccessModel>(clang::dataflow::ControlFlowContext const&, clang::dataflow::UncheckedOptionalAccessModel&, clang::dataflow::Environment const&, std::function<void (clang::Stmt const*, clang::dataflow::DataflowAnalysisState<clang::dataflow::UncheckedOptionalAccessModel::Lattice> const&)>)::{lambda(clang::Stmt const*, clang::dataflow::TypeErasedDataflowAnalysisState const&)#1}::operator()(clang::Stmt const*, clang::dataflow::TypeErasedDataflowAnalysisState const&) const () pytorch#13 0x00000000030c18cc in std::_Function_handler<void (clang::CFGStmt const&, clang::dataflow::TypeErasedDataflowAnalysisState const&), clang::dataflow::runTypeErasedDataflowAnalysis(clang::dataflow::ControlFlowContext const&, clang::dataflow::TypeErasedDataflowAnalysis&, clang::dataflow::Environment const&, std::function<void (clang::Stmt const*, clang::dataflow::TypeErasedDataflowAnalysisState const&)>)::$_1>::_M_invoke(std::_Any_data const&, clang::CFGStmt const&, clang::dataflow::TypeErasedDataflowAnalysisState const&) () pytorch#14 0x00000000030bf069 in clang::dataflow::transferBlock(clang::dataflow::ControlFlowContext const&, std::vector<llvm::Optional<clang::dataflow::TypeErasedDataflowAnalysisState>, std::allocator<llvm::Optional<clang::dataflow::TypeErasedDataflowAnalysisState> > >&, clang::CFGBlock const&, clang::dataflow::Environment const&, clang::dataflow::TypeErasedDataflowAnalysis&, std::function<void (clang::CFGStmt const&, clang::dataflow::TypeErasedDataflowAnalysisState const&)>) () pytorch#15 0x00000000030bfaa5 in clang::dataflow::runTypeErasedDataflowAnalysis(clang::dataflow::ControlFlowContext const&, clang::dataflow::TypeErasedDataflowAnalysis&, clang::dataflow::Environment const&, std::function<void (clang::Stmt const*, clang::dataflow::TypeErasedDataflowAnalysisState const&)>) () pytorch#16 0x00000000029301b3 in llvm::Expected<std::vector<llvm::Optional<clang::dataflow::DataflowAnalysisState<clang::dataflow::UncheckedOptionalAccessModel::Lattice> >, std::allocator<llvm::Optional<clang::dataflow::DataflowAnalysisState<clang::dataflow::UncheckedOptionalAccessModel::Lattice> > > > > clang::dataflow::runDataflowAnalysis<clang::dataflow::UncheckedOptionalAccessModel>(clang::dataflow::ControlFlowContext const&, clang::dataflow::UncheckedOptionalAccessModel&, clang::dataflow::Environment const&, std::function<void (clang::Stmt const*, clang::dataflow::DataflowAnalysisState<clang::dataflow::UncheckedOptionalAccessModel::Lattice> const&)>) () pytorch#17 0x000000000292fbe8 in clang::tidy::bugprone::UncheckedOptionalAccessCheck::check(clang::ast_matchers::MatchFinder::MatchResult const&) () pytorch#18 0x00000000022e1572 in clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor::MatchVisitor::visitMatch(clang::ast_matchers::BoundNodes const&) () pytorch#19 0x0000000002797a1c in clang::ast_matchers::internal::BoundNodesTreeBuilder::visitMatches(clang::ast_matchers::internal::BoundNodesTreeBuilder::Visitor*) () pytorch#20 0x00000000022e0dc6 in clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor::matchWithFilter(clang::DynTypedNode const&) () pytorch#21 0x00000000022e3b57 in clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor::TraverseDecl(clang::Decl*) () pytorch#22 0x00000000022e4c0c in clang::RecursiveASTVisitor<clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor>::TraverseDecl(clang::Decl*) () pytorch#23 0x00000000022e3b62 in clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor::TraverseDecl(clang::Decl*) () pytorch#24 0x00000000022e4c0c in clang::RecursiveASTVisitor<clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor>::TraverseDecl(clang::Decl*) () pytorch#25 0x00000000022e3b62 in clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor::TraverseDecl(clang::Decl*) () pytorch#26 0x00000000022e4c0c in clang::RecursiveASTVisitor<clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor>::TraverseDecl(clang::Decl*) () pytorch#27 0x00000000022e3b62 in clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor::TraverseDecl(clang::Decl*) () pytorch#28 0x00000000022e4c0c in clang::RecursiveASTVisitor<clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor>::TraverseDecl(clang::Decl*) () pytorch#29 0x00000000022e3b62 in clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor::TraverseDecl(clang::Decl*) () pytorch#30 0x00000000022e8791 in clang::RecursiveASTVisitor<clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor>::TraverseDecl(clang::Decl*) () pytorch#31 0x00000000022e3b62 in clang::ast_matchers::internal::(anonymous namespace)::MatchASTVisitor::TraverseDecl(clang::Decl*) () pytorch#32 0x00000000022c017a in clang::ast_matchers::MatchFinder::matchAST(clang::ASTContext&) () pytorch#33 0x000000000370ad3c in clang::MultiplexConsumer::HandleTranslationUnit(clang::ASTContext&) () pytorch#34 0x00000000038ed4bb in clang::ParseAST(clang::Sema&, bool, bool) () pytorch#35 0x000000000369eda7 in clang::FrontendAction::Execute() () pytorch#36 0x000000000360d3f6 in clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) () pytorch#37 0x00000000027c475c in clang::tooling::FrontendActionFactory::runInvocation(std::shared_ptr<clang::CompilerInvocation>, clang::FileManager*, std::shared_ptr<clang::PCHContainerOperations>, clang::DiagnosticConsumer*) () pytorch#38 0x00000000022ad486 in clang::tidy::runClangTidy(clang::tidy::ClangTidyContext&, clang::tooling::CompilationDatabase const&, llvm::ArrayRef<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, llvm::IntrusiveRefCntPtr<llvm::vfs::OverlayFileSystem>, bool, bool, llvm::StringRef)::ActionFactory::runInvocation(std::shared_ptr<clang::CompilerInvocation>, clang::FileManager*, std::shared_ptr<clang::PCHContainerOperations>, clang::DiagnosticConsumer*) () pytorch#39 0x00000000027c44c6 in clang::tooling::ToolInvocation::runInvocation(char const*, clang::driver::Compilation*, std::shared_ptr<clang::CompilerInvocation>, std::shared_ptr<clang::PCHContainerOperations>) () pytorch#40 0x00000000027c360b in clang::tooling::ToolInvocation::run() () pytorch#41 0x00000000027c5bb1 in clang::tooling::ClangTool::run(clang::tooling::ToolAction*) () pytorch#42 0x00000000022a90c7 in clang::tidy::runClangTidy(clang::tidy::ClangTidyContext&, clang::tooling::CompilationDatabase const&, llvm::ArrayRef<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, llvm::IntrusiveRefCntPtr<llvm::vfs::OverlayFileSystem>, bool, bool, llvm::StringRef) () pytorch#43 0x0000000001ebc7f2 in clang::tidy::clangTidyMain(int, char const**) () pytorch#44 0x0000000004c54ba0 in __libc_start_main () pytorch#45 0x0000000001eb76ae in _start () ``` Another note is that clang-tidy is CPU-bound. So we could consider running lintrunner job on 4xlarge if needed. Pull Request resolved: pytorch#115124 Approved by: https://github.com/kit1980, https://github.com/Skylion007, https://github.com/malfet

github-actions bot added the clang Clang issues not falling into any other category label Oct 17, 2023

EugeneZelenko added clangd clang:dataflow Clang Dataflow Analysis framework - https://clang.llvm.org/docs/DataFlowAnalysisIntro.html and removed clang Clang issues not falling into any other category labels Oct 17, 2023

EugeneZelenko added the hang Compiler hang (infinite loop) label Oct 17, 2023

HighCommander4 added the clang-tidy label Oct 18, 2023

HighCommander4 changed the title ~~clangd dead loop in clang::dataflow::WatchedLiteralsSolverImpl::solve(long long)~~ clangd dead loop in clang::dataflow::WatchedLiteralsSolverImpl::solve(long long) when running bugprone-unchecked-optional-access checker Oct 18, 2023

kadircet mentioned this issue Oct 18, 2023

[clangd] Disable crashy unchecked-optional-access tidy check #69427

Merged

kadircet added a commit to kadircet/llvm-project that referenced this issue Oct 18, 2023

[clangd] Disable crashy unchecked-optional-access tidy check

e19bf2e

Fixes llvm#69369.

EugeneZelenko removed clangd clang:dataflow Clang Dataflow Analysis framework - https://clang.llvm.org/docs/DataFlowAnalysisIntro.html labels Oct 18, 2023

kadircet closed this as completed in #69427 Oct 19, 2023

EugeneZelenko reopened this Oct 19, 2023

owenca closed this as completed Oct 19, 2023

EugeneZelenko reopened this Oct 19, 2023

tru closed this as completed in llvm/llvm-project-release-prs@afbe354 Oct 25, 2023

EugeneZelenko reopened this Oct 25, 2023

neoncube2 mentioned this issue Nov 6, 2023

clang-tidy: bugprone-unchecked-optional-access regression (hang) from 15.0.7 to 16.0.6 #69298

Open

huydhn mentioned this issue Dec 5, 2023

Disable bugprone-unchecked-optional-access as it can cause clang-tidy to hang pytorch/pytorch#115124

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

dead loop in clang::dataflow::WatchedLiteralsSolverImpl::solve(long long) when running bugprone-unchecked-optional-access tidy-check #69369

dead loop in clang::dataflow::WatchedLiteralsSolverImpl::solve(long long) when running bugprone-unchecked-optional-access tidy-check #69369

andy-yx-chen commented Oct 17, 2023

llvmbot commented Oct 17, 2023

llvmbot commented Oct 18, 2023

HighCommander4 commented Oct 18, 2023

andy-yx-chen commented Oct 18, 2023

EugeneZelenko commented Oct 19, 2023

kadircet commented Oct 20, 2023

EugeneZelenko commented Oct 25, 2023

jcsxky commented Oct 26, 2023

HighCommander4 commented Oct 26, 2023

jcsxky commented Oct 27, 2023

dead loop in clang::dataflow::WatchedLiteralsSolverImpl::solve(long long) when running bugprone-unchecked-optional-access tidy-check #69369

dead loop in clang::dataflow::WatchedLiteralsSolverImpl::solve(long long) when running bugprone-unchecked-optional-access tidy-check #69369

Comments

andy-yx-chen commented Oct 17, 2023

llvmbot commented Oct 17, 2023

llvmbot commented Oct 18, 2023

HighCommander4 commented Oct 18, 2023

andy-yx-chen commented Oct 18, 2023

EugeneZelenko commented Oct 19, 2023

kadircet commented Oct 20, 2023

EugeneZelenko commented Oct 25, 2023

jcsxky commented Oct 26, 2023

HighCommander4 commented Oct 26, 2023

jcsxky commented Oct 27, 2023