C++20 lambdas in unevaluated context: crash on valid code

[jan-moeller]

Bugzilla Link	51032
Version	trunk
OS	Linux
CC	@AaronBallman,@zygoloid

Extended Description

clang (trunk) crashes on the following valid c++20 program:

///////////////////////////////////////////////////////////////////////////////
template <typename T, typename Fn>
struct foo_t {
foo_t(T ptr) {}
};

template
using alias = foo_t<T, decltype( { return 0; })>;

template
auto fun(T const& t) -> alias {
return alias{t};
}

int main() {
int i;
auto const error = fun(i);
}
///////////////////////////////////////////////////////////////////////////////

PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace, preprocessed source, and associated run script.
Stack dump:
0. Program arguments: /opt/compiler-explorer/clang-trunk/bin/clang++ -g -o /app/output.s -mllvm --x86-asm-syntax=intel -S --gcc-toolchain=/opt/compiler-explorer/gcc-snapshot -fcolor-diagnostics -fno-crash-diagnostics --std=c++20

1. parser at end of file
2. :14:5: LLVM IR generation of declaration 'main'
3. :14:5: Generating code for declaration 'main'

#​0 0x000055bedd4185cf PrintStackTraceSignalHandler(void*) Signals.cpp:0:0
#​1 0x000055bedd4164c0 llvm::sys::CleanupOnSignal(unsigned long) (/opt/compiler-explorer/clang-trunk/bin/clang+++0x33664c0)
#​2 0x000055bedd36abe8 CrashRecoverySignalHandler(int) CrashRecoveryContext.cpp:0:0
#​3 0x00007fb1c17453c0 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x153c0)
#​4 0x000055bedfc0547c clang::ASTContext::getTypeInfoImpl(clang::Type const*) const (/opt/compiler-explorer/clang-trunk/bin/clang+++0x5b5547c)
#​5 0x000055bedfbf886d clang::ASTContext::getTypeInfo(clang::Type const*) const (/opt/compiler-explorer/clang-trunk/bin/clang+++0x5b4886d)
#​6 0x000055bedfc0426b clang::ASTContext::getPreferredTypeAlign(clang::Type const*) const (/opt/compiler-explorer/clang-trunk/bin/clang+++0x5b5426b)
#​7 0x000055bedfc066bb clang::ASTContext::getDeclAlign(clang::Decl const*, bool) const (/opt/compiler-explorer/clang-trunk/bin/clang+++0x5b566bb)
#​8 0x000055bedd9c6505 clang::CodeGen::CodeGenFunction::EmitAutoVarAlloca(clang::VarDecl const&) (/opt/compiler-explorer/clang-trunk/bin/clang+++0x3916505)
#​9 0x000055bedd9c727a clang::CodeGen::CodeGenFunction::EmitVarDecl(clang::VarDecl const&) (/opt/compiler-explorer/clang-trunk/bin/clang+++0x391727a)
#​10 0x000055bedd9c757d clang::CodeGen::CodeGenFunction::EmitDecl(clang::Decl const&) (/opt/compiler-explorer/clang-trunk/bin/clang+++0x391757d)
#​11 0x000055bedd73f327 clang::CodeGen::CodeGenFunction::EmitDeclStmt(clang::DeclStmt const&) (/opt/compiler-explorer/clang-trunk/bin/clang+++0x368f327)
#​12 0x000055bedd74eaf5 clang::CodeGen::CodeGenFunction::EmitSimpleStmt(clang::Stmt const*, llvm::ArrayRef<clang::Attr const*>) (/opt/compiler-explorer/clang-trunk/bin/clang+++0x369eaf5)
#​13 0x000055bedd7493e2 clang::CodeGen::CodeGenFunction::EmitStmt(clang::Stmt const*, llvm::ArrayRef<clang::Attr const*>) (/opt/compiler-explorer/clang-trunk/bin/clang+++0x36993e2)
#​14 0x000055bedd74e60c clang::CodeGen::CodeGenFunction::EmitCompoundStmtWithoutScope(clang::CompoundStmt const&, bool, clang::CodeGen::AggValueSlot) (/opt/compiler-explorer/clang-trunk/bin/clang+++0x369e60c)
#​15 0x000055bedd79b101 clang::CodeGen::CodeGenFunction::EmitFunctionBody(clang::Stmt const*) (/opt/compiler-explorer/clang-trunk/bin/clang+++0x36eb101)
#​16 0x000055bedd7a3a17 clang::CodeGen::CodeGenFunction::GenerateCode(clang::GlobalDecl, llvm::Function*, clang::CodeGen::CGFunctionInfo const&) (/opt/compiler-explorer/clang-trunk/bin/clang+++0x36f3a17)
#​17 0x000055bedd7e8ffe clang::CodeGen::CodeGenModule::EmitGlobalFunctionDefinition(clang::GlobalDecl, llvm::GlobalValue*) (/opt/compiler-explorer/clang-trunk/bin/clang+++0x3738ffe)
#​18 0x000055bedd7e5e25 clang::CodeGen::CodeGenModule::EmitGlobalDefinition(clang::GlobalDecl, llvm::GlobalValue*) (/opt/compiler-explorer/clang-trunk/bin/clang+++0x3735e25)
#​19 0x000055bedd7e652b clang::CodeGen::CodeGenModule::EmitGlobal(clang::GlobalDecl) (/opt/compiler-explorer/clang-trunk/bin/clang+++0x373652b)
#​20 0x000055bedd7ec101 clang::CodeGen::CodeGenModule::EmitTopLevelDecl(clang::Decl*) (.part.5273) CodeGenModule.cpp:0:0
#​21 0x000055bede34be51 (anonymous namespace)::CodeGeneratorImpl::HandleTopLevelDecl(clang::DeclGroupRef) ModuleBuilder.cpp:0:0
#​22 0x000055bede33fc42 clang::BackendConsumer::HandleTopLevelDecl(clang::DeclGroupRef) (/opt/compiler-explorer/clang-trunk/bin/clang+++0x428fc42)
#​23 0x000055bedf19f1c4 clang::ParseAST(clang::Sema&, bool, bool) (/opt/compiler-explorer/clang-trunk/bin/clang+++0x50ef1c4)
#​24 0x000055bede349d62 clang::CodeGenAction::ExecuteAction() (/opt/compiler-explorer/clang-trunk/bin/clang+++0x4299d62)
#​25 0x000055beddcfbe61 clang::FrontendAction::Execute() (/opt/compiler-explorer/clang-trunk/bin/clang+++0x3c4be61)
#​26 0x000055beddc99bc2 clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) (/opt/compiler-explorer/clang-trunk/bin/clang+++0x3be9bc2)
#​27 0x000055bedddc8343 clang::ExecuteCompilerInvocation(clang::CompilerInstance*) (/opt/compiler-explorer/clang-trunk/bin/clang+++0x3d18343)
#​28 0x000055bedb21f00c cc1_main(llvm::ArrayRef<char const*>, char const*, void*) (/opt/compiler-explorer/clang-trunk/bin/clang+++0x116f00c)
#​29 0x000055bedb21b19d ExecuteCC1Tool(llvm::SmallVectorImpl<char const*>&) driver.cpp:0:0
#​30 0x000055beddb486c5 void llvm::function_ref<void ()>::callback_fn<clang::driver::CC1Command::Execute(llvm::ArrayRef<llvm::Optionalllvm::StringRef >, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, bool) const::'lambda'()>(long) Job.cpp:0:0
#​31 0x000055bedd36b1d3 llvm::CrashRecoveryContext::RunSafely(llvm::function_ref<void ()>) (/opt/compiler-explorer/clang-trunk/bin/clang+++0x32bb1d3)
#​32 0x000055beddb4a59e clang::driver::CC1Command::Execute(llvm::ArrayRef<llvm::Optionalllvm::StringRef >, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, bool) const (/opt/compiler-explorer/clang-trunk/bin/clang+++0x3a9a59e)
#​33 0x000055beddb2075a clang::driver::Compilation::ExecuteCommand(clang::driver::Command const&, clang::driver::Command const*&) const (/opt/compiler-explorer/clang-trunk/bin/clang+++0x3a7075a)
#​34 0x000055beddb212af clang::driver::Compilation::ExecuteJobs(clang::driver::JobList const&, llvm::SmallVectorImpl<std::pair<int, clang::driver::Command const*> >&) const (/opt/compiler-explorer/clang-trunk/bin/clang+++0x3a712af)
#​35 0x000055beddb2a255 clang::driver::Driver::ExecuteCompilation(clang::driver::Compilation&, llvm::SmallVectorImpl<std::pair<int, clang::driver::Command const*> >&) (/opt/compiler-explorer/clang-trunk/bin/clang+++0x3a7a255)
#​36 0x000055bedb133993 main (/opt/compiler-explorer/clang-trunk/bin/clang+++0x1083993)
#​37 0x00007fb1c11f50b3 __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b3)
#​38 0x000055bedb21ad1a _start (/opt/compiler-explorer/clang-trunk/bin/clang+++0x116ad1a)
clang-13: error: clang frontend command failed with exit code 139 (use -v to see invocation)
Compiler returned: 139

This was found while reporting a gcc bug (101315) and comparing gcc's output to clang's. See this link on godbolt: https://godbolt.org/z/5fcK1az5M

[cor3ntin]

I spent some time looking at this, without getting at a solution. Maybe the notes can help whoever look at this (@AaronBallman or @erichkeane ?)

Most/all crashes related to unevaluated lambda seem to boil done to this:

template<typename T>
auto foo(decltype([]{}) p){
}

void test(){
    foo<int>({});
}

What I think is happening is that the substitution of p is done in the not yet instantiated declaration of foo,
such that when Creating the lambda class (and therefore its type), isDependentContext() is true, and so the instantiated lambda type is always dependent and so anything that try to use the type, whether codegen - when generating the function type - or when constant evaluating it - crashes.

The following code is fine, because when the body is evaluated, it is not considered a dependent context by the time it is instantiated.

template<typename T>
auto foo(){
    decltype([]{}) bar;
    bar();
}

void test(){
    foo<int>();
}

The specific issue is here,

llvm-project/clang/lib/AST/DeclBase.cpp

Lines 1166 to 1168 in f6ee45e

	if (const auto *Function = dyn_cast<FunctionDecl>(this)) {
	if (Function->getDescribedFunctionTemplate())
	return true;

This gets called in several places during the creation of the lambda.
We might need to check inTemplateInstantiation() somewhere... but where?

Edit a few hours later:

Fundamentally I think the issue is that lambdas should be declared in the context in which they appear, but at the time we instantiate parameters, the instantiated function declaration ( of foo in the first code snippet) does not yet exist.
I managed to "fix" the crash by changing the context of the instantiated lambda class, I'm not sure how to do that cleanly,

[llvmbot]

@llvm/issue-subscribers-clang-codegen

[erichkeane]

I'd be a little surprised to see that dependent stuff makes it to codegen, typically uninstantiated templates don't end up needing to get emitted.

That said, your suspicion of it being in the wrong context (I too would expect this to be in the context of 'foo' in the first example) is perhaps right! If it ends up in the global DeclContext, I'd expect it would get caught int he normal 'emit globals' functionality.

As for when to set this to the right context, I would probably look to see where ParmVarDecls have their context set (as they have the same problem; we don't have a declaration to associate them with at the point in which they are created I think, so the DeclContext gets set later), and see if you can assign it there. Additionally, the TemplateTypeParmDecl I think have to go through the same rigamarole, so you might find that reasonable as well (though that might not get a chance to set the non-dependent context correctly).

One thing to note: Make sure the context gets set correctly on the instantiation (that is, foo<int>, not foo<T> or the TU Context on instantiation). I mention that, because if you follow the example of TemplateTypeParmDecl, it MIGHT end up not setting the DeclContext correctly (though the non-templated case doesn't crash, so maybe that already works?!).

I'm weekending, so I don't have my debugger in front of me to play with for a bit to figure out better than that, but that would be my guidance.

[cor3ntin]

@erichkeane I suspect it has not been an issue until then as I can't imagine a scenario in which clang was previously minting new record types during instantiations, in the parameter list of a function.

Everything else just transform expressions with the substituted types, or introduce a DC before introducing types.

One thing to note: Make sure the context gets set correctly on the instantiation (that is, foo, not foo or the TU Context on instantiation).

That's what we should do, but the function declaration does not yet exist, so parameters would have to be adjusted after, which might mean doing the whole substitution twice... doesn't seem ideal.

Alternatively, we need to keep track of all the decl created during the instantiation of parameters and adjust their DC later - temporarily put them in the TU' DC, as we do during parsing of parsing of parameters declaration.

The cleanest general solution might be to first construct the FuctionDecl just to attach the parameters to its DC, and then fill its type/other info later.

[erichkeane]

I was saying I'm about 95% sure that with parameters we already create them in the TU DC, then adjust the DC.

See: https://clang.llvm.org/doxygen/Decl_8h_source.html#l01810 and the comment on it!
"/// Sets the function declaration that owns this
/// ParmVarDecl. Since ParmVarDecls are often created before the
/// FunctionDecls that own them, this routine is required to update
/// the DeclContext appropriately."

My suggestion was to find where that happens and do that to any lambda types that are used to declare the parameter as well.

[tstellar]

/cherry-pick 3784e8c

[llvmbot]

/branch llvmbot/llvm-project/issue50376

[llvmbot]

/pull-request llvmbot#151

[tstellar]

The backport of 3784e8c was also requested in #51416 , so I'm dropping the release milestone from this issue. We'll track the progress in #51416.