Multithreading issue with DWARFContext::getLineInfoForAddress #167285
Description
We are running DWARFContext in MT threading environment and have found a memory issiue with it that is reasonably hard to reproduce, but we managed to get callstacks.
Initialization:
llvm::DWARFContext::create(*objectFile.macho,
/*RelocAction=*/llvm::DWARFContext::ProcessDebugRelocations::Process,
/*L=*/nullptr,
/*DWPName=*/{},
llvm::WithColor::defaultErrorHandler,
llvm::WithColor::defaultWarningHandler,
/*ThreadSafe=*/true)
Here are the call stacks
Thread 4:: llvm-worker-2
0 libsystem_kernel.dylib 0x194f4ba68 __psynch_mutexdrop + 8
1 libsystem_pthread.dylib 0x194f87ec8 _pthread_mutex_firstfit_wake + 28
2 libsystem_pthread.dylib 0x194f85b08 _pthread_mutex_firstfit_unlock_slow + 240
3 libc++.1.dylib 0x194ebc5c8 std::__1::recursive_mutex::unlock() + 16
4 reproducer-tool 0x103281248 (anonymous namespace)::ThreadSafeState::getNormalUnits() + 124
5 reproducer-tool 0x10327a65c llvm::DWARFContext::getLineInfoForAddress(llvm::object::SectionedAddress, llvm::DILineInfoSpecifier) + 224
Thread 5 Crashed:: llvm-worker-3
0 libsystem_kernel.dylib 0x194f51388 __pthread_kill + 8
1 libsystem_pthread.dylib 0x194f8a88c pthread_kill + 296
2 libsystem_c.dylib 0x194e93c60 abort + 124
3 libsystem_malloc.dylib 0x194d98174 malloc_vreport + 892
4 libsystem_malloc.dylib 0x194d9bc90 malloc_report + 64
5 libsystem_malloc.dylib 0x194da021c ___BUG_IN_CLIENT_OF_LIBMALLOC_POINTER_BEING_FREED_WAS_NOT_ALLOCATED + 32
6 reproducer-tool 0x1032c00a4 llvm::DWARFUnit::extractDIEsToVector(bool, bool, std::__1::vector<llvm::DWARFDebugInfoEntry, std::__1::allocator<llvm::DWARFDebugInfoEntry>>&) const + 1104
7 reproducer-tool 0x1032c03b0 llvm::DWARFUnit::tryExtractDIEsIfNeeded(bool) + 140
8 reproducer-tool 0x1032c02c0 llvm::DWARFUnit::extractDIEsIfNeeded(bool) + 28
9 reproducer-tool 0x1032a7868 llvm::DWARFDie::getAttributeValueAsReferencedDie(llvm::DWARFFormValue const&) const + 188
10 reproducer-tool 0x1032a7520 llvm::DWARFDie::findRecursively(llvm::ArrayRef<llvm::dwarf::Attribute>) const + 352
11 reproducer-tool 0x1032a8874 llvm::DWARFDie::getDeclFile(llvm::DILineInfoSpecifier::FileLineInfoKind) const + 48
12 reproducer-tool 0x10327a858 getFunctionNameAndStartLineForAddress(llvm::DWARFCompileUnit*, unsigned long long, llvm::DINameKind, llvm::DILineInfoSpecifier::FileLineInfoKind, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&, unsigned int&, std::__1::optional<unsigned long long>&) + 164
13 reproducer-tool 0x10327a6a0 llvm::DWARFContext::getLineInfoForAddress(llvm::object::SectionedAddress, llvm::DILineInfoSpecifier) + 292
Thread 6:: llvm-worker-4
0 reproducer-tool 0x1032aca60 std::__1::pair<llvm::SmallSetIterator<llvm::DWARFDie, 3u, std::__1::less<llvm::DWARFDie>>, bool> llvm::SmallSet<llvm::DWARFDie, 3u, std::__1::less<llvm::DWARFDie>>::insertImpl<llvm::DWARFDie const&>(llvm::DWARFDie const&) + 396
1 reproducer-tool 0x1032a75b8 llvm::DWARFDie::findRecursively(llvm::ArrayRef<llvm::dwarf::Attribute>) const + 504
2 reproducer-tool 0x1032a8874 llvm::DWARFDie::getDeclFile(llvm::DILineInfoSpecifier::FileLineInfoKind) const + 48
3 reproducer-tool 0x10327a858 getFunctionNameAndStartLineForAddress(llvm::DWARFCompileUnit*, unsigned long long, llvm::DINameKind, llvm::DILineInfoSpecifier::FileLineInfoKind, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&, unsigned int&, std::__1::optional<unsigned long long>&) + 164
4 reproducer-tool 0x10327a6a0 llvm::DWARFContext::getLineInfoForAddress(llvm::object::SectionedAddress, llvm::DILineInfoSpecifier) + 292
Thread 4 shows we are doing MT-safe DWARFContext.
Thread 5 crashes.
Thread 6 shows that findRecursively has entered as well.
Suspicious transition happens here:
8 reproducer-tool 0x1032c02c0 llvm::DWARFUnit::extractDIEsIfNeeded(bool) + 28
9 reproducer-tool 0x1032a7868 llvm::DWARFDie::getAttributeValueAsReferencedDie(llvm::DWARFFormValue const&) const + 188
DWARFDie is constant, but DWARFUnit is not which would make sense if DWARFUnit was not somehow a state of DWARFDie, but:
DWARFDie
DWARFDie::getAttributeValueAsReferencedDie(const DWARFFormValue &V) const {
DWARFDie Result;
if (std::optional<uint64_t> Offset = V.getAsRelativeReference()) {
Result = const_cast<DWARFUnit *>(V.getUnit())
->getDIEForOffset(V.getUnit()->getOffset() + *Offset);
const_cast's are generally sus.
Turns out there is lazy initialization happening under the hood
Error DWARFUnit::tryExtractDIEsIfNeeded(bool CUDieOnly) {
if ((CUDieOnly && !DieArray.empty()) || DieArray.size() > 1)
return Error::success(); // Already parsed.
bool HasCUDie = !DieArray.empty();
extractDIEsToVector(!HasCUDie, !CUDieOnly, DieArray);
What is interesting, then the similar code in lldb is MT aware:
// Parses a compile unit and indexes its DIEs if it hasn't already been done.
// It will leave this compile unit extracted forever.
void DWARFUnit::ExtractDIEsIfNeeded() {
m_cancel_scopes = true;
{
llvm::sys::ScopedReader lock(m_die_array_mutex);
if (!m_die_array.empty())
return; // Already parsed
}
llvm::sys::ScopedWriter lock(m_die_array_mutex);
if (!m_die_array.empty())
return; // Already parsed
ExtractDIEsRWLocked();
}