Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TSAN detected flaky race in concurrent sweeper #44091

Closed
mkustermann opened this issue Nov 6, 2020 · 0 comments
Closed

TSAN detected flaky race in concurrent sweeper #44091

mkustermann opened this issue Nov 6, 2020 · 0 comments
Labels
area-vm Use area-vm for VM related issues, including code coverage, and the AOT and JIT backends. gardening

Comments

@mkustermann
Copy link
Member

From this log:

/=================================================================================================\
| vm/dart_2/entrypoints/jit/polymorphic_optional_this_test/1 broke (Pass -> Crash, expected Pass) |
\=================================================================================================/

--- Command "vm" (took 09.000973s):
DART_CONFIGURATION=ReleaseTSANX64 out/ReleaseTSANX64/dart --enable-testing-pragmas --no-background-compilation --optimization-counter-threshold=10 --compilation-counter-threshold=1 --ignore-unrecognized-flags --packages=/b/s/w/ir/cache/builder/sdk/.packages /b/s/w/ir/cache/builder/sdk/runtime/tests/vm/dart_2/entrypoints/jit/polymorphic_optional_this_test.dart

exit code:
-6

stdout:
7, 93

stderr:
==================
WARNING: ThreadSanitizer: data race (pid=12056)
  Write of size 8 at 0x7fade22fdfb8 by thread T3 (mutexes: write M182):
    #0 dart::FreeListElement::set_next(dart::FreeListElement*) ../../out/ReleaseTSANX64/../../runtime/vm/heap/freelist.h:29 (dart+0x1f95c82)
    #1 dart::FreeListElement::AsElement(unsigned long, long) ../../out/ReleaseTSANX64/../../runtime/vm/heap/freelist.cc:41 (dart+0x1f95c82)
    #2 dart::FreeList::FreeLocked(unsigned long, long) ../../out/ReleaseTSANX64/../../runtime/vm/heap/freelist.cc:188 (dart+0x1f95c82)
    #3 dart::FreeList::Free(unsigned long, long) ../../out/ReleaseTSANX64/../../runtime/vm/heap/freelist.cc:177 (dart+0x1f95c82)
    #4 dart::GCSweeper::SweepPage(dart::OldPage*, dart::FreeList*, bool) ../../out/ReleaseTSANX64/../../runtime/vm/heap/sweeper.cc:67 (dart+0x1fb8728)
    #5 dart::ConcurrentSweeperTask::Run() ../../out/ReleaseTSANX64/../../runtime/vm/heap/sweeper.cc:183 (dart+0x1fb8de7)
    #6 dart::ThreadPool::WorkerLoop(dart::ThreadPool::Worker*) ../../out/ReleaseTSANX64/../../runtime/vm/thread_pool.cc:158 (dart+0x1f5d6ec)
    #7 dart::ThreadPool::Worker::Main(unsigned long) ../../out/ReleaseTSANX64/../../runtime/vm/thread_pool.cc:323 (dart+0x1f5e0d5)
    #8 dart::ThreadStart(void*) ../../out/ReleaseTSANX64/../../runtime/vm/os_thread_linux.cc:153 (dart+0x1e6c2d2)

  Previous write of size 8 at 0x7fade22fdfb8 by thread T5:
    [failed to restore the stack]

  Location is global '<null>' at 0x000000000000 (memfd:dart-oldspace (deleted)+0x00000007dfb8)

  Mutex M182 (0x7b7c00011300) created at:
    #0 pthread_mutex_init ../recipe_cleanup/clangshYTOG/llvm_build_dir/tools/clang/stage2-bins/runtimes/runtimes-x86_64-unknown-linux-gnu-bins/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1220 (dart+0x1a22fcd)
    #1 Mutex ../../out/ReleaseTSANX64/../../runtime/vm/os_thread_linux.cc:300 (dart+0x1e6c707)
    #2 FreeList ../../out/ReleaseTSANX64/../../runtime/vm/heap/freelist.cc:57 (dart+0x1f95147)
    #3 PageSpace ../../out/ReleaseTSANX64/../../runtime/vm/heap/pages.cc:224 (dart+0x1fa3b67)
    #4 Heap ../../out/ReleaseTSANX64/../../runtime/vm/heap/heap.cc:67 (dart+0x1f97966)
    #5 dart::Heap::Init(dart::IsolateGroup*, bool, long, long) ../../out/ReleaseTSANX64/../../runtime/vm/heap/heap.cc:690 (dart+0x1f9c023)
    #6 dart::IsolateGroup::CreateHeap(bool, bool) ../../out/ReleaseTSANX64/../../runtime/vm/isolate.cc:451 (dart+0x1d319f6)
    #7 Dart_CreateIsolateGroup ../../out/ReleaseTSANX64/../../runtime/vm/dart_api_impl.cc:1522 (dart+0x261605d)
    #8 dart::bin::CreateAndSetupServiceIsolate(char const*, char const*, Dart_IsolateFlags*, char**, int*) ../../out/ReleaseTSANX64/../../runtime/bin/main.cc:530 (dart+0x1a9a6be)
    #9 dart::bin::CreateIsolateGroupAndSetup(char const*, char const*, char const*, char const*, Dart_IsolateFlags*, void*, char**) ../../out/ReleaseTSANX64/../../runtime/bin/main.cc:800 (dart+0x1a9a6be)
    #10 dart::RunServiceTask::Run() ../../out/ReleaseTSANX64/../../runtime/vm/service_isolate.cc:356 (dart+0x1f2efc2)
    #11 dart::ThreadPool::WorkerLoop(dart::ThreadPool::Worker*) ../../out/ReleaseTSANX64/../../runtime/vm/thread_pool.cc:158 (dart+0x1f5d6ec)
    #12 dart::ThreadPool::Worker::Main(unsigned long) ../../out/ReleaseTSANX64/../../runtime/vm/thread_pool.cc:323 (dart+0x1f5e0d5)
    #13 dart::ThreadStart(void*) ../../out/ReleaseTSANX64/../../runtime/vm/os_thread_linux.cc:153 (dart+0x1e6c2d2)

  Thread T3 'DartWorker' (tid=12060, running) created by main thread at:
    #0 pthread_create ../recipe_cleanup/clangshYTOG/llvm_build_dir/tools/clang/stage2-bins/runtimes/runtimes-x86_64-unknown-linux-gnu-bins/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:962 (dart+0x1a21b5b)
    #1 dart::OSThread::Start(char const*, void (*)(unsigned long), unsigned long) ../../out/ReleaseTSANX64/../../runtime/vm/os_thread_linux.cc:172 (dart+0x1e6c15c)
    #2 dart::ThreadPool::Worker::StartThread() ../../out/ReleaseTSANX64/../../runtime/vm/thread_pool.cc:296 (dart+0x1f5cda9)
    #3 dart::ThreadPool::RunImpl(std::__2::unique_ptr<dart::ThreadPool::Task, std::__2::default_delete<dart::ThreadPool::Task> >) ../../out/ReleaseTSANX64/../../runtime/vm/thread_pool.cc:94 (dart+0x1f5cda9)
    #4 bool dart::ThreadPool::Run<dart::RunKernelTask>() ../../out/ReleaseTSANX64/../../runtime/vm/thread_pool.h:46 (dart+0x1d6bcfb)
    #5 dart::KernelIsolate::Start() ../../out/ReleaseTSANX64/../../runtime/vm/kernel_isolate.cc:265 (dart+0x1d6bcfb)
    #6 dart::KernelIsolate::DetectNullSafety(char const*, char const*, char const*) ../../out/ReleaseTSANX64/../../runtime/vm/kernel_isolate.cc:1043 (dart+0x1d6cf5a)
    #7 dart::Dart::DetectNullSafety(char const*, unsigned char const*, unsigned char const*, unsigned char const*, long, char const*, char const*) ../../out/ReleaseTSANX64/../../runtime/vm/dart.cc:810 (dart+0x1ce0dd9)
    #8 Dart_DetectNullSafety ../../out/ReleaseTSANX64/../../runtime/vm/dart_api_impl.cc:6260 (dart+0x264d279)
    #9 dart::bin::CreateIsolateGroupAndSetupHelper(bool, char const*, char const*, char const*, Dart_IsolateFlags*, void*, char**, int*) ../../out/ReleaseTSANX64/../../runtime/bin/main.cc:698 (dart+0x1a991f6)
    #10 dart::bin::RunMainIsolate(char const*, dart::bin::CommandLineOptions*) ../../out/ReleaseTSANX64/../../runtime/bin/main.cc:914 (dart+0x1a98853)
    #11 dart::bin::main(int, char**) ../../out/ReleaseTSANX64/../../runtime/bin/main.cc:1317 (dart+0x1a9a0bf)
    #12 main ../../out/ReleaseTSANX64/../../runtime/bin/main.cc:1357 (dart+0x1a9b232)

  Thread T5 'DartWorker' (tid=12066, running) created by thread T2 at:
    #0 pthread_create ../recipe_cleanup/clangshYTOG/llvm_build_dir/tools/clang/stage2-bins/runtimes/runtimes-x86_64-unknown-linux-gnu-bins/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:962 (dart+0x1a21b5b)
    #1 dart::OSThread::Start(char const*, void (*)(unsigned long), unsigned long) ../../out/ReleaseTSANX64/../../runtime/vm/os_thread_linux.cc:172 (dart+0x1e6c15c)
    #2 dart::ThreadPool::Worker::StartThread() ../../out/ReleaseTSANX64/../../runtime/vm/thread_pool.cc:296 (dart+0x1f5cda9)
    #3 dart::ThreadPool::RunImpl(std::__2::unique_ptr<dart::ThreadPool::Task, std::__2::default_delete<dart::ThreadPool::Task> >) ../../out/ReleaseTSANX64/../../runtime/vm/thread_pool.cc:94 (dart+0x1f5cda9)
    #4 bool dart::ThreadPool::Run<dart::ConcurrentMarkTask, dart::GCMarker*, dart::IsolateGroup* const&, dart::PageSpace*&, dart::MarkingVisitorBase<true>*&>(dart::GCMarker*&&, dart::IsolateGroup* const&, dart::PageSpace*&, dart::MarkingVisitorBase<true>*&) ../../out/ReleaseTSANX64/../../runtime/vm/thread_pool.h:46 (dart+0x1f9fe38)
    #5 dart::GCMarker::StartConcurrentMark(dart::PageSpace*) ../../out/ReleaseTSANX64/../../runtime/vm/heap/marker.cc:749 (dart+0x1f9fe38)
    #6 dart::PageSpace::CollectGarbageHelper(bool, bool, long, long) ../../out/ReleaseTSANX64/../../runtime/vm/heap/pages.cc:1178 (dart+0x1fa97dd)
    #7 dart::PageSpace::CollectGarbage(bool, bool) ../../out/ReleaseTSANX64/../../runtime/vm/heap/pages.cc:1118 (dart+0x1fa922e)
    #8 dart::Heap::StartConcurrentMarking(dart::Thread*) ../../out/ReleaseTSANX64/../../runtime/vm/heap/heap.cc:610 (dart+0x1f993fe)
    #9 dart::Heap::CheckStartConcurrentMarking(dart::Thread*, dart::Heap::GCReason) ../../out/ReleaseTSANX64/../../runtime/vm/heap/heap.cc:604 (dart+0x1f993fe)
    #10 dart::PageSpace::EvaluateConcurrentMarking(dart::PageSpace::GrowthPolicy) ../../out/ReleaseTSANX64/../../runtime/vm/heap/pages.cc:462 (dart+0x1fa5ae7)
    #11 dart::PageSpace::TryAllocateInFreshLargePage(long, dart::OldPage::PageType, dart::PageSpace::GrowthPolicy) ../../out/ReleaseTSANX64/../../runtime/vm/heap/pages.cc:511 (dart+0x1fa5ae7)
    #12 dart::PageSpace::TryAllocateInternal(long, dart::FreeList*, dart::OldPage::PageType, dart::PageSpace::GrowthPolicy, bool, bool) ../../out/ReleaseTSANX64/../../runtime/vm/heap/pages.cc:558 (dart+0x1fa5d0b)
    #13 dart::PageSpace::TryAllocate(long, dart::OldPage::PageType, dart::PageSpace::GrowthPolicy) ../../out/ReleaseTSANX64/../../runtime/vm/heap/pages.h:306 (dart+0x1f98a78)
    #14 dart::Heap::AllocateOld(long, dart::OldPage::PageType) ../../out/ReleaseTSANX64/../../runtime/vm/heap/heap.cc:118 (dart+0x1f98a78)
    #15 dart::Object::Allocate(long, long, dart::Heap::Space) ??:? (dart+0x1d8f744)
    #16 dart::Object::Allocate(long, long, dart::Heap::Space) ../../out/ReleaseTSANX64/../../runtime/vm/object.cc:2601 (dart+0x1d8f744)
    #17 dart::Array::New(long, long, dart::Heap::Space) ../../out/ReleaseTSANX64/../../runtime/vm/object.cc:23021 (dart+0x1d97d90)
    #18 dart::Array::New(long, dart::Heap::Space) ../../out/ReleaseTSANX64/../../runtime/vm/object.cc:22993 (dart+0x1d97d90)
    #19 dart::ArrayPtr dart::HashTables::New<dart::HashSet<dart::UnorderedHashTable<dart::SymbolTraits, 0l> > >(long, dart::Heap::Space) ../../out/ReleaseTSANX64/../../runtime/vm/hash_table.h:448 (dart+0x1cb53f7)
    #20 void dart::HashTables::EnsureLoadFactor<dart::HashSet<dart::UnorderedHashTable<dart::SymbolTraits, 0l> > >(double, dart::HashSet<dart::UnorderedHashTable<dart::SymbolTraits, 0l> > const&) ../../out/ReleaseTSANX64/../../runtime/vm/hash_table.h:506 (dart+0x1cb522b)
    #21 dart::HashSet<dart::UnorderedHashTable<dart::SymbolTraits, 0l> >::EnsureCapacity() const ../../out/ReleaseTSANX64/../../runtime/vm/hash_table.h:708 (dart+0x1f4fe5c)
    #22 dart::ObjectPtr dart::HashSet<dart::UnorderedHashTable<dart::SymbolTraits, 0l> >::InsertNewOrGet<dart::ConcatString>(dart::ConcatString const&) const ../../out/ReleaseTSANX64/../../runtime/vm/hash_table.h:671 (dart+0x1f4fe5c)
    #23 operator() ../../out/ReleaseTSANX64/../../runtime/vm/symbols.cc:393 (dart+0x1f50785)
    #24 dart::LambdaCallable<dart::StringPtr dart::Symbols::NewSymbol<dart::ConcatString>(dart::Thread*, dart::ConcatString const&)::{lambda()#1}>::Call() ../../out/ReleaseTSANX64/../../runtime/vm/isolate.h:141 (dart+0x1f50785)
    #25 dart::IsolateGroup::RunWithStoppedMutatorsCallable(dart::Callable*, dart::Callable*, bool) ??:? (dart+0x1d3e428)
    #26 void dart::IsolateGroup::RunWithStoppedMutators<dart::StringPtr dart::Symbols::NewSymbol<dart::ConcatString>(dart::Thread*, dart::ConcatString const&)::{lambda()#1}>(dart::StringPtr dart::Symbols::NewSymbol<dart::ConcatString>(dart::Thread*, dart::ConcatString const&)::{lambda()#1}, bool) ../../out/ReleaseTSANX64/../../runtime/vm/isolate.h:526 (dart+0x1f4bec8)
    #27 dart::StringPtr dart::Symbols::NewSymbol<dart::ConcatString>(dart::Thread*, dart::ConcatString const&) ../../out/ReleaseTSANX64/../../runtime/vm/symbols.cc:404 (dart+0x1f4bec8)
    #28 dart::Symbols::FromConcat(dart::Thread*, dart::String const&, dart::String const&) ../../out/ReleaseTSANX64/../../runtime/vm/symbols.cc:241 (dart+0x1f4b769)
    #29 dart::Library::PrivateName(dart::String const&) const ../../out/ReleaseTSANX64/../../runtime/vm/object.cc:13236 (dart+0x1d998fb)
    #30 dart::kernel::TranslationHelper::ManglePrivateName(dart::kernel::NameIndex, dart::String*, bool, bool) ../../out/ReleaseTSANX64/../../runtime/vm/compiler/frontend/kernel_translation_helper.cc:806 (dart+0x2593788)
    #31 dart::kernel::KernelReaderHelper::ReadNameAsFieldName() ??:? (dart+0x25a7279)
    #32 dart::kernel::KernelLoader::FinishTopLevelClassLoading(dart::Class const&, dart::Library const&, dart::kernel::LibraryIndex const&) ../../out/ReleaseTSANX64/../../runtime/vm/kernel_loader.cc:1192 (dart+0x1d783a9)
    #33 dart::kernel::KernelLoader::FinishLoading(dart::Class const&) ../../out/ReleaseTSANX64/../../runtime/vm/kernel_loader.cc:1775 (dart+0x1d7c558)
    #34 dart::ClassFinalizer::FinalizeClass(dart::Class const&) ../../out/ReleaseTSANX64/../../runtime/vm/class_finalizer.cc:1117 (dart+0x1c79201)
    #35 dart::ClassFinalizer::LoadClassMembers(dart::Class const&) ../../out/ReleaseTSANX64/../../runtime/vm/class_finalizer.cc:1211 (dart+0x1c7b24b)
    #36 dart::Class::EnsureIsFinalized(dart::Thread*) const ../../out/ReleaseTSANX64/../../runtime/vm/object.cc:4308 (dart+0x1db5314)
    #37 dart::Library::EnsureTopLevelClassIsFinalized() const ../../out/ReleaseTSANX64/../../runtime/vm/object.cc:12327 (dart+0x1dedce8)
    #38 dart::Library::LookupLocalOrReExportObject(dart::String const&) const ../../out/ReleaseTSANX64/../../runtime/vm/object.cc:12340 (dart+0x1df20b7)
    #39 dart::Library::Invoke(dart::String const&, dart::Array const&, dart::Array const&, bool, bool) const ../../out/ReleaseTSANX64/../../runtime/vm/object.cc:12950 (dart+0x1df5ca0)
    #40 Dart_Invoke ../../out/ReleaseTSANX64/../../runtime/vm/dart_api_impl.cc:4786 (dart+0x263c8f9)
    #41 dart::bin::DartUtils::PrepareIOLibrary(_Dart_Handle*) ../../out/ReleaseTSANX64/../../runtime/bin/dartutils.cc:523 (dart+0x1c1eb91)
    #42 dart::bin::DartUtils::PrepareForScriptLoading(bool, bool) ../../out/ReleaseTSANX64/../../runtime/bin/dartutils.cc:595 (dart+0x1c1ef7c)
    #43 dart::bin::VmService::Setup(char const*, long, bool, bool, char const*, bool, bool, bool, bool) ../../out/ReleaseTSANX64/../../runtime/bin/vmservice_impl.cc:132 (dart+0x1a95973)
    #44 dart::bin::CreateAndSetupServiceIsolate(char const*, char const*, Dart_IsolateFlags*, char**, int*) ../../out/ReleaseTSANX64/../../runtime/bin/main.cc:556 (dart+0x1a9acea)
    #45 dart::bin::CreateIsolateGroupAndSetup(char const*, char const*, char const*, char const*, Dart_IsolateFlags*, void*, char**) ../../out/ReleaseTSANX64/../../runtime/bin/main.cc:800 (dart+0x1a9acea)
    #46 dart::RunServiceTask::Run() ../../out/ReleaseTSANX64/../../runtime/vm/service_isolate.cc:356 (dart+0x1f2efc2)
    #47 dart::ThreadPool::WorkerLoop(dart::ThreadPool::Worker*) ../../out/ReleaseTSANX64/../../runtime/vm/thread_pool.cc:158 (dart+0x1f5d6ec)
    #48 dart::ThreadPool::Worker::Main(unsigned long) ../../out/ReleaseTSANX64/../../runtime/vm/thread_pool.cc:323 (dart+0x1f5e0d5)
    #49 dart::ThreadStart(void*) ../../out/ReleaseTSANX64/../../runtime/vm/os_thread_linux.cc:153 (dart+0x1e6c2d2)

SUMMARY: ThreadSanitizer: data race ../../out/ReleaseTSANX64/../../runtime/vm/heap/freelist.h:29 in dart::FreeListElement::set_next(dart::FreeListElement*)
==================
ThreadSanitizer: reported 1 warnings

--- Re-run this test:
python tools/test.py -n dartk-tsan-linux-release-x64 vm/dart_2/entrypoints/jit/polymorphic_optional_this_test/1

/cc @rmacnak-google
@mkustermann mkustermann added area-vm Use area-vm for VM related issues, including code coverage, and the AOT and JIT backends. gardening labels Nov 6, 2020
dart-bot pushed a commit that referenced this issue Nov 30, 2020
@rmacnak-google
[vm, tsan] Consistently modify the object header as a full word.
d3d92e3 
An address containing an object header is sometimes interpreted as a object slot due to array truncation and concurrent marking/sweeping. Always accessing it as a word assuages TSAN that it won't be subject to tearing, and allows for the removal of TSAN suppressions from header access. In turn, this will allow TSAN to verify some acquire/release header accesses.

TEST=tsan
Bug: #44091
Change-Id: I7bf6449bf7dfeabd9db76e4bb37456e3be2d5124
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/172861
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
dart-bot pushed a commit that referenced this issue Dec 2, 2020
@rmacnak-google
[vm, gc] Route most heap reads through LoadPointer.
ca58537 
Provides a good point to apply a read barrier, pointer decompression, or TSAN annotations.

TEST=ci
Bug: #44091
Change-Id: I4e6930264f8ef8399e261fd24fcab78731d6e29c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/174484
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
dart-bot pushed a commit that referenced this issue Dec 7, 2020
@rmacnak-google
[vm, gc] Fix write-write race between array truncation and the concur…
fc4726e 
…rent sweeper.

TEST=tsan
Bug: #44091
Change-Id: I589b31c0b8664d6b10aa5aad5ed6079c55cfba0d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/174843
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area-vm Use area-vm for VM related issues, including code coverage, and the AOT and JIT backends. gardening
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mkustermann @rmacnak-google