Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[C++] Data race in GroupByNode found by ThreadSanitizer #29247

Closed
asfimport opened this issue Aug 11, 2021 · 2 comments
Closed

[C++] Data race in GroupByNode found by ThreadSanitizer #29247

asfimport opened this issue Aug 11, 2021 · 2 comments
Assignees
@lidavidm
Labels
Component: C++ Type: bug
Milestone
6.0.0

Comments

@asfimport
Copy link

https://github.com/ursacomputing/crossbow/runs/3297591336


[ RUN      ] GroupBy.CountOnly
==================
WARNING: ThreadSanitizer: data race (pid=11155)
  Write of size 8 at 0x7b5000000978 by main thread:
    #0 operator delete(void*) <null> (arrow-compute-aggregate-test+0x1be4ae)
    #1 arrow::compute::(anonymous namespace)::GroupByNode::~GroupByNode() /arrow/cpp/src/arrow/compute/exec/aggregate_node.cc:254:8 (libarrow.so.600+0x162985b)
    #2 std::default_delete<arrow::compute::ExecNode>::operator()(arrow::compute::ExecNode*) const /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2 (libarrow_testing.so.600+0x389716)
    #3 std::unique_ptr<arrow::compute::ExecNode, std::default_delete<arrow::compute::ExecNode> >::~unique_ptr() /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4 (libarrow_testing.so.600+0x3851d3)
    #4 void std::_Destroy<std::unique_ptr<arrow::compute::ExecNode, std::default_delete<arrow::compute::ExecNode> > >(std::unique_ptr<arrow::compute::ExecNode, std::default_delete<arrow::compute::ExecNode> >*) /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/stl_construct.h:98:19 (libarrow.so.600+0x164dfd8)
    #5 void std::_Destroy_aux<false>::__destroy<std::unique_ptr<arrow::compute::ExecNode, std::default_delete<arrow::compute::ExecNode> >*>(std::unique_ptr<arrow::compute::ExecNode, std::default_delete<arrow::compute::ExecNode> >*, std::unique_ptr<arrow::compute::ExecNode, std::default_delete<arrow::compute::ExecNode> >*) /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/stl_construct.h:108:6 (libarrow.so.600+0x164df8a)
    #6 void std::_Destroy<std::unique_ptr<arrow::compute::ExecNode, std::default_delete<arrow::compute::ExecNode> >*>(std::unique_ptr<arrow::compute::ExecNode, std::default_delete<arrow::compute::ExecNode> >*, std::unique_ptr<arrow::compute::ExecNode, std::default_delete<arrow::compute::ExecNode> >*) /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/stl_construct.h:136:7 (libarrow.so.600+0x164df18)
    #7 void std::_Destroy<std::unique_ptr<arrow::compute::ExecNode, std::default_delete<arrow::compute::ExecNode> >*, std::unique_ptr<arrow::compute::ExecNode, std::default_delete<arrow::compute::ExecNode> > >(std::unique_ptr<arrow::compute::ExecNode, std::default_delete<arrow::compute::ExecNode> >*, std::unique_ptr<arrow::compute::ExecNode, std::default_delete<arrow::compute::ExecNode> >*, std::allocator<std::unique_ptr<arrow::compute::ExecNode, std::default_delete<arrow::compute::ExecNode> > >&) /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/stl_construct.h:206:7 (libarrow.so.600+0x164ddc4)
    #8 std::vector<std::unique_ptr<arrow::compute::ExecNode, std::default_delete<arrow::compute::ExecNode> >, std::allocator<std::unique_ptr<arrow::compute::ExecNode, std::default_delete<arrow::compute::ExecNode> > > >::~vector() /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/stl_vector.h:677:2 (libarrow.so.600+0x164dd3a)
    #9 arrow::compute::(anonymous namespace)::ExecPlanImpl::~ExecPlanImpl() /arrow/cpp/src/arrow/compute/exec/exec_plan.cc:52:3 (libarrow.so.600+0x1648da2)
    #10 arrow::compute::(anonymous namespace)::ExecPlanImpl::~ExecPlanImpl() /arrow/cpp/src/arrow/compute/exec/exec_plan.cc:46:28 (libarrow.so.600+0x1648e6f)
    #11 std::_Sp_counted_ptr<arrow::compute::(anonymous namespace)::ExecPlanImpl*, (__gnu_cxx::_Lock_policy)2>::_M_dispose() /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/shared_ptr_base.h:377:9 (libarrow.so.600+0x164a048)
    #12 std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/shared_ptr_base.h:155:6 (arrow-compute-aggregate-test+0x202b28)
    #13 std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/shared_ptr_base.h:730:11 (arrow-compute-aggregate-test+0x202aa8)
    #14 std::__shared_ptr<arrow::compute::ExecPlan, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/shared_ptr_base.h:1169:31 (arrow-compute-aggregate-test+0x62beb2)
    #15 std::shared_ptr<arrow::compute::ExecPlan>::~shared_ptr() /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/shared_ptr.h:103:11 (arrow-compute-aggregate-test+0x606eeb)
    #16 arrow::compute::(anonymous namespace)::GroupByUsingExecPlan(std::vector<arrow::Datum, std::allocator<arrow::Datum> > const&, std::vector<arrow::Datum, std::allocator<arrow::Datum> > const&, std::vector<arrow::compute::internal::Aggregate, std::allocator<arrow::compute::internal::Aggregate> > const&, bool, arrow::compute::ExecContext*) /arrow/cpp/src/arrow/compute/kernels/hash_aggregate_test.cc:207:1 (arrow-compute-aggregate-test+0x5effd7)
    #17 arrow::compute::(anonymous namespace)::GroupByTest(std::vector<arrow::Datum, std::allocator<arrow::Datum> > const&, std::vector<arrow::Datum, std::allocator<arrow::Datum> > const&, std::vector<arrow::compute::internal::Aggregate, std::allocator<arrow::compute::internal::Aggregate> > const&, bool, bool) /arrow/cpp/src/arrow/compute/kernels/hash_aggregate_test.cc:234:12 (arrow-compute-aggregate-test+0x5c3494)
    #18 arrow::compute::GroupBy_CountOnly_Test::TestBody() /arrow/cpp/src/arrow/compute/kernels/hash_aggregate_test.cc:675:7 (arrow-compute-aggregate-test+0x5c1615)
    #19 void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /build/cpp/googletest_ep-prefix/src/googletest_ep/googletest/src/gtest.cc:2433:10 (libgtestd.so+0xabc73)

  Previous atomic read of size 4 at 0x7b5000000978 by thread T1:
    #0 __tsan_atomic32_load <null> (arrow-compute-aggregate-test+0x1792ec)
    #1 std::__atomic_base<int>::load(std::memory_order) const /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/atomic_base.h:419:9 (libarrow.so.600+0x163a52f)
    #2 arrow::compute::AtomicCounter::Increment() /arrow/cpp/src/arrow/compute/exec/util.h:207:25 (libarrow.so.600+0x163a52f)
    #3 arrow::compute::(anonymous namespace)::GroupByNode::OutputNthBatch(int) /arrow/cpp/src/arrow/compute/exec/aggregate_node.cc:436:25 (libarrow.so.600+0x162dc14)
    #4 arrow::compute::(anonymous namespace)::GroupByNode::OutputResult()::'lambda'()::operator()() const /arrow/cpp/src/arrow/compute/exec/aggregate_node.cc:454:9 (libarrow.so.600+0x162e116)
    #5 arrow::internal::FnOnce<void ()>::FnImpl<arrow::compute::(anonymous namespace)::GroupByNode::OutputResult()::'lambda'()>::invoke() /arrow/cpp/src/arrow/util/functional.h:152:42 (libarrow.so.600+0x162e097)
    #6 arrow::internal::FnOnce<void ()>::operator()() && /arrow/cpp/src/arrow/util/functional.h:140:17 (libarrow.so.600+0x12c3d00)
    #7 arrow::internal::WorkerLoop(std::shared_ptr<arrow::internal::ThreadPool::State>, std::_List_iterator<std::thread>) /arrow/cpp/src/arrow/util/thread_pool.cc:176:11 (libarrow.so.600+0x12c30fb)
    #8 arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::$_2::operator()() const /arrow/cpp/src/arrow/util/thread_pool.cc:336:7 (libarrow.so.600+0x12c2d26)
    #9 void std::__invoke_impl<void, arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::$_2>(std::__invoke_other, arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::$_2&&) /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/invoke.h:60:14 (libarrow.so.600+0x12c2c50)
    #10 std::__invoke_result<arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::$_2>::type std::__invoke<arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::$_2>(arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::$_2&&) /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/invoke.h:95:14 (libarrow.so.600+0x12c2b60)
    #11 void std::thread::_Invoker<std::tuple<arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::$_2> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/thread:244:13 (libarrow.so.600+0x12c2b08)
    #12 std::thread::_Invoker<std::tuple<arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::$_2> >::operator()() /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/thread:251:11 (libarrow.so.600+0x12c2aa8)
    #13 std::thread::_State_impl<std::thread::_Invoker<std::tuple<arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::$_2> > >::_M_run() /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/thread:195:13 (libarrow.so.600+0x12c27ef)
    #14 <null> <null> (libstdc++.so.6+0xd6de3)

  Thread T1 (tid=11157, running) created by main thread at:
    #0 pthread_create <null> (arrow-compute-aggregate-test+0x12f21b)
    #1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xd70a8)
    #2 arrow::internal::ThreadPool::LaunchWorkersUnlocked(int) /arrow/cpp/src/arrow/util/thread_pool.cc:334:11 (libarrow.so.600+0x12bd945)
    #3 arrow::internal::ThreadPool::SpawnReal(arrow::internal::TaskHints, arrow::internal::FnOnce<void ()>, arrow::StopToken, arrow::internal::FnOnce<void (arrow::Status const&)>&&) /arrow/cpp/src/arrow/util/thread_pool.cc:354:7 (libarrow.so.600+0x12be1a4)
    #4 arrow::Status arrow::internal::Executor::Spawn<arrow::internal::(anonymous namespace)::ThreadedTaskGroup::AppendReal(arrow::internal::FnOnce<arrow::Status ()>)::Callable>(arrow::internal::(anonymous namespace)::ThreadedTaskGroup::AppendReal(arrow::internal::FnOnce<arrow::Status ()>)::Callable&&) /arrow/cpp/src/arrow/util/thread_pool.h:86:12 (libarrow.so.600+0x12a61f1)
    #5 arrow::internal::(anonymous namespace)::ThreadedTaskGroup::AppendReal(arrow::internal::FnOnce<arrow::Status ()>) /arrow/cpp/src/arrow/util/task_group.cc:125:22 (libarrow.so.600+0x12a5c0c)
    #6 void arrow::internal::TaskGroup::Append<arrow::compute::internal::GroupBy(std::vector<arrow::Datum, std::allocator<arrow::Datum> > const&, std::vector<arrow::Datum, std::allocator<arrow::Datum> > const&, std::vector<arrow::compute::internal::Aggregate, std::allocator<arrow::compute::internal::Aggregate> > const&, bool, arrow::compute::ExecContext*)::$_2>(arrow::compute::internal::GroupBy(std::vector<arrow::Datum, std::allocator<arrow::Datum> > const&, std::vector<arrow::Datum, std::allocator<arrow::Datum> > const&, std::vector<arrow::compute::internal::Aggregate, std::allocator<arrow::compute::internal::Aggregate> > const&, bool, arrow::compute::ExecContext*)::$_2&&) /arrow/cpp/src/arrow/util/task_group.h:59:12 (libarrow.so.600+0x18ac8af)
    #7 arrow::compute::internal::GroupBy(std::vector<arrow::Datum, std::allocator<arrow::Datum> > const&, std::vector<arrow::Datum, std::allocator<arrow::Datum> > const&, std::vector<arrow::compute::internal::Aggregate, std::allocator<arrow::compute::internal::Aggregate> > const&, bool, arrow::compute::ExecContext*) /arrow/cpp/src/arrow/compute/kernels/hash_aggregate.cc:1959:17 (libarrow.so.600+0x18aacd8)
    #8 arrow::compute::(anonymous namespace)::GroupByTest(std::vector<arrow::Datum, std::allocator<arrow::Datum> > const&, std::vector<arrow::Datum, std::allocator<arrow::Datum> > const&, std::vector<arrow::compute::internal::Aggregate, std::allocator<arrow::compute::internal::Aggregate> > const&, bool, bool) /arrow/cpp/src/arrow/compute/kernels/hash_aggregate_test.cc:237:12 (arrow-compute-aggregate-test+0x5c3536)
    #9 arrow::compute::GroupBy_CountOnly_Test::TestBody() /arrow/cpp/src/arrow/compute/kernels/hash_aggregate_test.cc:675:7 (arrow-compute-aggregate-test+0x5c1615)
    #10 void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /build/cpp/googletest_ep-prefix/src/googletest_ep/googletest/src/gtest.cc:2433:10 (libgtestd.so+0xabc73)

SUMMARY: ThreadSanitizer: data race (/build/cpp/debug/arrow-compute-aggregate-test+0x1be4ae) in operator delete(void*)
==================
[       OK ] GroupBy.CountOnly (952 ms)

Reporter: David Li / @lidavidm
Assignee: David Li / @lidavidm

PRs and other links:

Note: This issue was originally created as ARROW-13605. Please see the migration documentation for further details.
@asfimport
Copy link
Author

David Li / @lidavidm:
Seems similar to the case in google/sanitizers#602

@asfimport
Copy link
Author

Antoine Pitrou / @pitrou:
Issue resolved by pull request 10919
#10919

@asfimport asfimport added this to the 6.0.0 milestone Jan 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@lidavidm lidavidm

Labels
Component: C++ Type: bug
Projects
None yet
Milestone
6.0.0
Development

No branches or pull requests
2 participants
@lidavidm @asfimport