Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI][C++] arrow-dataset-dataset-writer-test in the ASAN UBSAN job failed #38817

Open
kou opened this issue Nov 21, 2023 · 5 comments
Open

[CI][C++] arrow-dataset-dataset-writer-test in the ASAN UBSAN job failed #38817

kou opened this issue Nov 21, 2023 · 5 comments
Labels
Component: C++ Component: Continuous Integration Type: bug

Comments

@kou
Copy link
Member

kou commented Nov 21, 2023 •
edited

Describe the bug, including details regarding any error messages, version, and platform.

Since 7df1cdd but the change isn't related to this failure because it doesn't touch the dataset module.

https://github.com/apache/arrow/actions/runs/6919998294/job/18823986112#step:6:3047

53/92 Test #56: arrow-dataset-dataset-writer-test ............***Failed    1.88 sec
Running arrow-dataset-dataset-writer-test, redirecting output into /build/cpp/build/test-logs/arrow-dataset-dataset-writer-test.txt (attempt 1/1)
Running main() from ./googletest/src/gtest_main.cc
[==========] Running 20 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 20 tests from DatasetWriterTestFixture
[ RUN      ] DatasetWriterTestFixture.Basic
[       OK ] DatasetWriterTestFixture.Basic (13 ms)
[ RUN      ] DatasetWriterTestFixture.BasicFilePrefix
[       OK ] DatasetWriterTestFixture.BasicFilePrefix (3 ms)
[ RUN      ] DatasetWriterTestFixture.BasicFileDirectoryPrefix
[       OK ] DatasetWriterTestFixture.BasicFileDirectoryPrefix (3 ms)
[ RUN      ] DatasetWriterTestFixture.DirectoryCreateFails
[       OK ] DatasetWriterTestFixture.DirectoryCreateFails (2 ms)
[ RUN      ] DatasetWriterTestFixture.MaxRowsOneWrite
[       OK ] DatasetWriterTestFixture.MaxRowsOneWrite (12 ms)
[ RUN      ] DatasetWriterTestFixture.MaxRowsOneWriteWithFunctor
[       OK ] DatasetWriterTestFixture.MaxRowsOneWriteWithFunctor (8 ms)
[ RUN      ] DatasetWriterTestFixture.MaxRowsOneWriteWithBrokenFunctor
[       OK ] DatasetWriterTestFixture.MaxRowsOneWriteWithBrokenFunctor (2 ms)
[ RUN      ] DatasetWriterTestFixture.MaxRowsManyWrites
[       OK ] DatasetWriterTestFixture.MaxRowsManyWrites (11 ms)
[ RUN      ] DatasetWriterTestFixture.MinRowGroup
[       OK ] DatasetWriterTestFixture.MinRowGroup (9 ms)
[ RUN      ] DatasetWriterTestFixture.MaxRowGroup
[       OK ] DatasetWriterTestFixture.MaxRowGroup (8 ms)
[ RUN      ] DatasetWriterTestFixture.MinAndMaxRowGroup
[       OK ] DatasetWriterTestFixture.MinAndMaxRowGroup (8 ms)
[ RUN      ] DatasetWriterTestFixture.MinRowGroupBackpressure
[       OK ] DatasetWriterTestFixture.MinRowGroupBackpressure (34 ms)
[ RUN      ] DatasetWriterTestFixture.ConcurrentWritesSameFile
[       OK ] DatasetWriterTestFixture.ConcurrentWritesSameFile (13 ms)
[ RUN      ] DatasetWriterTestFixture.ConcurrentWritesDifferentFiles
[       OK ] DatasetWriterTestFixture.ConcurrentWritesDifferentFiles (18 ms)
[ RUN      ] DatasetWriterTestFixture.MaxOpenFiles
[       OK ] DatasetWriterTestFixture.MaxOpenFiles (12 ms)
[ RUN      ] DatasetWriterTestFixture.NoExistingDirectory
[       OK ] DatasetWriterTestFixture.NoExistingDirectory (4 ms)
[ RUN      ] DatasetWriterTestFixture.DeleteExistingData
[       OK ] DatasetWriterTestFixture.DeleteExistingData (4 ms)
[ RUN      ] DatasetWriterTestFixture.PartitionedDeleteExistingData
[       OK ] DatasetWriterTestFixture.PartitionedDeleteExistingData (4 ms)
[ RUN      ] DatasetWriterTestFixture.LeaveExistingData
[       OK ] DatasetWriterTestFixture.LeaveExistingData (4 ms)
[ RUN      ] DatasetWriterTestFixture.ErrOnExistingData
[       OK ] DatasetWriterTestFixture.ErrOnExistingData (1 ms)
[----------] 20 tests from DatasetWriterTestFixture (181 ms total)

[----------] Global test environment tear-down
[==========] 20 tests from 1 test suite ran. (182 ms total)
[  PASSED  ] 20 tests.

=================================================================
==9995==ERROR: LeakSanitizer: detected memory leaks
    #15 0x7fa79e9a497f in arrow::dataset::internal::(anonymous namespace)::DatasetWriterFileQueue::Start(arrow::util::AsyncTaskScheduler*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::'lambda'()::operator()() const::'lambda'()::operator()() const /arrow/cpp/src/arrow/dataset/dataset_writer.cc:149:13
    #16 0x7fa79e9a459b in std::enable_if<((!(std::is_void<arrow::Status>::value)) && (!(is_future<arrow::Status>::value))) && ((!(arrow::Future<arrow::internal::Empty>::is_empty)) || (std::is_same<arrow::Status, arrow::Status>::value)), void>::type arrow::detail::ContinueFuture::operator()<arrow::dataset::internal::(anonymous namespace)::DatasetWriterFileQueue::Start(arrow::util::AsyncTaskScheduler*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::'lambda'()::operator()() const::'lambda'()&, arrow::Status, arrow::Future<arrow::internal::Empty> >(arrow::Future<arrow::internal::Empty>, arrow::dataset::internal::(anonymous namespace)::DatasetWriterFileQueue::Start(arrow::util::AsyncTaskScheduler*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::'lambda'()::operator()() const::'lambda'()&) const /arrow/cpp/src/arrow/util/future.h:150:23
    #17 0x7fa79e9a4271 in void std::__invoke_impl<void, arrow::detail::ContinueFuture&, arrow::Future<arrow::internal::Empty>&, arrow::dataset::internal::(anonymous namespace)::DatasetWriterFileQueue::Start(arrow::util::AsyncTaskScheduler*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::'lambda'()::operator()() const::'lambda'()&>(std::__invoke_other, arrow::detail::ContinueFuture&, arrow::Future<arrow::internal::Empty>&, arrow::dataset::internal::(anonymous namespace)::DatasetWriterFileQueue::Start(arrow::util::AsyncTaskScheduler*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::'lambda'()::operator()() const::'lambda'()&) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14
    #18 0x7fa79e9a3d75 in std::__invoke_result<arrow::detail::ContinueFuture&, arrow::Future<arrow::internal::Empty>&, arrow::dataset::internal::(anonymous namespace)::DatasetWriterFileQueue::Start(arrow::util::AsyncTaskScheduler*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::'lambda'()::operator()() const::'lambda'()&>::type std::__invoke<arrow::detail::ContinueFuture&, arrow::Future<arrow::internal::Empty>&, arrow::dataset::internal::(anonymous namespace)::DatasetWriterFileQueue::Start(arrow::util::AsyncTaskScheduler*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::'lambda'()::operator()() const::'lambda'()&>(arrow::detail::ContinueFuture&, arrow::Future<arrow::internal::Empty>&, arrow::dataset::internal::(anonymous namespace)::DatasetWriterFileQueue::Start(arrow::util::AsyncTaskScheduler*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::'lambda'()::operator()() const::'lambda'()&) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:96:14
    #19 0x7fa79e9a3b5f in void std::_Bind<arrow::detail::ContinueFuture (arrow::Future<arrow::internal::Empty>, arrow::dataset::internal::(anonymous namespace)::DatasetWriterFileQueue::Start(arrow::util::AsyncTaskScheduler*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::'lambda'()::operator()() const::'lambda'())>::__call<void, 0ul, 1ul>(std::tuple<>&&, std::_Index_tuple<0ul, 1ul>) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/functional:420:11
    #20 0x7fa79e9a3738 in void std::_Bind<arrow::detail::ContinueFuture (arrow::Future<arrow::internal::Empty>, arrow::dataset::internal::(anonymous namespace)::DatasetWriterFileQueue::Start(arrow::util::AsyncTaskScheduler*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::'lambda'()::operator()() const::'lambda'())>::operator()<void>() /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/functional:503:17
    #21 0x7fa79e9a3606 in arrow::internal::FnOnce<void ()>::FnImpl<std::_Bind<arrow::detail::ContinueFuture (arrow::Future<arrow::internal::Empty>, arrow::dataset::internal::(anonymous namespace)::DatasetWriterFileQueue::Start(arrow::util::AsyncTaskScheduler*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::'lambda'()::operator()() const::'lambda'())> >::invoke() /arrow/cpp/src/arrow/util/functional.h:152:42
    #22 0x7fa7843c42e6 in arrow::internal::FnOnce<void ()>::operator()() && /arrow/cpp/src/arrow/util/functional.h:140:17
    #23 0x7fa7843f14b7 in arrow::internal::ThreadPool::SpawnReal(arrow::internal::TaskHints, arrow::internal::FnOnce<void ()>, arrow::StopToken, arrow::internal::FnOnce<void (arrow::Status const&)>&&)::$_0::operator()() /arrow/cpp/src/arrow/util/thread_pool.cc:634:9
    #24 0x7fa7843f12b6 in arrow::internal::FnOnce<void ()>::FnImpl<arrow::internal::ThreadPool::SpawnReal(arrow::internal::TaskHints, arrow::internal::FnOnce<void ()>, arrow::StopToken, arrow::internal::FnOnce<void (arrow::Status const&)>&&)::$_0>::invoke() /arrow/cpp/src/arrow/util/functional.h:152:42
    #25 0x7fa7843c42e6 in arrow::internal::FnOnce<void ()>::operator()() && /arrow/cpp/src/arrow/util/functional.h:140:17
    #26 0x7fa7843ea9c4 in arrow::internal::WorkerLoop(std::shared_ptr<arrow::internal::ThreadPool::State>, std::_List_iterator<std::thread>) /arrow/cpp/src/arrow/util/thread_pool.cc:457:11
    #27 0x7fa7843e946b in arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::$_5::operator()() const /arrow/cpp/src/arrow/util/thread_pool.cc:618:7
    #28 0x7fa7843e9293 in void std::__invoke_impl<void, arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::$_5>(std::__invoke_other, arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::$_5&&) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14
    #29 0x7fa7843e9083 in std::__invoke_result<arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::$_5>::type std::__invoke<arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::$_5>(arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::$_5&&) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:96:14

Indirect leak of 17 byte(s) in 1 object(s) allocated from:
    #0 0x55e00f5a4f1d in operator new(unsigned long) (/build/cpp/debug/arrow-dataset-dataset-writer-test+0x398f1d) (BuildId: 176377c2697ecb6a7274ce334f7231a363068218)
    #1 0x7fa7990820e2 in void std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct<char*>(char*, char*, std::forward_iterator_tag) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.tcc:219:14
    #2 0x7fa78bd03cb4 in arrow::fs::internal::MockFileSystem::Impl::OpenOutputStream(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool, std::shared_ptr<arrow::KeyValueMetadata const> const&) /arrow/cpp/src/arrow/filesystem/mockfs.cc:398:44
    #3 0x7fa78bd02c58 in arrow::fs::internal::MockFileSystem::OpenOutputStream(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::shared_ptr<arrow::KeyValueMetadata const> const&) /arrow/cpp/src/arrow/filesystem/mockfs.cc:735:17
    #4 0x7fa78bb45b27 in arrow::fs::FileSystem::OpenOutputStream(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) /arrow/cpp/src/arrow/filesystem/filesystem.cc:250:10
    #5 0x7fa79e9a51d6 in arrow::dataset::internal::(anonymous namespace)::OpenWriter(arrow::dataset::FileSystemDatasetWriteOptions const&, std::shared_ptr<arrow::Schema>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) /arrow/cpp/src/arrow/dataset/dataset_writer.cc:127:3
    #6 0x7fa79e9a497f in arrow::dataset::internal::(anonymous namespace)::DatasetWriterFileQueue::Start(arrow::util::AsyncTaskScheduler*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::'lambda'()::operator()() const::'lambda'()::operator()() const /arrow/cpp/src/arrow/dataset/dataset_writer.cc:149:13
    #7 0x7fa79e9a459b in std::enable_if<((!(std::is_void<arrow::Status>::value)) && (!(is_future<arrow::Status>::value))) && ((!(arrow::Future<arrow::internal::Empty>::is_empty)) || (std::is_same<arrow::Status, arrow::Status>::value)), void>::type arrow::detail::ContinueFuture::operator()<arrow::dataset::internal::(anonymous namespace)::DatasetWriterFileQueue::Start(arrow::util::AsyncTaskScheduler*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::'lambda'()::operator()() const::'lambda'()&, arrow::Status, arrow::Future<arrow::internal::Empty> >(arrow::Future<arrow::internal::Empty>, arrow::dataset::internal::(anonymous namespace)::DatasetWriterFileQueue::Start(arrow::util::AsyncTaskScheduler*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::'lambda'()::operator()() const::'lambda'()&) const /arrow/cpp/src/arrow/util/future.h:150:23
    #8 0x7fa79e9a4271 in void std::__invoke_impl<void, arrow::detail::ContinueFuture&, arrow::Future<arrow::internal::Empty>&, arrow::dataset::internal::(anonymous namespace)::DatasetWriterFileQueue::Start(arrow::util::AsyncTaskScheduler*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::'lambda'()::operator()() const::'lambda'()&>(std::__invoke_other, arrow::detail::ContinueFuture&, arrow::Future<arrow::internal::Empty>&, arrow::dataset::internal::(anonymous namespace)::DatasetWriterFileQueue::Start(arrow::util::AsyncTaskScheduler*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::'lambda'()::operator()() const::'lambda'()&) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14
    #9 0x7fa79e9a3d75 in std::__invoke_result<arrow::detail::ContinueFuture&, arrow::Future<arrow::internal::Empty>&, arrow::dataset::internal::(anonymous namespace)::DatasetWriterFileQueue::Start(arrow::util::AsyncTaskScheduler*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::'lambda'()::operator()() const::'lambda'()&>::type std::__invoke<arrow::detail::ContinueFuture&, arrow::Future<arrow::internal::Empty>&, arrow::dataset::internal::(anonymous namespace)::DatasetWriterFileQueue::Start(arrow::util::AsyncTaskScheduler*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::'lambda'()::operator()() const::'lambda'()&>(arrow::detail::ContinueFuture&, arrow::Future<arrow::internal::Empty>&, arrow::dataset::internal::(anonymous namespace)::DatasetWriterFileQueue::Start(arrow::util::AsyncTaskScheduler*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::'lambda'()::operator()() const::'lambda'()&) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:96:14
    #10 0x7fa79e9a3b5f in void std::_Bind<arrow::detail::ContinueFuture (arrow::Future<arrow::internal::Empty>, arrow::dataset::internal::(anonymous namespace)::DatasetWriterFileQueue::Start(arrow::util::AsyncTaskScheduler*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::'lambda'()::operator()() const::'lambda'())>::__call<void, 0ul, 1ul>(std::tuple<>&&, std::_Index_tuple<0ul, 1ul>) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/functional:420:11
    #11 0x7fa79e9a3738 in void std::_Bind<arrow::detail::ContinueFuture (arrow::Future<arrow::internal::Empty>, arrow::dataset::internal::(anonymous namespace)::DatasetWriterFileQueue::Start(arrow::util::AsyncTaskScheduler*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::'lambda'()::operator()() const::'lambda'())>::operator()<void>() /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/functional:503:17
    #12 0x7fa79e9a3606 in arrow::internal::FnOnce<void ()>::FnImpl<std::_Bind<arrow::detail::ContinueFuture (arrow::Future<arrow::internal::Empty>, arrow::dataset::internal::(anonymous namespace)::DatasetWriterFileQueue::Start(arrow::util::AsyncTaskScheduler*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::'lambda'()::operator()() const::'lambda'())> >::invoke() /arrow/cpp/src/arrow/util/functional.h:152:42
    #13 0x7fa7843c42e6 in arrow::internal::FnOnce<void ()>::operator()() && /arrow/cpp/src/arrow/util/functional.h:140:17
    #14 0x7fa7843f14b7 in arrow::internal::ThreadPool::SpawnReal(arrow::internal::TaskHints, arrow::internal::FnOnce<void ()>, arrow::StopToken, arrow::internal::FnOnce<void (arrow::Status const&)>&&)::$_0::operator()() /arrow/cpp/src/arrow/util/thread_pool.cc:634:9
    #15 0x7fa7843f12b6 in arrow::internal::FnOnce<void ()>::FnImpl<arrow::internal::ThreadPool::SpawnReal(arrow::internal::TaskHints, arrow::internal::FnOnce<void ()>, arrow::StopToken, arrow::internal::FnOnce<void (arrow::Status const&)>&&)::$_0>::invoke() /arrow/cpp/src/arrow/util/functional.h:152:42
    #16 0x7fa7843c42e6 in arrow::internal::FnOnce<void ()>::operator()() && /arrow/cpp/src/arrow/util/functional.h:140:17
    #17 0x7fa7843ea9c4 in arrow::internal::WorkerLoop(std::shared_ptr<arrow::internal::ThreadPool::State>, std::_List_iterator<std::thread>) /arrow/cpp/src/arrow/util/thread_pool.cc:457:11
    #18 0x7fa7843e946b in arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::$_5::operator()() const /arrow/cpp/src/arrow/util/thread_pool.cc:618:7
    #19 0x7fa7843e9293 in void std::__invoke_impl<void, arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::$_5>(std::__invoke_other, arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::$_5&&) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14
    #20 0x7fa7843e9083 in std::__invoke_result<arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::$_5>::type std::__invoke<arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::$_5>(arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::$_5&&) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:96:14
    #21 0x7fa7843e8fd1 in void std::thread::_Invoker<std::tuple<arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::$_5> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:259:13
    #22 0x7fa7843e8ea0 in std::thread::_Invoker<std::tuple<arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::$_5> >::operator()() /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:266:11
    #23 0x7fa7843e8373 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<arrow::internal::ThreadPool::LaunchWorkersUnlocked(int)::$_5> > >::_M_run() /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:211:13
    #5 0x7fa77805e252 in

Indirect leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x55e00f5a4f1d in operator new(unsigned long) (/build/cpp/debug/arrow-dataset-dataset-writer-test+0x398f1d) (BuildId: 176377c2697ecb6a7274ce334f7231a363068218)
    #1 0x55e00f6689a0 in __gnu_cxx::new_allocator<std::shared_ptr<arrow::Field> >::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/ext/new_allocator.h:127:27
    #2 0x55e00f66882c in std::allocator_traits<std::allocator<std::shared_ptr<arrow::Field> > >::allocate(std::allocator<std::shared_ptr<arrow::Field> >&, unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:464:20
    #3 0x55e00f6684c4 in std::_Vector_base<std::shared_ptr<arrow::Field>, std::allocator<std::shared_ptr<arrow::Field> > >::_M_allocate(unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:346:20
    #4 0x55e00f667a2e in void std::vector<std::shared_ptr<arrow::Field>, std::allocator<std::shared_ptr<arrow::Field> > >::_M_range_initialize<std::shared_ptr<arrow::Field> const*>(std::shared_ptr<arrow::Field> const*, std::shared_ptr<arrow::Field> const*, std::forward_iterator_tag) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:1582:14
    #5 0x55e00f5b9097 in std::vector<std::shared_ptr<arrow::Field>, std::allocator<std::shared_ptr<arrow::Field> > >::vector(std::initializer_list<std::shared_ptr<arrow::Field> >, std::allocator<std::shared_ptr<arrow::Field> > const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:629:2
    #6 0x55e00f7677ab in arrow::dataset::internal::DatasetWriterTestFixture::SetUp() /arrow/cpp/src/arrow/dataset/dataset_writer_test.cc:73:22
    #7 0x7fa7994c98ee in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) (/usr/local/lib/libarrow_testing.so.1500+0xd608ee) (BuildId: 4068fe9148ba0adf20466726aadbd4ac18ef4795)

SUMMARY: AddressSanitizer: 3603 byte(s) leaked in 43 allocation(s).
Test had memory leaks. Editing XML
sed: can't read /build/cpp/build/test-logs/arrow-dataset-dataset-writer-test.xml: No such file or directory
mv: cannot stat '/build/cpp/build/test-logs/arrow-dataset-dataset-writer-test.xml.bak': No such file or directory
/build/cpp/src/arrow/dataset

Component(s)

C++, Continuous Integration
@kou kou added the Type: bug label Nov 21, 2023
@mapleFU
Copy link
Member

mapleFU commented Nov 21, 2023

I believe dataset-writer has many bugs orz... I'll try to take a look

@mapleFU
Copy link
Member

mapleFU commented Nov 21, 2023 •
edited

Could you re-produce this fail? I'm compiling this on My MacOS but I'm not able to reproduce it...

But I guess the cause is from DatasetWriterTestFixture.ErrOnExistingData didn't graceful shutdown, I'll dive into it later.

@kou
Copy link
Member Author

kou commented Nov 21, 2023

I could reproduce this by UBUNTU=22.04 archery docker run ubuntu-cpp-sanitizer. Could you try it?

@mapleFU
Copy link
Member

mapleFU commented Nov 22, 2023

Do you using debug build or release build? I use RelWithDebInfo (-O2) with asan/ubsan, it doesn't trigger the leak

@kou
Copy link
Member Author

kou commented Nov 22, 2023

UBUNTU=22.04 archery docker run ubuntu-cpp-sanitizer doesn't change the default. (It just changes the Ubuntu version.)
So build type is debug like the CI job: https://github.com/apache/arrow/actions/runs/6919998294/job/18823986112#step:6:486

-- Build Type: DEBUG

If you can't reproduce this on your local machine, I'll fix this later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Component: C++ Component: Continuous Integration Type: bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@kou @mapleFU