-
Notifications
You must be signed in to change notification settings - Fork 411
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
test(bindings/c): Implement valgrind check in test and ci #2653
Conversation
Bravo! It seems that valgrind helps us catch a memory leak. cc @Ji-Xinyou, would you like to take a look and have a fix? ==6590== 16 bytes in 1 blocks are definitely lost in loss record 1 of 1
==6590== at 0x4848899: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==6590== by 0x49A653E: alloc::alloc::Global::alloc_impl (alloc.rs:102)
==6590== by 0x49A63F3: alloc::alloc::exchange_malloc (alloc.rs:245)
==6590== by 0x49A52F5: opendal_operator_blocking_read (boxed.rs:217)
==6590== by 0x1159A3: OpendalBddTest_FeatureTest_Test::TestBody() (in /home/runner/work/incubator-opendal/incubator-opendal/bindings/c/build/bdd)
==6590== by 0x149DBE: void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) (in /home/runner/work/incubator-opendal/incubator-opendal/bindings/c/build/bdd)
==6590== by 0x13DB55: testing::Test::Run() (in /home/runner/work/incubator-opendal/incubator-opendal/bindings/c/build/bdd)
==6590== by 0x13DCD4: testing::TestInfo::Run() (in /home/runner/work/incubator-opendal/incubator-opendal/bindings/c/build/bdd)
==6590== by 0x13E288: testing::TestSuite::Run() (in /home/runner/work/incubator-opendal/incubator-opendal/bindings/c/build/bdd)
==6590== by 0x13E98E: testing::internal::UnitTestImpl::RunAllTests() (in /home/runner/work/incubator-opendal/incubator-opendal/bindings/c/build/bdd)
==6590== by 0x14A[38](https://github.com/apache/incubator-opendal/actions/runs/5575185496/jobs/10184759670?pr=2653#step:7:39)6: bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) (in /home/runner/work/incubator-opendal/incubator-opendal/bindings/c/build/bdd)
==6590== by 0x13DD9B: testing::UnitTest::Run() (in /home/runner/work/incubator-opendal/incubator-opendal/bindings/c/build/bdd) |
This PR mostly LGTM! We can merge this PR after the memory leak fixed (in other PR). |
Definitely! I would take a look as soon as I get my hands on my machine. Thanks for the work🥰 |
There are also leaks in list test: valgrind --error-exitcode=1 --leak-check=full -- ./build/list
==309525== Memcheck, a memory error detector
==309525== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==309525== Using Valgrind-3.21.0 and LibVEX; rerun with -h for copyright info
==309525== Command: ./build/list
==309525==
[==========] Running 3 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 3 tests from OpendalListTest
[ RUN ] OpendalListTest.ListDirTest
==309525== Invalid read of size 1
==309525== at 0x4847ED4: strlen (vg_replace_strmem.c:501)
==309525== by 0x4E30412: core::ffi::c_str::CStr::from_ptr (c_str.rs:293)
==309525== by 0x4E3465A: opendal_operator_stat (lib.rs:416)
==309525== by 0x10BA63: OpendalListTest_ListDirTest_Test::TestBody() (in /home/xuanwo/Code/apache/incubator-opendal/bindings/c/build/list)
==309525== by 0x5FDCB3B: UnknownInlinedFun (gtest.cc:2621)
==309525== by 0x5FDCB3B: void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) [clone .constprop.0] (gtest.cc:2657)
==309525== by 0x5FC9197: UnknownInlinedFun (gtest.cc:2696)
==309525== by 0x5FC9197: testing::Test::Run() (gtest.cc:2686)
==309525== by 0x5FC93A6: testing::TestInfo::Run() (gtest.cc:2845)
==309525== by 0x5FC9503: UnknownInlinedFun (gtest.cc:3004)
==309525== by 0x5FC9503: testing::TestSuite::Run() (gtest.cc:2977)
==309525== by 0x5FD5A6E: testing::internal::UnitTestImpl::RunAllTests() (gtest.cc:5890)
==309525== by 0x5FD46A8: UnknownInlinedFun (gtest.cc:2621)
==309525== by 0x5FD46A8: UnknownInlinedFun (gtest.cc:2657)
==309525== by 0x5FD46A8: testing::UnitTest::Run() (gtest.cc:5455)
==309525== by 0x10CAC3: RUN_ALL_TESTS() (in /home/xuanwo/Code/apache/incubator-opendal/bindings/c/build/list)
==309525== by 0x10C2F5: main (in /home/xuanwo/Code/apache/incubator-opendal/bindings/c/build/list)
==309525== Address 0x659a751 is 0 bytes after a block of size 65 alloc'd
==309525== at 0x4841848: malloc (vg_replace_malloc.c:431)
==309525== by 0x575FF6D: alloc (alloc.rs:102)
==309525== by 0x575FF6D: alloc::alloc::Global::alloc_impl (alloc.rs:185)
==309525== by 0x5760998: <alloc::alloc::Global as core::alloc::Allocator>::allocate (alloc.rs:245)
==309525== by 0x575C09B: alloc::raw_vec::RawVec<T,A>::allocate_in (raw_vec.rs:184)
==309525== by 0x575E9EB: with_capacity_in<u8, alloc::alloc::Global> (raw_vec.rs:130)
==309525== by 0x575E9EB: with_capacity_in<u8, alloc::alloc::Global> (mod.rs:672)
==309525== by 0x575E9EB: <T as alloc::slice::hack::ConvertVec>::to_vec (slice.rs:162)
==309525== by 0x4E9F84B: to_vec<u8, alloc::alloc::Global> (slice.rs:111)
==309525== by 0x4E9F84B: to_vec_in<u8, alloc::alloc::Global> (slice.rs:441)
==309525== by 0x4E9F84B: to_vec<u8> (slice.rs:416)
==309525== by 0x4E9F84B: to_owned<u8> (slice.rs:823)
==309525== by 0x4E9F84B: to_owned (str.rs:209)
==309525== by 0x4E9F84B: from (string.rs:2667)
==309525== by 0x4E9F84B: <str as alloc::string::ToString>::to_string (string.rs:2604)
==309525== by 0x53A7CBF: opendal::raw::oio::entry::Entry::new (entry.rs:37)
==309525== by 0x54BAE78: opendal::raw::adapters::typed_kv::backend::KvPager::inner_next_page::{{closure}} (backend.rs:339)
==309525== by 0x53B1DD1: core::iter::adapters::map::map_fold::{{closure}} (map.rs:84)
==309525== by 0x52DBC4A: core::iter::traits::iterator::Iterator::fold (iterator.rs:2481)
==309525== by 0x53AAC23: <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::fold (map.rs:124)
==309525== by 0x53AD95E: core::iter::traits::iterator::Iterator::for_each (iterator.rs:856)
==309525==
==309525== Invalid read of size 1
==309525== at 0x4849018: strcmp (vg_replace_strmem.c:939)
==309525== by 0x10BB7C: OpendalListTest_ListDirTest_Test::TestBody() (in /home/xuanwo/Code/apache/incubator-opendal/bindings/c/build/list)
==309525== by 0x5FDCB3B: UnknownInlinedFun (gtest.cc:2621)
==309525== by 0x5FDCB3B: void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) [clone .constprop.0] (gtest.cc:2657)
==309525== by 0x5FC9197: UnknownInlinedFun (gtest.cc:2696)
==309525== by 0x5FC9197: testing::Test::Run() (gtest.cc:2686)
==309525== by 0x5FC93A6: testing::TestInfo::Run() (gtest.cc:2845)
==309525== by 0x5FC9503: UnknownInlinedFun (gtest.cc:3004)
==309525== by 0x5FC9503: testing::TestSuite::Run() (gtest.cc:2977)
==309525== by 0x5FD5A6E: testing::internal::UnitTestImpl::RunAllTests() (gtest.cc:5890)
==309525== by 0x5FD46A8: UnknownInlinedFun (gtest.cc:2621)
==309525== by 0x5FD46A8: UnknownInlinedFun (gtest.cc:2657)
==309525== by 0x5FD46A8: testing::UnitTest::Run() (gtest.cc:5455)
==309525== by 0x10CAC3: RUN_ALL_TESTS() (in /home/xuanwo/Code/apache/incubator-opendal/bindings/c/build/list)
==309525== by 0x10C2F5: main (in /home/xuanwo/Code/apache/incubator-opendal/bindings/c/build/list)
==309525== Address 0x659a751 is 0 bytes after a block of size 65 alloc'd
==309525== at 0x4841848: malloc (vg_replace_malloc.c:431)
==309525== by 0x575FF6D: alloc (alloc.rs:102)
==309525== by 0x575FF6D: alloc::alloc::Global::alloc_impl (alloc.rs:185)
==309525== by 0x5760998: <alloc::alloc::Global as core::alloc::Allocator>::allocate (alloc.rs:245)
==309525== by 0x575C09B: alloc::raw_vec::RawVec<T,A>::allocate_in (raw_vec.rs:184)
==309525== by 0x575E9EB: with_capacity_in<u8, alloc::alloc::Global> (raw_vec.rs:130)
==309525== by 0x575E9EB: with_capacity_in<u8, alloc::alloc::Global> (mod.rs:672)
==309525== by 0x575E9EB: <T as alloc::slice::hack::ConvertVec>::to_vec (slice.rs:162)
==309525== by 0x4E9F84B: to_vec<u8, alloc::alloc::Global> (slice.rs:111)
==309525== by 0x4E9F84B: to_vec_in<u8, alloc::alloc::Global> (slice.rs:441)
==309525== by 0x4E9F84B: to_vec<u8> (slice.rs:416)
==309525== by 0x4E9F84B: to_owned<u8> (slice.rs:823)
==309525== by 0x4E9F84B: to_owned (str.rs:209)
==309525== by 0x4E9F84B: from (string.rs:2667)
==309525== by 0x4E9F84B: <str as alloc::string::ToString>::to_string (string.rs:2604)
==309525== by 0x53A7CBF: opendal::raw::oio::entry::Entry::new (entry.rs:37)
==309525== by 0x54BAE78: opendal::raw::adapters::typed_kv::backend::KvPager::inner_next_page::{{closure}} (backend.rs:339)
==309525== by 0x53B1DD1: core::iter::adapters::map::map_fold::{{closure}} (map.rs:84)
==309525== by 0x52DBC4A: core::iter::traits::iterator::Iterator::fold (iterator.rs:2481)
==309525== by 0x53AAC23: <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::fold (map.rs:124)
==309525== by 0x53AD95E: core::iter::traits::iterator::Iterator::for_each (iterator.rs:856)
==309525==
[ OK ] OpendalListTest.ListDirTest (2160 ms)
[ RUN ] OpendalListTest.ListEmptyDirTest
[ OK ] OpendalListTest.ListEmptyDirTest (1 ms)
[ RUN ] OpendalListTest.ListNotExistDirTest
[ OK ] OpendalListTest.ListNotExistDirTest (0 ms)
[----------] 3 tests from OpendalListTest (2165 ms total)
[----------] Global test environment tear-down
[==========] 3 tests from 1 test suite ran. (2186 ms total)
[ PASSED ] 3 tests.
==309525==
==309525== HEAP SUMMARY:
==309525== in use at exit: 577 bytes in 4 blocks
==309525== total heap usage: 350 allocs, 346 frees, 8,514,401 bytes allocated
==309525==
==309525== 248 (8 direct, 240 indirect) bytes in 1 blocks are definitely lost in loss record 3 of 4
==309525== at 0x4841848: malloc (vg_replace_malloc.c:431)
==309525== by 0x4E3592D: alloc (alloc.rs:102)
==309525== by 0x4E3592D: alloc::alloc::Global::alloc_impl (alloc.rs:185)
==309525== by 0x4E356D6: allocate (alloc.rs:245)
==309525== by 0x4E356D6: alloc::alloc::exchange_malloc (alloc.rs:334)
==309525== by 0x4E3476A: new<opendal_c::types::opendal_metadata> (boxed.rs:217)
==309525== by 0x4E3476A: opendal_operator_stat (lib.rs:419)
==309525== by 0x10BA63: OpendalListTest_ListDirTest_Test::TestBody() (in /home/xuanwo/Code/apache/incubator-opendal/bindings/c/build/list)
==309525== by 0x5FDCB3B: UnknownInlinedFun (gtest.cc:2621)
==309525== by 0x5FDCB3B: void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) [clone .constprop.0] (gtest.cc:2657)
==309525== by 0x5FC9197: UnknownInlinedFun (gtest.cc:2696)
==309525== by 0x5FC9197: testing::Test::Run() (gtest.cc:2686)
==309525== by 0x5FC93A6: testing::TestInfo::Run() (gtest.cc:2845)
==309525== by 0x5FC9503: UnknownInlinedFun (gtest.cc:3004)
==309525== by 0x5FC9503: testing::TestSuite::Run() (gtest.cc:2977)
==309525== by 0x5FD5A6E: testing::internal::UnitTestImpl::RunAllTests() (gtest.cc:5890)
==309525== by 0x5FD46A8: UnknownInlinedFun (gtest.cc:2621)
==309525== by 0x5FD46A8: UnknownInlinedFun (gtest.cc:2657)
==309525== by 0x5FD46A8: testing::UnitTest::Run() (gtest.cc:5455)
==309525== by 0x10CAC3: RUN_ALL_TESTS() (in /home/xuanwo/Code/apache/incubator-opendal/bindings/c/build/list)
==309525==
==309525== 329 (264 direct, 65 indirect) bytes in 1 blocks are definitely lost in loss record 4 of 4
==309525== at 0x4841848: malloc (vg_replace_malloc.c:431)
==309525== by 0x4E3592D: alloc (alloc.rs:102)
==309525== by 0x4E3592D: alloc::alloc::Global::alloc_impl (alloc.rs:185)
==309525== by 0x4E356D6: allocate (alloc.rs:245)
==309525== by 0x4E356D6: alloc::alloc::exchange_malloc (alloc.rs:334)
==309525== by 0x4E333E6: new<opendal::types::entry::Entry> (boxed.rs:217)
==309525== by 0x4E333E6: opendal_c::types::opendal_list_entry::new (types.rs:373)
==309525== by 0x4E3319C: opendal_lister_next (types.rs:343)
==309525== by 0x10BA24: OpendalListTest_ListDirTest_Test::TestBody() (in /home/xuanwo/Code/apache/incubator-opendal/bindings/c/build/list)
==309525== by 0x5FDCB3B: UnknownInlinedFun (gtest.cc:2621)
==309525== by 0x5FDCB3B: void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) [clone .constprop.0] (gtest.cc:2657)
==309525== by 0x5FC9197: UnknownInlinedFun (gtest.cc:2696)
==309525== by 0x5FC9197: testing::Test::Run() (gtest.cc:2686)
==309525== by 0x5FC93A6: testing::TestInfo::Run() (gtest.cc:2845)
==309525== by 0x5FC9503: UnknownInlinedFun (gtest.cc:3004)
==309525== by 0x5FC9503: testing::TestSuite::Run() (gtest.cc:2977)
==309525== by 0x5FD5A6E: testing::internal::UnitTestImpl::RunAllTests() (gtest.cc:5890)
==309525== by 0x5FD46A8: UnknownInlinedFun (gtest.cc:2621)
==309525== by 0x5FD46A8: UnknownInlinedFun (gtest.cc:2657)
==309525== by 0x5FD46A8: testing::UnitTest::Run() (gtest.cc:5455)
==309525==
==309525== LEAK SUMMARY:
==309525== definitely lost: 272 bytes in 2 blocks
==309525== indirectly lost: 305 bytes in 2 blocks
==309525== possibly lost: 0 bytes in 0 blocks
==309525== still reachable: 0 bytes in 0 blocks
==309525== suppressed: 0 bytes in 0 blocks
==309525==
==309525== For lists of detected and suppressed errors, rerun with: -s
==309525== ERROR SUMMARY: 4 errors from 4 contexts (suppressed: 0 from 0)
make: *** [Makefile:48: test] Error 1 |
Just some update, my libc was messed up today, I am rebuilding the system and libraries. So I think the fix should come later tomorrow. |
Hi @wildbartty , I have raised the fix in #2673. I believe this could be merged as soon as that one get merged. |
#2673 merged! |
Hey @wildbartty, do you want to |
@Xuanwo I've updated my branch. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the work! The PR LGTM. Let's wait the CI to pass.
This is for issue #2495
Note that it currently breaks the ci testing because it has found memory leaks