Skip to content
This repository has been archived by the owner on Jun 23, 2022. It is now read-only.

valgrind: use static-allocated array to store mapping from task_code to task_spec #173

Merged
merged 9 commits into from
Oct 16, 2018

Conversation

neverchanje
Copy link
Contributor

@neverchanje neverchanje commented Oct 15, 2018

Valgrind warns that value pointer in dsn::utils::singleton_vector_store<task_spec *, nullptr> is not deleted after destruction, so we use a static allocated (512 size of capacity, enough for task_code) array of std::unique_ptr<task_spec> to replace the singleton_vector_store implementation.

valgrind report:

==11840==    by 0x5D1C3DF: std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(char const*, std::allocator<char> const&) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25) ==11840==    by 0x86D299: dsn::join_point_base::join_point_base(char const*) (join_point.cpp:43)                                                                                                             
==11840==    by 0x7E8B9E: dsn::join_point<void, dsn::task*, dsn::task*, bool>::join_point(char const*) (join_point.h:142)                                                                                    
==11840==    by 0x7E2764: dsn::task_spec::task_spec(int, char const*, dsn_task_type_t, dsn_task_priority_t, dsn::threadpool_code) (task_spec.cpp:156)                                                        
==11840==    by 0x7E1A9B: dsn::task_spec::register_task_code(dsn::task_code, dsn_task_type_t, dsn_task_priority_t, dsn::threadpool_code) (task_spec.cpp:53)                                                  
==11840==    by 0x7D5824: dsn::task_code::task_code(char const*, dsn_task_type_t, dsn_task_priority_t, dsn::threadpool_code) (task_code.cpp:43)                                                              
==11840==    by 0x7E1BAA: dsn::task_spec::registe==11454==
==11454== Conditional jump or move depends on uninitialised value(s)
==11454==    at 0x4601EC: i64toa (itoa.h:293)
==11454==    by 0x4601EC: WriteInt64 (writer.h:301)
==11454==    by 0x4601EC: rapidjson::Writer<rapidjson::BasicOStreamWrapper<std::ostream>, rapidjson::UTF8<char>, rapidjson::UTF8<char>, rapidjson::CrtAllocator, 0u>::Int64(long) (writer.h:176)
==11454==    by 0x464085: json_encode (json_helper.h:271)
==11454==    by 0x464085: encode_inner (json_helper.h:517)
==11454==    by 0x464085: encode (json_helper.h:541)
==11454==    by 0x464085: encode_json_state (replica_context.h:228)
==11454==    by 0x464085: encode_inner (json_helper.h:505)
==11454==    by 0x464085: encode (json_helper.h:541)
==11454==    by 0x464085: encode (json_helper.h:547)
==11454==    by 0x464085: dsn::json::json_forwarder<dsn::replication::cold_backup_metadata>::encode(dsn::replication::cold_backup_metadata const&) (json_helper.h:552)
==11454==    by 0x45D3F5: replication_service_test_app::read_backup_metadata_test() (cold_backup_context_test.cpp:230)
==11454==    by 0x494365: void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) (in /home/mi/git/release/pegasus/rdsn/builder/src/dist/replication/test/replica_test/unit_test/dsn.replica.test)
==11454==    by 0x48E125: void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) (in /home/mi/git/release/pegasus/rdsn/builder/src/dist/replication/test/replica_test/unit_test/dsn.replica.test)
==11454==    by 0x473EBE: testing::Test::Run() (in /home/mi/git/release/pegasus/rdsn/builder/src/dist/replication/test/replica_test/unit_test/dsn.replica.test)
==11454==    by 0x474747: testing::TestInfo::Run() (in /home/mi/git/release/pegasus/rdsn/builder/src/dist/replication/test/replica_test/unit_test/dsn.replica.test)
==11454==    by 0x474DD7: testing::TestCase::Run() (in /home/mi/git/release/pegasus/rdsn/builder/src/dist/replication/test/replica_test/unit_test/dsn.replica.test)
==11454==    by 0x47B51B: testing::internal::UnitTestImpl::RunAllTests() (in /home/mi/git/release/pegasus/rdsn/builder/src/dist/replication/test/replica_test/unit_test/dsn.replica.test)
==11454==    by 0x495743: bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) (in /home/mi/git/release/pegasus/rdsn/builder/src/dist/replication/test/replica_test/unit_test/dsn.replica.test)
==11454==    by 0x48EF1B: bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) (in /home/mi/git/release/pegasus/rdsn/builder/src/dist/replication/test/replica_test/unit_test/dsn.replica.test)
==11454==    by 0x47A1AD: testing::UnitTest::Run() (in /home/mi/git/release/pegasus/rdsn/builder/src/dist/replication/test/replica_test/unit_test/dsn.replica.test)
==11454==r_task_code(dsn::task_code, dsn_task_type_t, dsn_task_priority_t, dsn::threadpool_code) (task_spec.cpp:63)                                                  
==11840==    by 0x7D5824: dsn::task_code::task_code(char const*, dsn_task_type_t, dsn_task_priority_t, dsn::threadpool_code) (task_code.cpp:43)                                                              
==11840==    by 0x8DA7B0: __static_initialization_and_destruction_0(int, int) (partition_resolver_simple.cpp:233)

singleton_vector_store is a complicated container only used for storing task_spec, we removed it also to make the code simple.

This PR also suppress the warning of json_helper, see include/dsn/cpp/json_helper.h:

==11454==
==11454== Conditional jump or move depends on uninitialised value(s)
==11454==    at 0x4601EC: i64toa (itoa.h:293)
==11454==    by 0x4601EC: WriteInt64 (writer.h:301)
==11454==    by 0x4601EC: rapidjson::Writer<rapidjson::BasicOStreamWrapper<std::ostream==11454==
==11454== Conditional jump or move depends on uninitialised value(s)
==11454==    at 0x4601EC: i64toa (itoa.h:293)
==11454==    by 0x4601EC: WriteInt64 (writer.h:301)
==11454==    by 0x4601EC: rapidjson::Writer<rapidjson::BasicOStreamWrapper<std::ostream>, rapidjson::UTF8<char>, rapidjson::UTF8<char>, rapidjson::CrtAllocator, 0u>::Int64(long) (writer.h:176)
==11454==    by 0x464085: json_encode (json_helper.h:271)
==11454==    by 0x464085: encode_inner (json_helper.h:517)
==11454==    by 0x464085: encode (json_helper.h:541)
==11454==    by 0x464085: encode_json_state (replica_context.h:228)
==11454==    by 0x464085: encode_inner (json_helper.h:505)
==11454==    by 0x464085: encode (json_helper.h:541)
==11454==    by 0x464085: encode (json_helper.h:547)
==11454==    by 0x464085: dsn::json::json_forwarder<dsn::replication::cold_backup_metadata>::encode(dsn::replication::cold_backup_metadata const&) (json_helper.h:552)
==11454==    by 0x45D3F5: replication_service_test_app::read_backup_metadata_test() (cold_backup_context_test.cpp:230)
==11454==    by 0x494365: void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) (in /home/mi/git/release/pegasus/rdsn/builder/src/dist/replication/test/replica_test/unit_test/dsn.replica.test)
==11454==    by 0x48E125: void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) (in /home/mi/git/release/pegasus/rdsn/builder/src/dist/replication/test/replica_test/unit_test/dsn.replica.test)
==11454==    by 0x473EBE: testing::Test::Run() (in /home/mi/git/release/pegasus/rdsn/builder/src/dist/replication/test/replica_test/unit_test/dsn.replica.test)
==11454==    by 0x474747: testing::TestInfo::Run() (in /home/mi/git/release/pegasus/rdsn/builder/src/dist/replication/test/replica_test/unit_test/dsn.replica.test)
==11454==    by 0x474DD7: testing::TestCase::Run() (in /home/mi/git/release/pegasus/rdsn/builder/src/dist/replication/test/replica_test/unit_test/dsn.replica.test)
==11454==    by 0x47B51B: testing::internal::UnitTestImpl::RunAllTests() (in /home/mi/git/release/pegasus/rdsn/builder/src/dist/replication/test/replica_test/unit_test/dsn.replica.test)
==11454==    by 0x495743: bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) (in /home/mi/git/release/pegasus/rdsn/builder/src/dist/replication/test/replica_test/unit_test/dsn.replica.test)
==11454==    by 0x48EF1B: bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) (in /home/mi/git/release/pegasus/rdsn/builder/src/dist/replication/test/replica_test/unit_test/dsn.replica.test)
==11454==    by 0x47A1AD: testing::UnitTest::Run() (in /home/mi/git/release/pegasus/rdsn/builder/src/dist/replication/test/replica_test/unit_test/dsn.replica.test)

@neverchanje neverchanje added the area/memcheck PR or issues related to valgrind memcheck label Oct 15, 2018
@neverchanje neverchanje mentioned this pull request Oct 15, 2018
9 tasks
@shengofsun
Copy link
Contributor

不太赞同这种预分配512个元素的做法,几乎就是在挖坑……

@neverchanje
Copy link
Contributor Author

@shengofsun 你指的是什么坑,我觉得未来也不会超过 512 task_code 了,内存占用也不大。而且其实也只是换了容器的实现方式,未来有更简单的实现或者更明确的需求都可以替换掉。
我主要实在没觉着专门写个 singleton_vector_store 来动态存 task_code 有何意义,一个本来很简单的东西设计的那么麻烦...

{
return dsn::utils::singleton_vector_store<task_spec *, nullptr>::instance().get(code);
}
task_spec *task_spec::get(int code) { return s_task_spec_store[code].get(); }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

该为:

return code < 512 ? s_task_spec_store[code].get() : nullptr;


namespace dsn {

// A sequential storage maps task_code to task_spec.
static std::array<std::unique_ptr<task_spec>, 512> s_task_spec_store;

void task_spec::register_task_code(task_code code,
dsn_task_type_t type,
dsn_task_priority_t pri,
dsn::threadpool_code pool)
{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

加上: dassert(code < 512, "code = %d", code);

{
return dsn::utils::singleton_vector_store<task_spec *, nullptr>::instance().get(code);
}
task_spec *task_spec::get(int code) { return code < 512 ? s_task_spec_store[code].get() : nullptr; }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这么多512,定义个常量?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个建议可以采纳一下,我也这么想过

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cool

@neverchanje neverchanje merged commit 9fdc118 into XiaoMi:master Oct 16, 2018
vagetablechicken pushed a commit to vagetablechicken/rdsn that referenced this pull request Nov 5, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area/memcheck PR or issues related to valgrind memcheck
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants