ASAN in trace code in pggate_test_catalog when built with devtoolset-8 gcc-8

[mbautin]

gcc 8 supports ASAN. I modified compilation flags in an ad-hoc way to enable ASAN while investigating core dumps in pggate_test_catalog under gcc-8 and got this:

=================================================================
==7490==ERROR: AddressSanitizer: use-after-poison on address 0x62500000c918 at pc 0x7f4d206b1021 bp 0x7ffc6e5368c0 sp 0x7ffc6e5368b0
WRITE of size 8 at 0x62500000c918 thread T0
[m-1] I1030 19:13:43.652909 7520 sys_catalog.cc:336] Determining permanent_uuid for [127.0.0.1:23341]
[m-1] =================================================================
[m-1] ==7496==ERROR: AddressSanitizer: use-after-poison on address 0x625000032118 at pc 0x7fd951d93021 bp 0x7fd93c99cc80 sp 0x7fd93c99cc70
[m-1] WRITE of size 8 at 0x625000032118 thread T22 (init [worker]xx)
#0 0x7f4d206b1020 in yb::Trace::NewEntry(int, char const*, int, yb::MonoTime) ../../src/yb/util/trace.cc:289
[m-1] #0 0x7fd951d93020 in yb::Trace::NewEntry(int, char const*, int, yb::MonoTime) ../../src/yb/util/trace.cc:289
[m-1] #1 0x7fd951d93289 in yb::Trace::SubstituteAndTrace(char const*, int, yb::MonoTime, GStringPiece) ../../src/yb/util/trace.cc:252
#1 0x7f4d206b1289 in yb::Trace::SubstituteAndTrace(char const*, int, yb::MonoTime, GStringPiece) ../../src/yb/util/trace.cc:252
#2 0x7f4d22372ad5 in yb::rpc::OutboundCall::OutboundCall(yb::rpc::RemoteMethod const*, std::shared_ptryb::rpc::OutboundCallMetrics const&, google::protobuf::Message*, yb::rpc::RpcController*, yb::rpc::RpcMetrics*, std::function<void ()>, yb::rpc::ThreadPool*) ../../src/yb/rpc/outbound_call.cc:204
[m-1] #2 0x7fd9538a9ad5 in yb::rpc::OutboundCall::OutboundCall(yb::rpc::RemoteMethod const*, std::shared_ptryb::rpc::OutboundCallMetrics const&, google::protobuf::Message*, yb::rpc::RpcController*, yb::rpc::RpcMetrics*, std::function<void ()>, yb::rpc::ThreadPool*) ../../src/yb/rpc/outbound_call.cc:204
[m-1] #3 0x7fd9538e1417 in void __gnu_cxx::new_allocatoryb::rpc::OutboundCall::construct<yb::rpc::OutboundCall, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*, std::function<void ()>, yb::rpc::ThreadPool*>(yb::rpc::OutboundCall*, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*&&, std::function<void ()>&&, yb::rpc::ThreadPool*&&) /opt/rh/devtoolset-8/root/usr/include/c++/8/ext/new_allocator.h:136
[m-1] #4 0x7fd9538e1417 in void std::allocator_traits<std::allocatoryb::rpc::OutboundCall >::construct<yb::rpc::OutboundCall, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*, std::function<void ()>, yb::rpc::ThreadPool*>(std::allocatoryb::rpc::OutboundCall&, yb::rpc::OutboundCall*, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*&&, std::function<void ()>&&, yb::rpc::ThreadPool*&&) /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/alloc_traits.h:475
[m-1] #5 0x7fd9538e1417 in std::_Sp_counted_ptr_inplace<yb::rpc::OutboundCall, std::allocatoryb::rpc::OutboundCall, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*, std::function<void ()>, yb::rpc::ThreadPool*>(std::allocatoryb::rpc::OutboundCall, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*&&, std::function<void ()>&&, yb::rpc::ThreadPool*&&) /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/shared_ptr_base.h:545
[m-1] #6 0x7fd9538e1417 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<yb::rpc::OutboundCall, std::allocatoryb::rpc::OutboundCall, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*, std::function<void ()>, yb::rpc::ThreadPool*>(yb::rpc::OutboundCall*&, std::_Sp_alloc_shared_tag<std::allocatoryb::rpc::OutboundCall >, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*&&, std::function<void ()>&&, yb::rpc::ThreadPool*&&) /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/shared_ptr_base.h:677
#3 0x7f4d223aa417 in void __gnu_cxx::new_allocatoryb::rpc::OutboundCall::construct<yb::rpc::OutboundCall, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*, std::function<void ()>, yb::rpc::ThreadPool*>(yb::rpc::OutboundCall*, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*&&, std::function<void ()>&&, yb::rpc::ThreadPool*&&) /opt/rh/devtoolset-8/root/usr/include/c++/8/ext/new_allocator.h:136
[m-1] #4 0x7f4d223aa417 in void std::allocator_traits<std::allocatoryb::rpc::OutboundCall >::construct<yb::rpc::OutboundCall, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*, std::function<void ()>, yb::rpc::ThreadPool*>(std::allocatoryb::rpc::OutboundCall&, yb::rpc::OutboundCall*, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*&&, std::function<void ()>&&, yb::rpc::ThreadPool*&&) /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/alloc_traits.h:475
#7 0x7fd9538e1417 in std::__shared_ptr<yb::rpc::OutboundCall, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocatoryb::rpc::OutboundCall, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*, std::function<void ()>, yb::rpc::ThreadPool*>(std::_Sp_alloc_shared_tag<std::allocatoryb::rpc::OutboundCall >, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*&&, std::function<void ()>&&, yb::rpc::ThreadPool*&&) /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/shared_ptr_base.h:1342
[m-1] #8 0x7fd9538e1417 in std::shared_ptryb::rpc::OutboundCall::shared_ptr<std::allocatoryb::rpc::OutboundCall, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*, std::function<void ()>, yb::rpc::ThreadPool*>(std::_Sp_alloc_shared_tag<std::allocatoryb::rpc::OutboundCall >, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*&&, std::function<void ()>&&, yb::rpc::ThreadPool*&&) /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/shared_ptr.h:359
#5 0x7f4d223aa417 in std::_Sp_counted_ptr_inplace<yb::rpc::OutboundCall, std::allocatoryb::rpc::OutboundCall, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*, std::function<void ()>, yb::rpc::ThreadPool*>(std::allocatoryb::rpc::OutboundCall, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*&&, std::function<void ()>&&, yb::rpc::ThreadPool*&&) /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/shared_ptr_base.h:545
[m-1] #9 0x7fd9538e1417 in std::shared_ptryb::rpc::OutboundCall std::allocate_shared<yb::rpc::OutboundCall, std::allocatoryb::rpc::OutboundCall, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*, std::function<void ()>, yb::rpc::ThreadPool*>(std::allocatoryb::rpc::OutboundCall const&, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*&&, std::function<void ()>&&, yb::rpc::ThreadPool*&&) /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/shared_ptr.h:706
#6 0x7f4d223aa417 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<yb::rpc::OutboundCall, std::allocatoryb::rpc::OutboundCall, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*, std::function<void ()>, yb::rpc::ThreadPool*>(yb::rpc::OutboundCall*&, std::_Sp_alloc_shared_tag<std::allocatoryb::rpc::OutboundCall >, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*&&, std::function<void ()>&&, yb::rpc::ThreadPool*&&) /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/shared_ptr_base.h:677
[m-1] #10 0x7fd9538e1417 in std::shared_ptryb::rpc::OutboundCall std::make_shared<yb::rpc::OutboundCall, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*, std::function<void ()>, yb::rpc::ThreadPool*>(yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*&&, std::function<void ()>&&, yb::rpc::ThreadPool*&&) /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/shared_ptr.h:722
[m-1] #11 0x7fd9538e1417 in yb::rpc::Proxy::DoAsyncRequest(yb::rpc::RemoteMethod const*, google::protobuf::Message const&, google::protobuf::Message*, yb::rpc::RpcController*, std::function<void ()>, bool) ../../src/yb/rpc/proxy.cc:150
#7 0x7f4d223aa417 in std::__shared_ptr<yb::rpc::OutboundCall, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocatoryb::rpc::OutboundCall, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*, std::function<void ()>, yb::rpc::ThreadPool*>(std::_Sp_alloc_shared_tag<std::allocatoryb::rpc::OutboundCall >, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*&&, std::function<void ()>&&, yb::rpc::ThreadPool*&&) /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/shared_ptr_base.h:1342
#8 0x7f4d223aa417 in std::shared_ptryb::rpc::OutboundCall::shared_ptr<std::allocatoryb::rpc::OutboundCall, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*, std::function<void ()>, yb::rpc::ThreadPool*>(std::_Sp_alloc_shared_tag<std::allocatoryb::rpc::OutboundCall >, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*&&, std::function<void ()>&&, yb::rpc::ThreadPool*&&) /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/shared_ptr.h:359
#9 0x7f4d223aa417 in std::shared_ptryb::rpc::OutboundCall std::allocate_shared<yb::rpc::OutboundCall, std::allocatoryb::rpc::OutboundCall, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*, std::function<void ()>, yb::rpc::ThreadPool*>(std::allocatoryb::rpc::OutboundCall const&, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*&&, std::function<void ()>&&, yb::rpc::ThreadPool*&&) /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/shared_ptr.h:706
[m-1] #12 0x7fd9538e484a in yb::rpc::Proxy::AsyncRequest(yb::rpc::RemoteMethod const*, google::protobuf::Message const&, google::protobuf::Message*, yb::rpc::RpcController*, std::function<void ()>) ../../src/yb/rpc/proxy.cc:118
#10 0x7f4d223aa417 in std::shared_ptryb::rpc::OutboundCall std::make_shared<yb::rpc::OutboundCall, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*, std::function<void ()>, yb::rpc::ThreadPool*>(yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*&&, std::function<void ()>&&, yb::rpc::ThreadPool*&&) /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/shared_ptr.h:722
#11 0x7f4d223aa417 in yb::rpc::Proxy::DoAsyncRequest(yb::rpc::RemoteMethod const*, google::protobuf::Message const&, google::protobuf::Message*, yb::rpc::RpcController*, std::function<void ()>, bool) ../../src/yb/rpc/proxy.cc:150
#12 0x7f4d223ad84a in yb::rpc::Proxy::AsyncRequest(yb::rpc::RemoteMethod const*, google::protobuf::Message const&, google::protobuf::Message*, yb::rpc::RpcController*, std::function<void ()>) ../../src/yb/rpc/proxy.cc:118
[m-1] #13 0x7fd9570c9222 in yb::consensus::ConsensusServiceProxy::GetNodeInstanceAsync(yb::consensus::GetNodeInstanceRequestPB const&, yb::consensus::GetNodeInstanceResponsePB*, yb::rpc::RpcController*, std::function<void ()>) src/yb/consensus/consensus.proxy.cc:71
#13 0x7f4d24ebd480 in yb::master::MasterServiceProxy::GetMasterRegistrationAsync(yb::master::GetMasterRegistrationRequestPB const&, yb::master::GetMasterRegistrationResponsePB*, yb::rpc::RpcController*, std::function<void ()>) src/yb/master/master.proxy.cc:604
[m-1] #14 0x7fd95bcffdd8 in yb::consensus::SetPermanentUuidForRemotePeer(yb::rpc::ProxyCache*, std::chrono::duration<long, std::ratio<1l, 1000000000l> >, std::vector<yb::HostPort, std::allocatoryb::HostPort > const&, yb::consensus::RaftPeerPB*) ../../src/yb/consensus/consensus_peers.cc:618
#14 0x7f4d29157e0f in SendRpc ../../src/yb/master/master_rpc.cc:110
#15 0x7f4d29154b93 in yb::master::GetLeaderMasterRpc::SendRpc() ../../src/yb/master/master_rpc.cc:209
[m-1] #15 0x7fd95fc0e0f8 in yb::master::SysCatalogTable::SetupConfig(yb::master::MasterOptions const&, yb::consensus::RaftConfigPB*) ../../src/yb/master/sys_catalog.cc:341
[m-1] #16 0x7fd95fc21245 in yb::master::SysCatalogTable::CreateNew(yb::FsManager*) ../../src/yb/master/sys_catalog.cc:314
#16 0x7f4d3204d3ea in std::shared_ptryb::rpc::RpcCommand yb::rpc::StartRpc<yb::master::GetLeaderMasterRpc, yb::Callback<void (yb::Status const&, yb::HostPort const&)>, std::vector<std::vector<yb::HostPort, std::allocatoryb::HostPort >, std::allocator<std::vector<yb::HostPort, std::allocatoryb::HostPort > > >&, std::chrono::time_point<yb::CoarseMonoClock, std::chrono::duration<long, std::ratio<1l, 1000000000l> > >&, yb::rpc::Messenger*&, yb::rpc::ProxyCache*, yb::rpc::Rpcs*>(yb::Callback<void (yb::Status const&, yb::HostPort const&)>&&, std::vector<std::vector<yb::HostPort, std::allocatoryb::HostPort >, std::allocator<std::vector<yb::HostPort, std::allocatoryb::HostPort > > >&, std::chrono::time_point<yb::CoarseMonoClock, std::chrono::duration<long, std::ratio<1l, 1000000000l> > >&, yb::rpc::Messenger*&, yb::rpc::ProxyCache*&&, yb::rpc::Rpcs*&&) ../../src/yb/rpc/rpc.h:265
#17 0x7f4d32008097 in yb::ExternalMiniCluster::GetPeerMasterIndex(int*, bool) ../../src/yb/integration-tests/external_mini_cluster.cc:1302
#18 0x7f4d3200c15d in yb::ExternalMiniCluster::GetLeaderMasterIndex(int*) ../../src/yb/integration-tests/external_mini_cluster.cc:1276
#19 0x7f4d3200c9f4 in yb::ExternalMiniCluster::GetLeaderMaster() ../../src/yb/integration-tests/external_mini_cluster.cc:1335
#20 0x7f4d32023773 in yb::ExternalMiniCluster::AddTabletServer(bool, std::vector<std::string, std::allocatorstd::string > const&) ../../src/yb/integration-tests/external_mini_cluster.cc:1058
#21 0x7f4d3203d5f5 in yb::ExternalMiniCluster::Start(yb::rpc::Messenger*) ../../src/yb/integration-tests/external_mini_cluster.cc:348
#22 0x7f4d3260d7ff in yb::pggate::PggateTest::CreateCluster(int) ../../src/yb/yql/pggate/test/pggate_test.cc:128
#23 0x7f4d3260e4f7 in yb::pggate::PggateTest::Init(char const*, int) ../../src/yb/yql/pggate/test/pggate_test.cc:98
#24 0x40825f in yb::pggate::PggateTestCatalog_TestDml_Test::TestBody() ../../src/yb/yql/pggate/test/pggate_test_catalog.cc:30
#25 0x7f4d20b3235d in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::)(), char const) /nfusr/dev-server/mbautin2/code/yugabyte-db-thirdparty-devtoolset-8/src/gmock-1.8.0/googletest/src/gtest.cc:2402
#26 0x7f4d20b3235d in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::)(), char const) /nfusr/dev-server/mbautin2/code/yugabyte-db-thirdparty-devtoolset-8/src/gmock-1.8.0/googletest/src/gtest.cc:2438
#27 0x7f4d20b28e71 in testing::Test::Run() /nfusr/dev-server/mbautin2/code/yugabyte-db-thirdparty-devtoolset-8/src/gmock-1.8.0/googletest/src/gtest.cc:2474
#28 0x7f4d20b28e71 in testing::Test::Run() /nfusr/dev-server/mbautin2/code/yugabyte-db-thirdparty-devtoolset-8/src/gmock-1.8.0/googletest/src/gtest.cc:2465
#29 0x7f4d20b28fb7 in testing::TestInfo::Run() /nfusr/dev-server/mbautin2/code/yugabyte-db-thirdparty-devtoolset-8/src/gmock-1.8.0/googletest/src/gtest.cc:2656
#30 0x7f4d20b28fb7 in testing::TestInfo::Run() /nfusr/dev-server/mbautin2/code/yugabyte-db-thirdparty-devtoolset-8/src/gmock-1.8.0/googletest/src/gtest.cc:2630
#31 0x7f4d20b29094 in testing::TestCase::Run() /nfusr/dev-server/mbautin2/code/yugabyte-db-thirdparty-devtoolset-8/src/gmock-1.8.0/googletest/src/gtest.cc:2774
#32 0x7f4d20b29094 in testing::TestCase::Run() /nfusr/dev-server/mbautin2/code/yugabyte-db-thirdparty-devtoolset-8/src/gmock-1.8.0/googletest/src/gtest.cc:2759
#33 0x7f4d20b2956b in testing::internal::UnitTestImpl::RunAllTests() /nfusr/dev-server/mbautin2/code/yugabyte-db-thirdparty-devtoolset-8/src/gmock-1.8.0/googletest/src/gtest.cc:4649
#34 0x7f4d20b2956b in testing::internal::UnitTestImpl::RunAllTests() /nfusr/dev-server/mbautin2/code/yugabyte-db-thirdparty-devtoolset-8/src/gmock-1.8.0/googletest/src/gtest.cc:4551
#35 0x7f4d20b3287d in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::)(), char const) /nfusr/dev-server/mbautin2/code/yugabyte-db-thirdparty-devtoolset-8/src/gmock-1.8.0/googletest/src/gtest.cc:2402
#36 0x7f4d20b3287d in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::)(), char const) /nfusr/dev-server/mbautin2/code/yugabyte-db-thirdparty-devtoolset-8/src/gmock-1.8.0/googletest/src/gtest.cc:2438
#37 0x7f4d20b29664 in testing::UnitTest::Run() /nfusr/dev-server/mbautin2/code/yugabyte-db-thirdparty-devtoolset-8/src/gmock-1.8.0/googletest/src/gtest.cc:4257
#38 0x7f4d323fea98 in RUN_ALL_TESTS() /nfusr/dev-server/mbautin2/code/yugabyte-db-thirdparty-devtoolset-8/installed/uninstrumented/include/gtest/gtest.h:2233
#39 0x7f4d323fea98 in main ../../src/yb/util/test_main.cc:104
#40 0x7f4d1b849554 in __libc_start_main (/lib64/libc.so.6+0x22554)
#41 0x407188 (/nfusr/dev-server/mbautin2/code/yugabyte-db-devtoolset8/build/debug-gcc-dynamic-ninja/tests-pggate/pggate_test_catalog+0x407188)

0x62500000c918 is located 24 bytes inside of 8192-byte region [0x62500000c900,0x62500000e900)
allocated by thread T0 here:
#0 0x7f4d32d38638 in __interceptor_malloc (/lib64/libasan.so.5+0x10c638)
#1 0x7f4d20426af1 in yb::HeapBufferAllocator::Malloc(unsigned long) ../../src/yb/util/memory/memory.cc:190
#2 0x7f4d20428386 in yb::HeapBufferAllocator::AllocateInternal(unsigned long, unsigned long, yb::BufferAllocator*) ../../src/yb/util/memory/memory.cc:141
#3 0x7f4d2041f2f5 in yb::BufferAllocator::BestEffortAllocate(unsigned long, unsigned long) ../../src/yb/util/memory/memory.h:177
#4 0x7f4d2041f2f5 in yb::internal::ArenaBaseyb::internal::ThreadSafeArenaTraits::NewBuffer(unsigned long, unsigned long) ../../src/yb/util/memory/arena.cc:204
#5 0x7f4d204213a8 in yb::internal::ArenaBaseyb::internal::ThreadSafeArenaTraits::ArenaBase(unsigned long, unsigned long) ../../src/yb/util/memory/arena.cc:138
#6 0x7f4d206a08f2 in operator() ../../src/yb/util/trace.cc:220
#7 0x7f4d206a08f2 in _M_invoke /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/std_function.h:282
#8 0x7f4d206b0b29 in std::function<yb::internal::ArenaBaseyb::internal::ThreadSafeArenaTraits* ()>::operator()() const /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/std_function.h:687
#9 0x7f4d206b0b29 in yb::ThreadSafeObjectPool<yb::internal::ArenaBaseyb::internal::ThreadSafeArenaTraits >::Take() ../../src/yb/util/object_pool.h:228
#10 0x7f4d206b0b29 in yb::Trace::GetAndInitArena() ../../src/yb/util/trace.cc:236
#11 0x7f4d206b0c9d in yb::Trace::NewEntry(int, char const*, int, yb::MonoTime) ../../src/yb/util/trace.cc:280
#12 0x7f4d206b1289 in yb::Trace::SubstituteAndTrace(char const*, int, yb::MonoTime, GStringPiece) ../../src/yb/util/trace.cc:252
#13 0x7f4d22372ad5 in yb::rpc::OutboundCall::OutboundCall(yb::rpc::RemoteMethod const*, std::shared_ptryb::rpc::OutboundCallMetrics const&, google::protobuf::Message*, yb::rpc::RpcController*, yb::rpc::RpcMetrics*, std::function<void ()>, yb::rpc::ThreadPool*) ../../src/yb/rpc/outbound_call.cc:204
#14 0x7f4d223aa417 in void __gnu_cxx::new_allocatoryb::rpc::OutboundCall::construct<yb::rpc::OutboundCall, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*, std::function<void ()>, yb::rpc::ThreadPool*>(yb::rpc::OutboundCall*, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*&&, std::function<void ()>&&, yb::rpc::ThreadPool*&&) /opt/rh/devtoolset-8/root/usr/include/c++/8/ext/new_allocator.h:136
#15 0x7f4d223aa417 in void std::allocator_traits<std::allocatoryb::rpc::OutboundCall >::construct<yb::rpc::OutboundCall, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*, std::function<void ()>, yb::rpc::ThreadPool*>(std::allocatoryb::rpc::OutboundCall&, yb::rpc::OutboundCall*, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*&&, std::function<void ()>&&, yb::rpc::ThreadPool*&&) /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/alloc_traits.h:475
#16 0x7f4d223aa417 in std::_Sp_counted_ptr_inplace<yb::rpc::OutboundCall, std::allocatoryb::rpc::OutboundCall, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*, std::function<void ()>, yb::rpc::ThreadPool*>(std::allocatoryb::rpc::OutboundCall, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*&&, std::function<void ()>&&, yb::rpc::ThreadPool*&&) /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/shared_ptr_base.h:545
#17 0x7f4d223aa417 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<yb::rpc::OutboundCall, std::allocatoryb::rpc::OutboundCall, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*, std::function<void ()>, yb::rpc::ThreadPool*>(yb::rpc::OutboundCall*&, std::_Sp_alloc_shared_tag<std::allocatoryb::rpc::OutboundCall >, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*&&, std::function<void ()>&&, yb::rpc::ThreadPool*&&) /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/shared_ptr_base.h:677
#18 0x7f4d223aa417 in std::__shared_ptr<yb::rpc::OutboundCall, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocatoryb::rpc::OutboundCall, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*, std::function<void ()>, yb::rpc::ThreadPool*>(std::_Sp_alloc_shared_tag<std::allocatoryb::rpc::OutboundCall >, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*&&, std::function<void ()>&&, yb::rpc::ThreadPool*&&) /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/shared_ptr_base.h:1342
#19 0x7f4d223aa417 in std::shared_ptryb::rpc::OutboundCall::shared_ptr<std::allocatoryb::rpc::OutboundCall, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*, std::function<void ()>, yb::rpc::ThreadPool*>(std::_Sp_alloc_shared_tag<std::allocatoryb::rpc::OutboundCall >, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*&&, std::function<void ()>&&, yb::rpc::ThreadPool*&&) /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/shared_ptr.h:359
#20 0x7f4d223aa417 in std::shared_ptryb::rpc::OutboundCall std::allocate_shared<yb::rpc::OutboundCall, std::allocatoryb::rpc::OutboundCall, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*, std::function<void ()>, yb::rpc::ThreadPool*>(std::allocatoryb::rpc::OutboundCall const&, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*&&, std::function<void ()>&&, yb::rpc::ThreadPool*&&) /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/shared_ptr.h:706
#21 0x7f4d223aa417 in std::shared_ptryb::rpc::OutboundCall std::make_shared<yb::rpc::OutboundCall, yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*, std::function<void ()>, yb::rpc::ThreadPool*>(yb::rpc::RemoteMethod const*&, std::shared_ptryb::rpc::OutboundCallMetrics&, google::protobuf::Message*&, yb::rpc::RpcController*&, yb::rpc::RpcMetrics*&&, std::function<void ()>&&, yb::rpc::ThreadPool*&&) /opt/rh/devtoolset-8/root/usr/include/c++/8/bits/shared_ptr.h:722
#22 0x7f4d223aa417 in yb::rpc::Proxy::DoAsyncRequest(yb::rpc::RemoteMethod const*, google::protobuf::Message const&, google::protobuf::Message*, yb::rpc::RpcController*, std::function<void ()>, bool) ../../src/yb/rpc/proxy.cc:150
#23 0x7f4d223ad84a in yb::rpc::Proxy::AsyncRequest(yb::rpc::RemoteMethod const*, google::protobuf::Message const&, google::protobuf::Message*, yb::rpc::RpcController*, std::function<void ()>) ../../src/yb/rpc/proxy.cc:118
#24 0x7f4d24ebd480 in yb::master::MasterServiceProxy::GetMasterRegistrationAsync(yb::master::GetMasterRegistrationRequestPB const&, yb::master::GetMasterRegistrationResponsePB*, yb::rpc::RpcController*, std::function<void ()>) src/yb/master/master.proxy.cc:604
#25 0x7f4d29157e0f in SendRpc ../../src/yb/master/master_rpc.cc:110
#26 0x7f4d29154b93 in yb::master::GetLeaderMasterRpc::SendRpc() ../../src/yb/master/master_rpc.cc:209
#27 0x7f4d3204d3ea in std::shared_ptryb::rpc::RpcCommand yb::rpc::StartRpc<yb::master::GetLeaderMasterRpc, yb::Callback<void (yb::Status const&, yb::HostPort const&)>, std::vector<std::vector<yb::HostPort, std::allocatoryb::HostPort >, std::allocator<std::vector<yb::HostPort, std::allocatoryb::HostPort > > >&, std::chrono::time_point<yb::CoarseMonoClock, std::chrono::duration<long, std::ratio<1l, 1000000000l> > >&, yb::rpc::Messenger*&, yb::rpc::ProxyCache*, yb::rpc::Rpcs*>(yb::Callback<void (yb::Status const&, yb::HostPort const&)>&&, std::vector<std::vector<yb::HostPort, std::allocatoryb::HostPort >, std::allocator<std::vector<yb::HostPort, std::allocatoryb::HostPort > > >&, std::chrono::time_point<yb::CoarseMonoClock, std::chrono::duration<long, std::ratio<1l, 1000000000l> > >&, yb::rpc::Messenger*&, yb::rpc::ProxyCache*&&, yb::rpc::Rpcs*&&) ../../src/yb/rpc/rpc.h:265
#28 0x7f4d32008097 in yb::ExternalMiniCluster::GetPeerMasterIndex(int*, bool) ../../src/yb/integration-tests/external_mini_cluster.cc:1302
#29 0x7f4d3200c15d in yb::ExternalMiniCluster::GetLeaderMasterIndex(int*) ../../src/yb/integration-tests/external_mini_cluster.cc:1276
#30 0x7f4d3200c9f4 in yb::ExternalMiniCluster::GetLeaderMaster() ../../src/yb/integration-tests/external_mini_cluster.cc:1335
#31 0x7f4d32023773 in yb::ExternalMiniCluster::AddTabletServer(bool, std::vector<std::string, std::allocatorstd::string > const&) ../../src/yb/integration-tests/external_mini_cluster.cc:1058
#32 0x7f4d3203d5f5 in yb::ExternalMiniCluster::Start(yb::rpc::Messenger*) ../../src/yb/integration-tests/external_mini_cluster.cc:348
#33 0x7f4d3260d7ff in yb::pggate::PggateTest::CreateCluster(int) ../../src/yb/yql/pggate/test/pggate_test.cc:128
#34 0x7f4d3260e4f7 in yb::pggate::PggateTest::Init(char const*, int) ../../src/yb/yql/pggate/test/pggate_test.cc:98
#35 0x40825f in yb::pggate::PggateTestCatalog_TestDml_Test::TestBody() ../../src/yb/yql/pggate/test/pggate_test_catalog.cc:30
#36 0x7f4d20b3235d in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::)(), char const) /nfusr/dev-server/mbautin2/code/yugabyte-db-thirdparty-devtoolset-8/src/gmock-1.8.0/googletest/src/gtest.cc:2402
#37 0x7f4d20b3235d in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::)(), char const) /nfusr/dev-server/mbautin2/code/yugabyte-db-thirdparty-devtoolset-8/src/gmock-1.8.0/googletest/src/gtest.cc:2438
#38 0x7f4d20b28e71 in testing::Test::Run() /nfusr/dev-server/mbautin2/code/yugabyte-db-thirdparty-devtoolset-8/src/gmock-1.8.0/googletest/src/gtest.cc:2474
#39 0x7f4d20b28e71 in testing::Test::Run() /nfusr/dev-server/mbautin2/code/yugabyte-db-thirdparty-devtoolset-8/src/gmock-1.8.0/googletest/src/gtest.cc:2465
#40 0x7f4d20b28fb7 in testing::TestInfo::Run() /nfusr/dev-server/mbautin2/code/yugabyte-db-thirdparty-devtoolset-8/src/gmock-1.8.0/googletest/src/gtest.cc:2656
#41 0x7f4d20b28fb7 in testing::TestInfo::Run() /nfusr/dev-server/mbautin2/code/yugabyte-db-thirdparty-devtoolset-8/src/gmock-1.8.0/googletest/src/gtest.cc:2630
#42 0x7f4d20b29094 in testing::TestCase::Run() /nfusr/dev-server/mbautin2/code/yugabyte-db-thirdparty-devtoolset-8/src/gmock-1.8.0/googletest/src/gtest.cc:2774
#43 0x7f4d20b29094 in testing::TestCase::Run() /nfusr/dev-server/mbautin2/code/yugabyte-db-thirdparty-devtoolset-8/src/gmock-1.8.0/googletest/src/gtest.cc:2759
#44 0x7f4d20b2956b in testing::internal::UnitTestImpl::RunAllTests() /nfusr/dev-server/mbautin2/code/yugabyte-db-thirdparty-devtoolset-8/src/gmock-1.8.0/googletest/src/gtest.cc:4649
#45 0x7f4d20b2956b in testing::internal::UnitTestImpl::RunAllTests() /nfusr/dev-server/mbautin2/code/yugabyte-db-thirdparty-devtoolset-8/src/gmock-1.8.0/googletest/src/gtest.cc:4551
#46 0x7f4d20b3287d in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::)(), char const) /nfusr/dev-server/mbautin2/code/yugabyte-db-thirdparty-devtoolset-8/src/gmock-1.8.0/googletest/src/gtest.cc:2402
#47 0x7f4d20b3287d in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::)(), char const) /nfusr/dev-server/mbautin2/code/yugabyte-db-thirdparty-devtoolset-8/src/gmock-1.8.0/googletest/src/gtest.cc:2438

SUMMARY: AddressSanitizer: use-after-poison ../../src/yb/util/trace.cc:289 in yb::Trace::NewEntry(int, char const*, int, yb::MonoTime)
Shadow bytes around the buggy address:
0x0c4a7fff98d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c4a7fff98e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c4a7fff98f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c4a7fff9900: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c4a7fff9910: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x0c4a7fff9920: 00 00 00[f7]f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
0x0c4a7fff9930: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
0x0c4a7fff9940: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
0x0c4a7fff9950: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
0x0c4a7fff9960: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
0x0c4a7fff9970: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Shadow gap: cc
==7490==ABORTING