Fix infinite loop in getVlanRangesByIface #5
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This code gets triggered on handlePortStatusUpdate and would result in the opflex agent getting stuck in an infinite loop because the iterator is not being incremented in the while loop. --------------------------------------------------------------------------- Simple way to repro the issue is to add or del the patch ports on br-fabric example: ovs-vsctl del-port br-fabric qpi2b4b79dd-6e ovs-vsctl add-port br-fabric qpi2b4b79dd-6e -- set interface qpi2b4b79dd-6e type=patch options:peer=qpf2b4b79dd-6e Or ovs-vsctl del-port br-fabric patch-fabric-ex ovs-vsctl add-port br-fabric patch-fabric-ex -- set interface patch-fabric-ex type=patch options:peer=patch-ex-fabric --------------------------------------------------------------------------- Verified we come out of the loop with the fix. Change-Id: Ia330e40942083e9d289e256b951cc803a35e4864 Signed-off-by: Madhu Challa <challa@gmail.com> (cherry picked from commit f03095e)
mchalla
requested review from
mandeepdhami,
msandhu2014,
tbachman and
tomflynn
gautvenk
added a commit
that referenced
this pull request
Jun 24, 2020
355 WARNING: ThreadSanitizer: data race (pid=8183) 356 Write of size 8 at 0x7b7800000548 by thread T11 (mutexes: write M1342): 357 #0 opflex::engine::internal::OpflexPool::setTunnelMac(opflex::modb::MAC const&) include/opflex/engine/internal/OpflexPool.h:338 (libopflex.so.0+0x115151) 358 #1 opflex::engine::Processor::setTunnelMac(opflex::modb::MAC const&) /home/noiro/work/opflex/libopflex/engine/Processor.cpp:584 (libopflex.so.0+0x115151) 359 #2 opflex::ofcore::OFFramework::setTunnelMac(opflex::modb::MAC const&) /home/noiro/work/opflex/libopflex/ofcore/OFFramework.cpp:103 (libopflex.so.0+0x18dcca) 360 #3 opflexagent::Agent::setUplinkMac(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) lib/Agent.cpp:795 (libopflex_agent.so.0+0x253e33) 361 #4 opflexagent::TunnelEpManager::on_timer(boost::system::error_code const&) lib/TunnelEpManager.cpp:281 (libopflex_agent.so.0+0x22c49f) 362 #5 boost::asio::detail::wait_handler<boost::_bi::bind_t<void, boost::_mfi::mf1<void, opflexagent::TunnelEpManager, boost::system::error_code const&>, boost::_bi::list2<boost::_bi::value<opflexagent::T 363 #6 boost::asio::detail::task_io_service_operation::complete(boost::asio::detail::task_io_service&, boost::system::error_code const&, unsigned long) /usr/include/boost/asio/detail/task_io_service_opera 364 #7 boost::asio::detail::task_io_service::do_run_one(boost::asio::detail::scoped_lock<boost::asio::detail::posix_mutex>&, boost::asio::detail::task_io_service_thread_info&, boost::system::error_code co 365 #8 boost::asio::detail::task_io_service::run(boost::system::error_code&) /usr/include/boost/asio/detail/impl/task_io_service.ipp:149 (libopflex.so.0+0x16b982) 366 #9 boost::asio::io_service::run() /usr/include/boost/asio/impl/io_service.ipp:59 (libopflex_agent.so.0+0x256259) 367 #10 operator() lib/Agent.cpp:606 (libopflex_agent.so.0+0x256259) 368 #11 __invoke_impl<void, opflexagent::Agent::start()::<lambda()> > /usr/include/c++/9/bits/invoke.h:60 (libopflex_agent.so.0+0x256259) 369 #12 __invoke<opflexagent::Agent::start()::<lambda()> > /usr/include/c++/9/bits/invoke.h:95 (libopflex_agent.so.0+0x256259) 370 #13 _M_invoke<0> /usr/include/c++/9/thread:244 (libopflex_agent.so.0+0x256259) 371 #14 operator() /usr/include/c++/9/thread:251 (libopflex_agent.so.0+0x256259) 372 #15 _M_run /usr/include/c++/9/thread:195 (libopflex_agent.so.0+0x256259) 373 #16 <null> <null> (libstdc++.so.6+0xd086f) 374 375 Previous read of size 8 at 0x7b7800000548 by thread T4: 376 #0 opflex::engine::internal::OpflexPool::getTunnelMac() include/opflex/engine/internal/OpflexPool.h:342 (libopflex.so.0+0x1457d5) 377 #1 opflex::engine::internal::OpflexPEHandler::connected() /home/noiro/work/opflex/libopflex/engine/OpflexPEHandler.cpp:139 (libopflex.so.0+0x1457d5) 378 #2 opflex::engine::internal::OpflexClientConnection::on_state_change(yajr::Peer*, void*, yajr::StateChange::To, int) /home/noiro/work/opflex/libopflex/engine/OpflexClientConnection.cpp:158 (libopflex. 379 #3 yajr::comms::internal::CommunicationPeer::onConnect() /home/noiro/work/opflex/libopflex/comms/CommunicationPeer.cpp:116 (libopflex.so.0+0xe2aac) 380 #4 yajr::comms::internal::on_active_connection(uv_connect_s*, int) /home/noiro/work/opflex/libopflex/comms/active_connection.cpp:185 (libopflex.so.0+0xb5d00) 381 #5 <null> <null> (libuv.so.1+0x162c3) 382 #6 <null> <null> (libtsan.so.0+0x2aba6) Signed-off-by: Gautam Venkataramanan <gautam.chennai@gmail.com>
gautvenk
added a commit
that referenced
this pull request
Jun 24, 2020
29266 WARNING: ThreadSanitizer: data race (pid=28611) 29267 Write of size 1 at 0x7b5400000e98 by main thread: 29268 #0 yajr::comms::internal::Peer::LoopData::destroy(bool) /home/noiro/work/opflex/libopflex/comms/loopdata.cpp:128 (libopflex.so.0+0xf5146) 29269 #1 yajr::finiLoop(uv_loop_s*) /home/noiro/work/opflex/libopflex/comms/common.cpp:33 (libopflex.so.0+0xbd27e) 29270 #2 opflexagent::OvsdbConnection::stop() ovs/OvsdbConnection.cpp:71 (agent_test+0x72ecc0) 29271 #3 opflexagent::OvsdbConnection_test::verify_connect::test_method() ovs/test/OvsdbConnection_test.cpp:102 (agent_test+0x594d65) 29272 #4 verify_connect_invoker ovs/test/OvsdbConnection_test.cpp:50 (agent_test+0x59720c) 29273 #5 boost::detail::function::void_function_invoker0<void (*)(), void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:118 (agent_test+0x13cd2c) 29274 #6 boost::detail::function::function_obj_invoker0<boost::detail::forward, int>::invoke(boost::detail::function::function_buffer&) <null> (libboost_unit_test_framework.so.1.65.1+0x4b2cd) 29275 #7 __libc_start_main <null> (libc.so.6+0x21b96) 29276 29277 Previous read of size 1 at 0x7b5400000e98 by thread T6: 29278 #0 yajr::comms::internal::Peer::LoopData::onPrepareLoop() /home/noiro/work/opflex/libopflex/comms/loopdata.cpp:38 (libopflex.so.0+0xf415f) 29279 #1 yajr::comms::internal::Peer::LoopData::onPrepareLoop(uv_prepare_s*) /home/noiro/work/opflex/libopflex/comms/loopdata.cpp:112 (libopflex.so.0+0xf48ed) 29280 #2 uv__run_prepare <null> (libuv.so.1+0x11d73) 29281 #3 <null> <null> (libtsan.so.0+0x2aba6) 29282 29283 Location is heap block of size 552 at 0x7b5400000c80 allocated by main thread: 29284 #0 operator new(unsigned long, std::nothrow_t const&) <null> (libtsan.so.0+0x771aa) 29285 #1 yajr::initLoop(uv_loop_s*) /home/noiro/work/opflex/libopflex/comms/common.cpp:22 (libopflex.so.0+0xbd060) 29286 #2 opflexagent::OvsdbConnection::start() ovs/OvsdbConnection.cpp:38 (agent_test+0x72f9a5) 29287 #3 opflexagent::OvsdbConnection_test::verify_connect::test_method() ovs/test/OvsdbConnection_test.cpp:51 (agent_test+0x59192e) 29288 #4 verify_connect_invoker ovs/test/OvsdbConnection_test.cpp:50 (agent_test+0x59720c) 29289 #5 boost::detail::function::void_function_invoker0<void (*)(), void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:118 (agent_test+0x13cd2c) 29290 #6 boost::detail::function::function_obj_invoker0<boost::detail::forward, int>::invoke(boost::detail::function::function_buffer&) <null> (libboost_unit_test_framework.so.1.65.1+0x4b2cd) 29291 #7 __libc_start_main <null> (libc.so.6+0x21b96) Signed-off-by: Gautam Venkataramanan <gautam.chennai@gmail.com>
gautvenk
added a commit
that referenced
this pull request
Jun 25, 2020
- making refCount atomic 28944 Read of size 8 at 0x7b5400000ea0 by thread T6: 28945 #0 yajr::comms::internal::Peer::LoopData::onPrepareLoop() /home/noiro/work/opflex/libopflex/comms/loopdata.cpp:38 (libopflex.so.0+0xf4188) 28946 #1 yajr::comms::internal::Peer::LoopData::onPrepareLoop(uv_prepare_s*) /home/noiro/work/opflex/libopflex/comms/loopdata.cpp:113 (libopflex.so.0+0xf494d) 28947 #2 uv__run_prepare <null> (libuv.so.1+0x11d73) 28948 #3 <null> <null> (libtsan.so.0+0x2aba6) 28949 28950 Previous write of size 8 at 0x7b5400000ea0 by main thread: 28951 #0 yajr::comms::internal::Peer::LoopData::down() /home/noiro/work/opflex/libopflex/comms/loopdata.cpp:238 (libopflex.so.0+0xf4bef) 28952 #1 yajr::comms::internal::Peer::LoopData::destroy(bool) /home/noiro/work/opflex/libopflex/comms/loopdata.cpp:130 (libopflex.so.0+0xf5211) 28953 #2 yajr::finiLoop(uv_loop_s*) /home/noiro/work/opflex/libopflex/comms/common.cpp:33 (libopflex.so.0+0xbd27e) 28954 #3 opflexagent::OvsdbConnection::stop() ovs/OvsdbConnection.cpp:71 (agent_test+0x72ecc0) 28955 #4 opflexagent::OvsdbConnection_test::verify_connect::test_method() ovs/test/OvsdbConnection_test.cpp:102 (agent_test+0x594d65) 28956 #5 verify_connect_invoker ovs/test/OvsdbConnection_test.cpp:50 (agent_test+0x59720c) 28957 #6 boost::detail::function::void_function_invoker0<void (*)(), void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:118 (agent_test+0x13cd2c) 28958 #7 boost::detail::function::function_obj_invoker0<boost::detail::forward, int>::invoke(boost::detail::function::function_buffer&) <null> (libboost_unit_test_framework.so.1.65.1+0x4b2cd) 28959 #8 __libc_start_main <null> (libc.so.6+0x21b96) 28960 28961 Location is heap block of size 552 at 0x7b5400000c80 allocated by main thread: 28962 #0 operator new(unsigned long, std::nothrow_t const&) <null> (libtsan.so.0+0x771aa) 28963 #1 yajr::initLoop(uv_loop_s*) /home/noiro/work/opflex/libopflex/comms/common.cpp:22 (libopflex.so.0+0xbd060) 28964 #2 opflexagent::OvsdbConnection::start() ovs/OvsdbConnection.cpp:38 (agent_test+0x72f9a5) 28965 #3 opflexagent::OvsdbConnection_test::verify_connect::test_method() ovs/test/OvsdbConnection_test.cpp:51 (agent_test+0x59192e) 28966 #4 verify_connect_invoker ovs/test/OvsdbConnection_test.cpp:50 (agent_test+0x59720c) 28967 #5 boost::detail::function::void_function_invoker0<void (*)(), void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:118 (agent_test+0x13cd2c) 28968 #6 boost::detail::function::function_obj_invoker0<boost::detail::forward, int>::invoke(boost::detail::function::function_buffer&) <null> (libboost_unit_test_framework.so.1.65.1+0x4b2cd) 28969 #7 __libc_start_main <null> (libc.so.6+0x21b96) - Protecting peers list with peerMutex 29314 Read of size 8 at 0x7b5400000ca0 by main thread (mutexes: write M217157256601723712): 29315 #0 boost::intrusive::list_impl<boost::intrusive::bhtraits<yajr::comms::internal::Peer, boost::intrusive::list_node_traits<void*>, (boost::intrusive::link_mode_type)2, boost::intrusive::dft_tag, 1u>, 29316 #1 void boost::intrusive::list_impl<boost::intrusive::bhtraits<yajr::comms::internal::Peer, boost::intrusive::list_node_traits<void*>, (boost::intrusive::link_mode_type)2, boost::intrusive::dft_tag, 29317 #2 yajr::comms::internal::Peer::LoopData::destroy(bool) /home/noiro/work/opflex/libopflex/comms/loopdata.cpp:134 (libopflex.so.0+0xf51fb) 29318 #3 yajr::finiLoop(uv_loop_s*) /home/noiro/work/opflex/libopflex/comms/common.cpp:33 (libopflex.so.0+0xbd27e) 29319 #4 opflexagent::OvsdbConnection::stop() ovs/OvsdbConnection.cpp:71 (agent_test+0x72ecc0) 29320 #5 opflexagent::OvsdbConnection_test::verify_connect::test_method() ovs/test/OvsdbConnection_test.cpp:102 (agent_test+0x594d65) 29321 #6 verify_connect_invoker ovs/test/OvsdbConnection_test.cpp:50 (agent_test+0x59720c) 29322 #7 boost::detail::function::void_function_invoker0<void (*)(), void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:118 (agent_test+0x13cd2c) 29323 #8 boost::detail::function::function_obj_invoker0<boost::detail::forward, int>::invoke(boost::detail::function::function_buffer&) <null> (libboost_unit_test_framework.so.1.65.1+0x4b2cd) 29324 #9 __libc_start_main <null> (libc.so.6+0x21b96) 29325 29326 Previous write of size 8 at 0x7b5400000ca0 by thread T6: 29327 #0 void boost::intrusive::list_impl<boost::intrusive::bhtraits<yajr::comms::internal::Peer, boost::intrusive::list_node_traits<void*>, (boost::intrusive::link_mode_type)2, boost::intrusive::dft_tag, 29328 #1 yajr::comms::internal::Peer::LoopData::onPrepareLoop() /home/noiro/work/opflex/libopflex/comms/loopdata.cpp:64 (libopflex.so.0+0xf42a2) 29329 #2 yajr::comms::internal::Peer::LoopData::onPrepareLoop(uv_prepare_s*) /home/noiro/work/opflex/libopflex/comms/loopdata.cpp:112 (libopflex.so.0+0xf490d) 29330 #3 uv__run_prepare <null> (libuv.so.1+0x11d73) 29331 #4 <null> <null> (libtsan.so.0+0x2aba6) 29332 29333 Location is heap block of size 552 at 0x7b5400000c80 allocated by main thread: 29334 #0 operator new(unsigned long, std::nothrow_t const&) <null> (libtsan.so.0+0x771aa) 29335 #1 yajr::initLoop(uv_loop_s*) /home/noiro/work/opflex/libopflex/comms/common.cpp:22 (libopflex.so.0+0xbd060) 29336 #2 opflexagent::OvsdbConnection::start() ovs/OvsdbConnection.cpp:38 (agent_test+0x72f9a5) 29337 #3 opflexagent::OvsdbConnection_test::verify_connect::test_method() ovs/test/OvsdbConnection_test.cpp:51 (agent_test+0x59192e) 29338 #4 verify_connect_invoker ovs/test/OvsdbConnection_test.cpp:50 (agent_test+0x59720c) 29339 #5 boost::detail::function::void_function_invoker0<void (*)(), void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:118 (agent_test+0x13cd2c) 29340 #6 boost::detail::function::function_obj_invoker0<boost::detail::forward, int>::invoke(boost::detail::function::function_buffer&) <null> (libboost_unit_test_framework.so.1.65.1+0x4b2cd) 29341 #7 __libc_start_main <null> (libc.so.6+0x21b96) - Making peerMutex a recursive mutex (acquired in #11 and then in #5 again) #4 0x00005555555ea3c6 in opflex::util::LockGuard::LockGuard(pthread_mutex_t*) () at LockGuard.cpp:24 #5 0x00005555555b194b in yajr::comms::internal::Peer::LoopData::addPeer (peer=0x7b5400000780, peerState=yajr::comms::internal::Peer::LoopData::LISTENING, uv_loop=0x7fffffffcb70) at ../include/opflex/yajr/internal/comms.hpp:192 #6 yajr::comms::internal::Peer::insert (this=this@entry=0x7b5400000780, peerState=peerState@entry=yajr::comms::internal::Peer::LoopData::LISTENING) at peer.cpp:82 #7 0x00005555555a7a55 in yajr::comms::internal::ListeningTcpPeer::retry (this=0x7b5400000780) at passive_listener.cpp:186 #8 0x00005555555cca1c in yajr::comms::internal::Peer::LoopData::RetryPeer::operator() (this=<optimized out>, peer=0x7b5400000780) at loopdata.cpp:212 #9 0x00005555555ccb3e in boost::intrusive::list_impl<boost::intrusive::bhtraits<yajr::comms::internal::Peer, boost::intrusive::list_node_traits<void*>, (boost::intrusive::link_mode_type)2, boost::intrusive::dft_tag, 1u>, unsigned long, false, void>::clear_and_dispose<yajr::comms::internal::Peer::LoopData::RetryPeer> (disposer=..., this=0x7b5400000030) at /usr/include/boost/move/detail/meta_utils.hpp:267 #10 yajr::comms::internal::Peer::LoopData::onPrepareLoop (this=0x7b5400000000) at loopdata.cpp:62 #11 0x00005555555cd25e in yajr::comms::internal::Peer::LoopData::onPrepareLoop (h=0x7b5400000080) at loopdata.cpp:112 #12 0x00007ffff674cd74 in uv.run_prepare () from /usr/lib/x86_64-linux-gnu/libuv.so.1 #13 0x00007ffff6746e58 in uv_run () from /usr/lib/x86_64-linux-gnu/libuv.so.1 #14 0x000055555557f726 in asynchronous_sockets::CommsFixture::loop_until_final (num_events=0, timeout=7200, should_timeout=false, transient_peers=..., post_conditions=0x555555578100 <asynchronous_sockets::pc_successful_connect()>, final_peers=..., this=0x7fffffffca60) at test/comms_test.cpp:380 #15 asynchronous_sockets::STABLE_test_ipv4::test_method() () at test/comms_test.cpp:518 Signed-off-by: Gautam Venkataramanan <gautam.chennai@gmail.com>
gautvenk
added a commit
that referenced
this pull request
Jul 23, 2020
…ver connection set in few cases
WARNING: ThreadSanitizer: data race (pid=24355)
Read of size 8 at 0x7b0c00012140 by thread T13:
#0 opflex::engine::internal::OpflexListener::sendTimeouts() /home/noiro/work/opflex/libopflex/engine/OpflexListener.cpp:213 (libopflex.so.0+0x154c70)
#1 opflex::engine::internal::GbpOpflexServerImpl::on_timer(boost::system::error_code const&) /home/noiro/work/opflex/libopflex/engine/GbpOpflexServer.cpp:146 (libopflex.so.0+0x163808)
#2 operator() /home/noiro/work/opflex/libopflex/engine/GbpOpflexServer.cpp:119 (libopflex.so.0+0x163dd4)
#3 operator() /usr/include/boost/asio/detail/bind_handler.hpp:47 (libopflex.so.0+0x163dd4)
#4 asio_handler_invoke<boost::asio::detail::binder1<opflex::engine::internal::GbpOpflexServerImpl::start()::<lambda(const boost::system::error_code&)>, boost::system::error_code> > /usr/include/boost/asio
/handler_invoke_hook.hpp:69 (libopflex.so.0+0x163dd4)
#5 invoke<boost::asio::detail::binder1<opflex::engine::internal::GbpOpflexServerImpl::start()::<lambda(const boost::system::error_code&)>, boost::system::error_code>, opflex::engine::internal::GbpOpflexSe
rverImpl::start()::<lambda(const boost::system::error_code&)> > /usr/include/boost/asio/detail/handler_invoke_helpers.hpp:37 (libopflex.so.0+0x163dd4)
#6 do_complete /usr/include/boost/asio/detail/wait_handler.hpp:70 (libopflex.so.0+0x163dd4)
#7 boost::asio::detail::task_io_service_operation::complete(boost::asio::detail::task_io_service&, boost::system::error_code const&, unsigned long) /usr/include/boost/asio/detail/task_io_service_operation
.hpp:38 (libopflex.so.0+0x16b292)
Previous write of size 8 at 0x7b0c00012140 by thread T14 (mutexes: write M620, write M198):
#9 opflex::engine::internal::OpflexListener::on_new_connection(yajr::Listener*, void*, int) /home/noiro/work/opflex/libopflex/engine/OpflexListener.cpp:162 (libopflex.so.0+0x15589f)
#10 yajr::comms::internal::ListeningPeer::getConnectionHandlerData() ../include/opflex/yajr/internal/comms.hpp:1116 (libopflex.so.0+0xb98a8)
#11 yajr::comms::internal::ListeningTcpPeer::getNewPassive() /home/noiro/work/opflex/libopflex/comms/passive_listener.cpp:333 (libopflex.so.0+0xb98a8)
#12 yajr::comms::internal::on_passive_connection(uv_stream_s*, int) /home/noiro/work/opflex/libopflex/comms/passive_listener.cpp:296 (libopflex.so.0+0xb7074)
Location is heap block of size 40 at 0x7b0c00012120 allocated by thread T14:
#9 opflex::engine::internal::OpflexListener::on_new_connection(yajr::Listener*, void*, int) /home/noiro/work/opflex/libopflex/engine/OpflexListener.cpp:162 (libopflex.so.0+0x15589f)
#10 yajr::comms::internal::ListeningPeer::getConnectionHandlerData() ../include/opflex/yajr/internal/comms.hpp:1116 (libopflex.so.0+0xb98a8)
#11 yajr::comms::internal::ListeningTcpPeer::getNewPassive() /home/noiro/work/opflex/libopflex/comms/passive_listener.cpp:333 (libopflex.so.0+0xb98a8)
#12 yajr::comms::internal::on_passive_connection(uv_stream_s*, int) /home/noiro/work/opflex/libopflex/comms/passive_listener.cpp:296 (libopflex.so.0+0xb7074)
SUMMARY: ThreadSanitizer: data race /home/noiro/work/opflex/libopflex/engine/OpflexListener.cpp:213 in opflex::engine::internal::OpflexListener::sendTimeouts()
Signed-off-by: Gautam Venkataramanan <gautam.chennai@gmail.com>
gautvenk
added a commit
that referenced
this pull request
Aug 7, 2020
==================
WARNING: ThreadSanitizer: data race (pid=19679)
Read of size 1 at 0x7b1000034990 by main thread:
#0 boost::asio::detail::deadline_timer_service<boost::asio::time_traits<boost::posix_time::ptime> >::cancel(boost::asio::detail::deadline_timer_service<boost::asio::time_traits<boost::posix_time::ptime> >
::implementation_type&, boost::system::error_code&) /usr/include/boost/asio/detail/deadline_timer_service.hpp:100 (libopflex.so.0+0x1630e3)
#1 boost::asio::deadline_timer_service<boost::posix_time::ptime, boost::asio::time_traits<boost::posix_time::ptime> >::cancel(boost::asio::detail::deadline_timer_service<boost::asio::time_traits<boost::po
six_time::ptime> >::implementation_type&, boost::system::error_code&) /usr/include/boost/asio/deadline_timer_service.hpp:96 (libopflex.so.0+0x1630e3)
#2 boost::asio::basic_deadline_timer<boost::posix_time::ptime, boost::asio::time_traits<boost::posix_time::ptime>, boost::asio::deadline_timer_service<boost::posix_time::ptime, boost::asio::time_traits<bo
ost::posix_time::ptime> > >::cancel() /usr/include/boost/asio/basic_deadline_timer.hpp:216 (libopflex.so.0+0x1630e3)
#3 opflex::engine::internal::GbpOpflexServerImpl::stop() /home/noiro/work/opflex/libopflex/engine/GbpOpflexServer.cpp:129 (libopflex.so.0+0x1630e3)
#4 opflex::test::GbpOpflexServer::stop() /home/noiro/work/opflex/libopflex/engine/GbpOpflexServer.cpp:53 (libopflex.so.0+0x1631f2)
#5 main server/opflex_server.cpp:297 (opflex_server+0x22fc3)
Previous write of size 1 at 0x7b1000034990 by thread T12:
#0 boost::asio::detail::deadline_timer_service<boost::asio::time_traits<boost::posix_time::ptime> >::cancel(boost::asio::detail::deadline_timer_service<boost::asio::time_traits<boost::posix_time::ptime> >
::implementation_type&, boost::system::error_code&) /usr/include/boost/asio/detail/deadline_timer_service.hpp:109 (libopflex.so.0+0x163a50)
#1 boost::asio::detail::deadline_timer_service<boost::asio::time_traits<boost::posix_time::ptime> >::cancel(boost::asio::detail::deadline_timer_service<boost::asio::time_traits<boost::posix_time::ptime> >
::implementation_type&, boost::system::error_code&) /usr/include/boost/asio/detail/deadline_timer_service.hpp:98 (libopflex.so.0+0x163a50)
#2 boost::asio::detail::deadline_timer_service<boost::asio::time_traits<boost::posix_time::ptime> >::expires_at(boost::asio::detail::deadline_timer_service<boost::asio::time_traits<boost::posix_time::ptim
e> >::implementation_type&, boost::posix_time::ptime const&, boost::system::error_code&) /usr/include/boost/asio/detail/deadline_timer_service.hpp:144 (libopflex.so.0+0x163a50)
#3 boost::asio::deadline_timer_service<boost::posix_time::ptime, boost::asio::time_traits<boost::posix_time::ptime> >::expires_at(boost::asio::detail::deadline_timer_service<boost::asio::time_traits<boost
::posix_time::ptime> >::implementation_type&, boost::posix_time::ptime const&, boost::system::error_code&) /usr/include/boost/asio/deadline_timer_service.hpp:116 (libopflex.so.0+0x163a50)
#4 boost::asio::basic_deadline_timer<boost::posix_time::ptime, boost::asio::time_traits<boost::posix_time::ptime>, boost::asio::deadline_timer_service<boost::posix_time::ptime, boost::asio::time_traits<bo
ost::posix_time::ptime> > >::expires_at(boost::posix_time::ptime const&) /usr/include/boost/asio/basic_deadline_timer.hpp:344 (libopflex.so.0+0x163a50)
#5 opflex::engine::internal::GbpOpflexServerImpl::on_timer(boost::system::error_code const&) /home/noiro/work/opflex/libopflex/engine/GbpOpflexServer.cpp:149 (libopflex.so.0+0x163a50)
#6 operator() /home/noiro/work/opflex/libopflex/engine/GbpOpflexServer.cpp:119 (libopflex.so.0+0x163f24)
#7 operator() /usr/include/boost/asio/detail/bind_handler.hpp:47 (libopflex.so.0+0x163f24)
#8 asio_handler_invoke<boost::asio::detail::binder1<opflex::engine::internal::GbpOpflexServerImpl::start()::<lambda(const boost::system::error_code&)>, boost::system::error_code> > /usr/include/boost/asio
/handler_invoke_hook.hpp:69 (libopflex.so.0+0x163f24)
#9 invoke<boost::asio::detail::binder1<opflex::engine::internal::GbpOpflexServerImpl::start()::<lambda(const boost::system::error_code&)>, boost::system::error_code>, opflex::engine::internal::GbpOpflexSe
rverImpl::start()::<lambda(const boost::system::error_code&)> > /usr/include/boost/asio/detail/handler_invoke_helpers.hpp:37 (libopflex.so.0+0x163f24)
#10 do_complete /usr/include/boost/asio/detail/wait_handler.hpp:70 (libopflex.so.0+0x163f24)
#11 boost::asio::detail::task_io_service_operation::complete(boost::asio::detail::task_io_service&, boost::system::error_code const&, unsigned long) /usr/include/boost/asio/detail/task_io_service_operatio
n.hpp:38 (libopflex.so.0+0x16b3e2)
#12 boost::asio::detail::task_io_service::do_run_one(boost::asio::detail::scoped_lock<boost::asio::detail::posix_mutex>&, boost::asio::detail::task_io_service_thread_info&, boost::system::error_code const
&) /usr/include/boost/asio/detail/impl/task_io_service.ipp:372 (libopflex.so.0+0x16b3e2)
#13 boost::asio::detail::task_io_service::run(boost::system::error_code&) /usr/include/boost/asio/detail/impl/task_io_service.ipp:149 (libopflex.so.0+0x16b3e2)
#14 boost::asio::io_service::run() /usr/include/boost/asio/impl/io_service.ipp:59 (libopflex.so.0+0x1625ff)
#15 operator() /home/noiro/work/opflex/libopflex/engine/GbpOpflexServer.cpp:121 (libopflex.so.0+0x1625ff)
#16 __invoke_impl<void, opflex::engine::internal::GbpOpflexServerImpl::start()::<lambda()> > /usr/include/c++/9/bits/invoke.h:60 (libopflex.so.0+0x1625ff)
#17 __invoke<opflex::engine::internal::GbpOpflexServerImpl::start()::<lambda()> > /usr/include/c++/9/bits/invoke.h:95 (libopflex.so.0+0x1625ff)
#18 _M_invoke<0> /usr/include/c++/9/thread:244 (libopflex.so.0+0x1625ff)
#19 operator() /usr/include/c++/9/thread:251 (libopflex.so.0+0x1625ff)
#20 _M_run /usr/include/c++/9/thread:195 (libopflex.so.0+0x1625ff)
#21 <null> <null> (libstdc++.so.6+0xd086f)
Signed-off-by: Gautam Venkataramanan <gautam.chennai@gmail.com>
This was referenced Aug 7, 2020
gautvenk
added a commit
that referenced
this pull request
Sep 3, 2020
WARNING: ThreadSanitizer: data race (pid=28641)
Read of size 1 at 0x7b1000012ad0 by thread T11:
#8 opflexagent::TunnelEpManager::on_timer(boost::system::error_code const&) lib/TunnelEpManager.cpp:172 (libopflex_agent.so.0+0x243cff)
Previous write of size 1 at 0x7b1000012ad0 by main thread (mutexes: write M180):
#4 opflexagent::TunnelEpManager::stop() lib/TunnelEpManager.cpp:88 (libopflex_agent.so.0+0x24393e)
#5 opflexagent::OVSRenderer::stop() ovs/OVSRenderer.cpp:285 (libopflex_agent_renderer_openvswitch.so+0xcb84b)
#6 opflexagent::Agent::stop() lib/Agent.cpp:718 (libopflex_agent.so.0+0x2723d8)
#7 AgentLauncher::run() cmd/opflex_agent.cpp:128 (opflex_agent+0x3a4a7)
#8 main cmd/opflex_agent.cpp:322 (opflex_agent+0x16dc2)
Location is heap block of size 64 at 0x7b1000012ac0 allocated by main thread:
#0 operator new(unsigned long) <null> (libtsan.so.0+0x77cf2)
#1 opflexagent::TunnelEpManager::start() lib/TunnelEpManager.cpp:75 (libopflex_agent.so.0+0x247908)
#2 opflexagent::OVSRenderer::start() ovs/OVSRenderer.cpp:119 (libopflex_agent_renderer_openvswitch.so+0xd55cc)
#3 opflexagent::Agent::start() lib/Agent.cpp:601 (libopflex_agent.so.0+0x2783ea)
#4 AgentLauncher::run() cmd/opflex_agent.cpp:120 (opflex_agent+0x3a438)
#5 main cmd/opflex_agent.cpp:322 (opflex_agent+0x16dc2)
Mutex M180 (0x7fff82ef6dc8) created at:
#0 pthread_mutex_lock <null> (libtsan.so.0+0x41d5b)
#1 __gthread_mutex_lock /usr/include/x86_64-linux-gnu/c++/9/bits/gthr-default.h:749 (opflex_agent+0x3a3f0)
#2 std::mutex::lock() /usr/include/c++/9/bits/std_mutex.h:100 (opflex_agent+0x3a3f0)
#3 std::unique_lock<std::mutex>::lock() /usr/include/c++/9/bits/unique_lock.h:141 (opflex_agent+0x3a3f0)
#4 std::unique_lock<std::mutex>::unique_lock(std::mutex&) /usr/include/c++/9/bits/unique_lock.h:71 (opflex_agent+0x3a3f0)
#5 AgentLauncher::run() cmd/opflex_agent.cpp:115 (opflex_agent+0x3a3f0)
#6 main cmd/opflex_agent.cpp:322 (opflex_agent+0x16dc2)
Thread T11 (tid=28681, running) created by main thread at:
#0 pthread_create <null> (libtsan.so.0+0x2d3be)
#1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xd0b04)
#2 AgentLauncher::run() cmd/opflex_agent.cpp:120 (opflex_agent+0x3a438)
#3 main cmd/opflex_agent.cpp:322 (opflex_agent+0x16dc2)
Signed-off-by: Gautam Venkataramanan <gautam.chennai@gmail.com>
gautvenk
added a commit
that referenced
this pull request
Sep 11, 2020
WARNING: ThreadSanitizer: data race (pid=18054)
Read of size 1 at 0x7b1000014d90 by thread T11:
#8 opflexagent::InterfaceStatsManager::on_timer(boost::system::error_code const&) ovs/InterfaceStatsManager.cpp:105 (libopflex_agent_renderer_openvswitch.so+0x1e74e1)
#14 operator() lib/Agent.cpp:605 (libopflex_agent.so.0+0x275439)
#15 __invoke_impl<void, opflexagent::Agent::start()::<lambda()> > /usr/include/c++/9/bits/invoke.h:60 (libopflex_agent.so.0+0x275439)
#16 __invoke<opflexagent::Agent::start()::<lambda()> > /usr/include/c++/9/bits/invoke.h:95 (libopflex_agent.so.0+0x275439)
#17 _M_invoke<0> /usr/include/c++/9/thread:244 (libopflex_agent.so.0+0x275439)
#18 operator() /usr/include/c++/9/thread:251 (libopflex_agent.so.0+0x275439)
#19 _M_run /usr/include/c++/9/thread:195 (libopflex_agent.so.0+0x275439)
#20 <null> <null> (libstdc++.so.6+0xd086f)
Previous write of size 1 at 0x7b1000014d90 by main thread (mutexes: write M180):
#4 opflexagent::InterfaceStatsManager::stop() ovs/InterfaceStatsManager.cpp:91 (libopflex_agent_renderer_openvswitch.so+0x1e81af)
#5 opflexagent::OVSRenderer::stop() ovs/OVSRenderer.cpp:259 (libopflex_agent_renderer_openvswitch.so+0xcb92b)
#6 opflexagent::Agent::stop() lib/Agent.cpp:718 (libopflex_agent.so.0+0x273368)
#7 AgentLauncher::run() cmd/opflex_agent.cpp:128 (opflex_agent+0x3a4a7)
#8 main cmd/opflex_agent.cpp:322 (opflex_agent+0x16dc2)
Location is heap block of size 64 at 0x7b1000014d80 allocated by main thread:
#0 operator new(unsigned long) <null> (libtsan.so.0+0x77cf2)
#1 opflexagent::InterfaceStatsManager::start() ovs/InterfaceStatsManager.cpp:73 (libopflex_agent_renderer_openvswitch.so+0x1e9646)
#2 opflexagent::OVSRenderer::start() ovs/OVSRenderer.cpp:187 (libopflex_agent_renderer_openvswitch.so+0xd5508)
#3 opflexagent::Agent::start() lib/Agent.cpp:601 (libopflex_agent.so.0+0x27937a)
#4 AgentLauncher::run() cmd/opflex_agent.cpp:120 (opflex_agent+0x3a438)
#5 main cmd/opflex_agent.cpp:322 (opflex_agent+0x16dc2)
Mutex M180 (0x7ffc4bcb7548) created at:
#0 pthread_mutex_lock <null> (libtsan.so.0+0x41d5b)
#1 __gthread_mutex_lock /usr/include/x86_64-linux-gnu/c++/9/bits/gthr-default.h:749 (opflex_agent+0x3a3f0)
#2 std::mutex::lock() /usr/include/c++/9/bits/std_mutex.h:100 (opflex_agent+0x3a3f0)
#3 std::unique_lock<std::mutex>::lock() /usr/include/c++/9/bits/unique_lock.h:141 (opflex_agent+0x3a3f0)
#4 std::unique_lock<std::mutex>::unique_lock(std::mutex&) /usr/include/c++/9/bits/unique_lock.h:71 (opflex_agent+0x3a3f0)
#5 AgentLauncher::run() cmd/opflex_agent.cpp:115 (opflex_agent+0x3a3f0)
#6 main cmd/opflex_agent.cpp:322 (opflex_agent+0x16dc2)
Thread T11 (tid=18066, running) created by main thread at:
#0 pthread_create <null> (libtsan.so.0+0x2d3be)
#1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xd0b04)
#2 AgentLauncher::run() cmd/opflex_agent.cpp:120 (opflex_agent+0x3a438)
#3 main cmd/opflex_agent.cpp:322 (opflex_agent+0x16dc2)
Signed-off-by: Gautam Venkataramanan <gautam.chennai@gmail.com>
tomflynn
pushed a commit
that referenced
this pull request
Sep 11, 2020
WARNING: ThreadSanitizer: data race (pid=18054)
Read of size 1 at 0x7b1000014d90 by thread T11:
#8 opflexagent::InterfaceStatsManager::on_timer(boost::system::error_code const&) ovs/InterfaceStatsManager.cpp:105 (libopflex_agent_renderer_openvswitch.so+0x1e74e1)
#14 operator() lib/Agent.cpp:605 (libopflex_agent.so.0+0x275439)
#15 __invoke_impl<void, opflexagent::Agent::start()::<lambda()> > /usr/include/c++/9/bits/invoke.h:60 (libopflex_agent.so.0+0x275439)
#16 __invoke<opflexagent::Agent::start()::<lambda()> > /usr/include/c++/9/bits/invoke.h:95 (libopflex_agent.so.0+0x275439)
#17 _M_invoke<0> /usr/include/c++/9/thread:244 (libopflex_agent.so.0+0x275439)
#18 operator() /usr/include/c++/9/thread:251 (libopflex_agent.so.0+0x275439)
#19 _M_run /usr/include/c++/9/thread:195 (libopflex_agent.so.0+0x275439)
#20 <null> <null> (libstdc++.so.6+0xd086f)
Previous write of size 1 at 0x7b1000014d90 by main thread (mutexes: write M180):
#4 opflexagent::InterfaceStatsManager::stop() ovs/InterfaceStatsManager.cpp:91 (libopflex_agent_renderer_openvswitch.so+0x1e81af)
#5 opflexagent::OVSRenderer::stop() ovs/OVSRenderer.cpp:259 (libopflex_agent_renderer_openvswitch.so+0xcb92b)
#6 opflexagent::Agent::stop() lib/Agent.cpp:718 (libopflex_agent.so.0+0x273368)
#7 AgentLauncher::run() cmd/opflex_agent.cpp:128 (opflex_agent+0x3a4a7)
#8 main cmd/opflex_agent.cpp:322 (opflex_agent+0x16dc2)
Location is heap block of size 64 at 0x7b1000014d80 allocated by main thread:
#0 operator new(unsigned long) <null> (libtsan.so.0+0x77cf2)
#1 opflexagent::InterfaceStatsManager::start() ovs/InterfaceStatsManager.cpp:73 (libopflex_agent_renderer_openvswitch.so+0x1e9646)
#2 opflexagent::OVSRenderer::start() ovs/OVSRenderer.cpp:187 (libopflex_agent_renderer_openvswitch.so+0xd5508)
#3 opflexagent::Agent::start() lib/Agent.cpp:601 (libopflex_agent.so.0+0x27937a)
#4 AgentLauncher::run() cmd/opflex_agent.cpp:120 (opflex_agent+0x3a438)
#5 main cmd/opflex_agent.cpp:322 (opflex_agent+0x16dc2)
Mutex M180 (0x7ffc4bcb7548) created at:
#0 pthread_mutex_lock <null> (libtsan.so.0+0x41d5b)
#1 __gthread_mutex_lock /usr/include/x86_64-linux-gnu/c++/9/bits/gthr-default.h:749 (opflex_agent+0x3a3f0)
#2 std::mutex::lock() /usr/include/c++/9/bits/std_mutex.h:100 (opflex_agent+0x3a3f0)
#3 std::unique_lock<std::mutex>::lock() /usr/include/c++/9/bits/unique_lock.h:141 (opflex_agent+0x3a3f0)
#4 std::unique_lock<std::mutex>::unique_lock(std::mutex&) /usr/include/c++/9/bits/unique_lock.h:71 (opflex_agent+0x3a3f0)
#5 AgentLauncher::run() cmd/opflex_agent.cpp:115 (opflex_agent+0x3a3f0)
#6 main cmd/opflex_agent.cpp:322 (opflex_agent+0x16dc2)
Thread T11 (tid=18066, running) created by main thread at:
#0 pthread_create <null> (libtsan.so.0+0x2d3be)
#1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xd0b04)
#2 AgentLauncher::run() cmd/opflex_agent.cpp:120 (opflex_agent+0x3a438)
#3 main cmd/opflex_agent.cpp:322 (opflex_agent+0x16dc2)
Signed-off-by: Gautam Venkataramanan <gautam.chennai@gmail.com>
gautvenk
added a commit
that referenced
this pull request
Sep 15, 2020
WARNING: ThreadSanitizer: data race (pid=4830)
Read of size 1 at 0x7b100000e050 by thread T11:
#7 opflexagent::SwitchManager::onConnectTimer(boost::system::error_code const&) ovs/SwitchManager.cpp:162 (libopflex_agent_renderer_openvswitch.so+0x1c4aa4)
#13 operator() lib/Agent.cpp:605 (libopflex_agent.so.0+0x2755d9)
Previous write of size 1 at 0x7b100000e050 by main thread (mutexes: write M180):
#4 opflexagent::SwitchManager::stop() ovs/SwitchManager.cpp:72 (libopflex_agent_renderer_openvswitch.so+0x1c4a06)
#5 opflexagent::OVSRenderer::stop() ovs/OVSRenderer.cpp:274 (libopflex_agent_renderer_openvswitch.so+0xcbdda)
#6 opflexagent::Agent::stop() lib/Agent.cpp:718 (libopflex_agent.so.0+0x273508)
#7 AgentLauncher::run() cmd/opflex_agent.cpp:128 (opflex_agent+0x3a4a7)
#8 main cmd/opflex_agent.cpp:322 (opflex_agent+0x16dc2)
Location is heap block of size 64 at 0x7b100000e040 allocated by thread T2:
#0 operator new(unsigned long) <null> (libtsan.so.0+0x77cf2)
#1 opflexagent::SwitchManager::enableSync() ovs/SwitchManager.cpp:110 (libopflex_agent_renderer_openvswitch.so+0x1c5212)
#2 opflexagent::IntFlowManager::configUpdated(opflex::modb::URI const&) ovs/IntFlowManager.cpp:523 (libopflex_agent_renderer_openvswitch.so+0x14ee12)
#3 opflexagent::PolicyManager::notifyConfig(opflex::modb::URI const&) lib/PolicyManager.cpp:234 (libopflex_agent.so.0+0x11ee01)
#4 opflexagent::PolicyManager::ConfigListener::objectUpdated(unsigned long, opflex::modb::URI const&) lib/PolicyManager.cpp:2862 (libopflex_agent.so.0+0x11ee8a)
#5 opflex::modb::ObjectStore::NotifQueueProc::processItem(opflex::modb::URI const&, boost::any const&) /home/noiro/work/opflex/libopflex/modb/ObjectStore.cpp:77 (libopflex.so.0+0xaf77a)
#6 opflex::modb::URIQueue::proc_async_func(uv_async_s*) /home/noiro/work/opflex/libopflex/modb/URIQueue.cpp:48 (libopflex.so.0+0x8df71)
Mutex M180 (0x7fff97118cf8) created at:
#0 pthread_mutex_lock <null> (libtsan.so.0+0x41d5b)
#1 __gthread_mutex_lock /usr/include/x86_64-linux-gnu/c++/9/bits/gthr-default.h:749 (opflex_agent+0x3a3f0)
#2 std::mutex::lock() /usr/include/c++/9/bits/std_mutex.h:100 (opflex_agent+0x3a3f0)
#3 std::unique_lock<std::mutex>::lock() /usr/include/c++/9/bits/unique_lock.h:141 (opflex_agent+0x3a3f0)
#4 std::unique_lock<std::mutex>::unique_lock(std::mutex&) /usr/include/c++/9/bits/unique_lock.h:71 (opflex_agent+0x3a3f0)
#5 AgentLauncher::run() cmd/opflex_agent.cpp:115 (opflex_agent+0x3a3f0)
#6 main cmd/opflex_agent.cpp:322 (opflex_agent+0x16dc2)
Thread T11 (tid=4849, running) created by main thread at:
#0 pthread_create <null> (libtsan.so.0+0x2d3be)
#1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xd0b04)
#2 AgentLauncher::run() cmd/opflex_agent.cpp:120 (opflex_agent+0x3a438)
#3 main cmd/opflex_agent.cpp:322 (opflex_agent+0x16dc2)
Thread T2 (tid=4840, running) created by main thread at:
#0 pthread_create <null> (libtsan.so.0+0x2d3be)
#1 uv_thread_create <null> (libuv.so.1+0x17cc0)
#2 opflex::modb::URIQueue::start() /home/noiro/work/opflex/libopflex/modb/URIQueue.cpp:73 (libopflex.so.0+0x8e4f1)
#3 opflex::modb::ObjectStore::start() /home/noiro/work/opflex/libopflex/modb/ObjectStore.cpp:87 (libopflex.so.0+0xaecbe)
#4 opflex::ofcore::OFFramework::start() /home/noiro/work/opflex/libopflex/ofcore/OFFramework.cpp:121 (libopflex.so.0+0x190a4f)
#5 opflexagent::Agent::start() lib/Agent.cpp:576 (libopflex_agent.so.0+0x27918c)
#6 AgentLauncher::run() cmd/opflex_agent.cpp:120 (opflex_agent+0x3a438)
#7 main cmd/opflex_agent.cpp:322 (opflex_agent+0x16dc2)
Signed-off-by: Gautam Venkataramanan <gautam.chennai@gmail.com>
tomflynn
pushed a commit
that referenced
this pull request
Sep 16, 2020
WARNING: ThreadSanitizer: data race (pid=4830)
Read of size 1 at 0x7b100000e050 by thread T11:
#7 opflexagent::SwitchManager::onConnectTimer(boost::system::error_code const&) ovs/SwitchManager.cpp:162 (libopflex_agent_renderer_openvswitch.so+0x1c4aa4)
#13 operator() lib/Agent.cpp:605 (libopflex_agent.so.0+0x2755d9)
Previous write of size 1 at 0x7b100000e050 by main thread (mutexes: write M180):
#4 opflexagent::SwitchManager::stop() ovs/SwitchManager.cpp:72 (libopflex_agent_renderer_openvswitch.so+0x1c4a06)
#5 opflexagent::OVSRenderer::stop() ovs/OVSRenderer.cpp:274 (libopflex_agent_renderer_openvswitch.so+0xcbdda)
#6 opflexagent::Agent::stop() lib/Agent.cpp:718 (libopflex_agent.so.0+0x273508)
#7 AgentLauncher::run() cmd/opflex_agent.cpp:128 (opflex_agent+0x3a4a7)
#8 main cmd/opflex_agent.cpp:322 (opflex_agent+0x16dc2)
Location is heap block of size 64 at 0x7b100000e040 allocated by thread T2:
#0 operator new(unsigned long) <null> (libtsan.so.0+0x77cf2)
#1 opflexagent::SwitchManager::enableSync() ovs/SwitchManager.cpp:110 (libopflex_agent_renderer_openvswitch.so+0x1c5212)
#2 opflexagent::IntFlowManager::configUpdated(opflex::modb::URI const&) ovs/IntFlowManager.cpp:523 (libopflex_agent_renderer_openvswitch.so+0x14ee12)
#3 opflexagent::PolicyManager::notifyConfig(opflex::modb::URI const&) lib/PolicyManager.cpp:234 (libopflex_agent.so.0+0x11ee01)
#4 opflexagent::PolicyManager::ConfigListener::objectUpdated(unsigned long, opflex::modb::URI const&) lib/PolicyManager.cpp:2862 (libopflex_agent.so.0+0x11ee8a)
#5 opflex::modb::ObjectStore::NotifQueueProc::processItem(opflex::modb::URI const&, boost::any const&) /home/noiro/work/opflex/libopflex/modb/ObjectStore.cpp:77 (libopflex.so.0+0xaf77a)
#6 opflex::modb::URIQueue::proc_async_func(uv_async_s*) /home/noiro/work/opflex/libopflex/modb/URIQueue.cpp:48 (libopflex.so.0+0x8df71)
Mutex M180 (0x7fff97118cf8) created at:
#0 pthread_mutex_lock <null> (libtsan.so.0+0x41d5b)
#1 __gthread_mutex_lock /usr/include/x86_64-linux-gnu/c++/9/bits/gthr-default.h:749 (opflex_agent+0x3a3f0)
#2 std::mutex::lock() /usr/include/c++/9/bits/std_mutex.h:100 (opflex_agent+0x3a3f0)
#3 std::unique_lock<std::mutex>::lock() /usr/include/c++/9/bits/unique_lock.h:141 (opflex_agent+0x3a3f0)
#4 std::unique_lock<std::mutex>::unique_lock(std::mutex&) /usr/include/c++/9/bits/unique_lock.h:71 (opflex_agent+0x3a3f0)
#5 AgentLauncher::run() cmd/opflex_agent.cpp:115 (opflex_agent+0x3a3f0)
#6 main cmd/opflex_agent.cpp:322 (opflex_agent+0x16dc2)
Thread T11 (tid=4849, running) created by main thread at:
#0 pthread_create <null> (libtsan.so.0+0x2d3be)
#1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xd0b04)
#2 AgentLauncher::run() cmd/opflex_agent.cpp:120 (opflex_agent+0x3a438)
#3 main cmd/opflex_agent.cpp:322 (opflex_agent+0x16dc2)
Thread T2 (tid=4840, running) created by main thread at:
#0 pthread_create <null> (libtsan.so.0+0x2d3be)
#1 uv_thread_create <null> (libuv.so.1+0x17cc0)
#2 opflex::modb::URIQueue::start() /home/noiro/work/opflex/libopflex/modb/URIQueue.cpp:73 (libopflex.so.0+0x8e4f1)
#3 opflex::modb::ObjectStore::start() /home/noiro/work/opflex/libopflex/modb/ObjectStore.cpp:87 (libopflex.so.0+0xaecbe)
#4 opflex::ofcore::OFFramework::start() /home/noiro/work/opflex/libopflex/ofcore/OFFramework.cpp:121 (libopflex.so.0+0x190a4f)
#5 opflexagent::Agent::start() lib/Agent.cpp:576 (libopflex_agent.so.0+0x27918c)
#6 AgentLauncher::run() cmd/opflex_agent.cpp:120 (opflex_agent+0x3a438)
#7 main cmd/opflex_agent.cpp:322 (opflex_agent+0x16dc2)
Signed-off-by: Gautam Venkataramanan <gautam.chennai@gmail.com>
gautvenk
added a commit
that referenced
this pull request
Dec 9, 2020
Pattern1:
WARNING: ThreadSanitizer: data race (pid=23522)
Write of size 8 at 0x7fff79e44c50 by thread T13 (mutexes: write M1028):
#0 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint
#1 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint
#2 std::__detail::_Map_base<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent:
#3 std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<opflexagent::Endpoint const>, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<cha
#4 opflexagent::EndpointManager::updateEndpointLocal(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, boost::optional<std::set<opflex::modb::URI, std::less<opflex::modb::URI>, std::allocator<opfl
#5 opflexagent::EndpointManager::updateEndpoint(opflexagent::Endpoint const&) lib/EndpointManager.cpp:311 (libopflex_agent.so.0+0x1bf259)
#6 opflexagent::EndpointSource::updateEndpoint(opflexagent::Endpoint const&) lib/EndpointSource.cpp:23 (libopflex_agent.so.0+0x1fae1a)
#7 opflexagent::FSEndpointSource::updated(boost::filesystem::path const&) lib/FSEndpointSource.cpp:521 (libopflex_agent.so.0+0x204af0)
#8 opflexagent::FSWatcher::operator()() lib/FSWatcher.cpp:215 (libopflex_agent.so.0+0x19e743)
Previous read of size 8 at 0x7fff79e44c50 by thread T8 (mutexes: write M1365):
#0 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint
#1 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint
#2 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint
#3 std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<opflexagent::Endpoint const>, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<cha
#4 opflexagent::IntFlowManager::updateSvcTgtStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool const&, bool const&) ovs/IntFlowManager.cpp:3774 (libopflex_agent_renderer_openvswitch
#5 opflexagent::IntFlowManager::updateSvcTgtStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool const&, bool const&) ovs/IntFlowManager.cpp:3821 (libopflex_agent_renderer_openvswitch
#6 opflexagent::IntFlowManager::handleUpdateSvcStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ovs/IntFlowManager.cpp:3027 (libopflex_agent_renderer_openvswitch.so+0x142442)
#7 operator() ovs/IntFlowManager.cpp:3080 (libopflex_agent_renderer_openvswitch.so+0x142852)
Pattern2:
WARNING: ThreadSanitizer: data race (pid=23522)
Write of size 8 at 0x7b1000044688 by thread T13 (mutexes: write M1028):
#0 operator delete(void*) <null> (libtsan.so.0+0x7747e)
#1 __gnu_cxx::new_allocator<std::__detail::_Hash_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint const> >, true> >::deallocate(std::__detail::
#2 std::allocator_traits<std::allocator<std::__detail::_Hash_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint const> >, true> > >::deallocate(s
#3 std::__detail::_Hashtable_alloc<std::allocator<std::__detail::_Hash_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint const> >, true> > >::_M
#4 std::__detail::_Hashtable_alloc<std::allocator<std::__detail::_Hash_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint const> >, true> > >::_M
#5 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint
#6 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint
#7 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint
#8 std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<opflexagent::Endpoint const>, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<cha
#9 opflexagent::EndpointManager::removeEndpoint(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) lib/EndpointManager.cpp:353 (libopflex_agent.so.0+0x1c3972)
#10 opflexagent::EndpointSource::removeEndpoint(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) lib/EndpointSource.cpp:27 (libopflex_agent.so.0+0x1fae5a)
#11 opflexagent::FSEndpointSource::deleted(boost::filesystem::path const&) lib/FSEndpointSource.cpp:544 (libopflex_agent.so.0+0x1fb49b)
Previous read of size 8 at 0x7b1000044688 by thread T8 (mutexes: write M1365):
#0 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::data() const /usr/include/c++/9/bits/basic_string.h:2313 (libopflex_agent_renderer_openvswitch.so+0x169c83)
#1 __gnu_cxx::__enable_if<std::__is_char<char>::__value, bool>::__type std::operator==<char>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<cha
#2 std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::operator()(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, st
#3 std::__detail::_Equal_helper<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexag
#4 std::__detail::_Hashtable_base<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflex
#5 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint
#6 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint
#7 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint
#8 std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<opflexagent::Endpoint const>, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<cha
#9 opflexagent::IntFlowManager::updateSvcNodeStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool const&, bool const&) ovs/IntFlowManager.cpp:3520 (libopflex_agent_renderer_openvswitc
#10 opflexagent::IntFlowManager::updateSvcNodeStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool const&, bool const&) ovs/IntFlowManager.cpp:3567 (libopflex_agent_renderer_openvswit
#11 opflexagent::IntFlowManager::handleUpdateSvcStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ovs/IntFlowManager.cpp:3028 (libopflex_agent_renderer_openvswitch.so+0x142453)
#12 operator() ovs/IntFlowManager.cpp:3080 (libopflex_agent_renderer_openvswitch.so+0x142852)
Pattern3:
#0 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint
#1 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint
[2020-Dec-08 19:51:10.927044] [error] [lib/FSEndpointSource.cpp:527:updated] Could not load endpoint from: "/usr/local/var/lib/opflex-agent-ovs/endpoints/h1.ep": /usr/local/var/lib/opflex-agent-ovs/endpoints/h1.ep: cannot open file
#2 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint
#3 std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<opflexagent::Endpoint const>, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<cha
#4 opflexagent::IntFlowManager::updateSvcTgtStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool const&, bool const&) ovs/IntFlowManager.cpp:3774 (libopflex_agent_renderer_openvswitch
#5 opflexagent::IntFlowManager::updateSvcTgtStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool const&, bool const&) ovs/IntFlowManager.cpp:3821 (libopflex_agent_renderer_openvswitch
#6 opflexagent::IntFlowManager::handleUpdateSvcStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ovs/IntFlowManager.cpp:3027 (libopflex_agent_renderer_openvswitch.so+0x142442)
#7 operator() ovs/IntFlowManager.cpp:3080 (libopflex_agent_renderer_openvswitch.so+0x142852)
Previous write of size 8 at 0x7b0c0000fdf0 by thread T12:
[failed to restore the stack]
Mutex M1365 (0x7ba400000920) created at:
[2020-Dec-08 19:51:10.927490] [debug] [ovs/TableState.cpp:489:apply] DEL|cookie=0x0, duration=0s, table=5, n_packets=0, n_bytes=0, idle_age=0, priority=20,icmp6,reg4=0x1,reg6=0x1,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00,icmp_type=135,ic
#0 pthread_mutex_lock <null> (libtsan.so.0+0x41d5b)
#1 __gthread_mutex_lock /usr/include/x86_64-linux-gnu/c++/9/bits/gthr-default.h:749 (libopflex_agent_renderer_openvswitch.so+0x141ae4)
#2 std::mutex::lock() /usr/include/c++/9/bits/std_mutex.h:100 (libopflex_agent_renderer_openvswitch.so+0x141ae4)
#3 std::lock_guard<std::mutex>::lock_guard(std::mutex&) /usr/include/c++/9/bits/std_mutex.h:159 (libopflex_agent_renderer_openvswitch.so+0x141ae4)
#4 opflexagent::IntFlowManager::handleUpdateSvcStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ovs/IntFlowManager.cpp:3015 (libopflex_agent_renderer_openvswitch.so+0x141ae4)
#5 operator() ovs/IntFlowManager.cpp:3080 (libopflex_agent_renderer_openvswitch.so+0x142852)
Pattern4:
WARNING: ThreadSanitizer: heap-use-after-free (pid=10699)
Read of size 8 at 0x7b6400021e88 by thread T8 (mutexes: write M1379, write M182):
#0 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std:
#1 std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::hash<std::__cxx11::basic_string<char, std::ch
#2 opflexagent::AgentPrometheusManager::createLabelMapFromSvcTargetAttr(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<ch
#3 opflexagent::AgentPrometheusManager::createDynamicGaugeSvcTarget(opflexagent::AgentPrometheusManager::SVC_TARGET_METRICS, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_s
#4 opflexagent::AgentPrometheusManager::addNUpdateSvcTargetCounter(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >
#5 opflexagent::IntFlowManager::updateSvcTgtStatsCounters(unsigned long const&, bool const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned long const&, unsigned long const&, std::unor
#6 operator() ovs/IntFlowManager.cpp:3703 (libopflex_agent_renderer_openvswitch.so+0x14127f)
#7 opflexagent::IntFlowManager::updateSvcTgtStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool const&, bool const&) ovs/IntFlowManager.cpp:3776 (libopflex_agent_renderer_openvswitch
#8 opflexagent::IntFlowManager::updateSvcTgtStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool const&, bool const&) ovs/IntFlowManager.cpp:3821 (libopflex_agent_renderer_openvswitch
#9 opflexagent::IntFlowManager::handleUpdateSvcStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ovs/IntFlowManager.cpp:3027 (libopflex_agent_renderer_openvswitch.so+0x142a72)
#10 operator() ovs/IntFlowManager.cpp:3080 (libopflex_agent_renderer_openvswitch.so+0x142e82)
Previous write of size 8 at 0x7b6400021e88 by thread T12:
#0 operator delete(void*) <null> (libtsan.so.0+0x7747e)
#1 std::_Sp_counted_ptr_inplace<opflexagent::Endpoint const, std::allocator<opflexagent::Endpoint>, (__gnu_cxx::_Lock_policy)2>::_M_destroy() /usr/include/c++/9/ext/new_allocator.h:128 (libopflex_agent.so.0+0x1c2f75)
#2 std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() /usr/include/c++/9/bits/shared_ptr_base.h:171 (libopflex_agent_renderer_openvswitch.so+0x1904e8)
#3 std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() /usr/include/c++/9/bits/shared_ptr_base.h:148 (libopflex_agent_renderer_openvswitch.so+0x1904e8)
#4 std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() /usr/include/c++/9/bits/shared_ptr_base.h:730 (libopflex_agent_renderer_openvswitch.so+0x1904e8)
#5 std::__shared_ptr<opflexagent::Endpoint const, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() /usr/include/c++/9/bits/shared_ptr_base.h:1169 (libopflex_agent_renderer_openvswitch.so+0x1904e8)
#6 std::shared_ptr<opflexagent::Endpoint const>::~shared_ptr() /usr/include/c++/9/bits/shared_ptr.h:103 (libopflex_agent_renderer_openvswitch.so+0x1904e8)
#7 opflexagent::AccessFlowManager::handleEndpointUpdate(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ovs/AccessFlowManager.cpp:411 (libopflex_agent_renderer_openvswitch.so+0x1904e8)
Signed-off-by: Gautam Venkataramanan <gautam.chennai@gmail.com>
tomflynn
pushed a commit
that referenced
this pull request
Dec 10, 2020
Pattern1:
WARNING: ThreadSanitizer: data race (pid=23522)
Write of size 8 at 0x7fff79e44c50 by thread T13 (mutexes: write M1028):
#0 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint
#1 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint
#2 std::__detail::_Map_base<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent:
#3 std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<opflexagent::Endpoint const>, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<cha
#4 opflexagent::EndpointManager::updateEndpointLocal(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, boost::optional<std::set<opflex::modb::URI, std::less<opflex::modb::URI>, std::allocator<opfl
#5 opflexagent::EndpointManager::updateEndpoint(opflexagent::Endpoint const&) lib/EndpointManager.cpp:311 (libopflex_agent.so.0+0x1bf259)
#6 opflexagent::EndpointSource::updateEndpoint(opflexagent::Endpoint const&) lib/EndpointSource.cpp:23 (libopflex_agent.so.0+0x1fae1a)
#7 opflexagent::FSEndpointSource::updated(boost::filesystem::path const&) lib/FSEndpointSource.cpp:521 (libopflex_agent.so.0+0x204af0)
#8 opflexagent::FSWatcher::operator()() lib/FSWatcher.cpp:215 (libopflex_agent.so.0+0x19e743)
Previous read of size 8 at 0x7fff79e44c50 by thread T8 (mutexes: write M1365):
#0 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint
#1 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint
#2 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint
#3 std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<opflexagent::Endpoint const>, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<cha
#4 opflexagent::IntFlowManager::updateSvcTgtStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool const&, bool const&) ovs/IntFlowManager.cpp:3774 (libopflex_agent_renderer_openvswitch
#5 opflexagent::IntFlowManager::updateSvcTgtStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool const&, bool const&) ovs/IntFlowManager.cpp:3821 (libopflex_agent_renderer_openvswitch
#6 opflexagent::IntFlowManager::handleUpdateSvcStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ovs/IntFlowManager.cpp:3027 (libopflex_agent_renderer_openvswitch.so+0x142442)
#7 operator() ovs/IntFlowManager.cpp:3080 (libopflex_agent_renderer_openvswitch.so+0x142852)
Pattern2:
WARNING: ThreadSanitizer: data race (pid=23522)
Write of size 8 at 0x7b1000044688 by thread T13 (mutexes: write M1028):
#0 operator delete(void*) <null> (libtsan.so.0+0x7747e)
#1 __gnu_cxx::new_allocator<std::__detail::_Hash_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint const> >, true> >::deallocate(std::__detail::
#2 std::allocator_traits<std::allocator<std::__detail::_Hash_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint const> >, true> > >::deallocate(s
#3 std::__detail::_Hashtable_alloc<std::allocator<std::__detail::_Hash_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint const> >, true> > >::_M
#4 std::__detail::_Hashtable_alloc<std::allocator<std::__detail::_Hash_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint const> >, true> > >::_M
#5 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint
#6 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint
#7 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint
#8 std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<opflexagent::Endpoint const>, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<cha
#9 opflexagent::EndpointManager::removeEndpoint(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) lib/EndpointManager.cpp:353 (libopflex_agent.so.0+0x1c3972)
#10 opflexagent::EndpointSource::removeEndpoint(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) lib/EndpointSource.cpp:27 (libopflex_agent.so.0+0x1fae5a)
#11 opflexagent::FSEndpointSource::deleted(boost::filesystem::path const&) lib/FSEndpointSource.cpp:544 (libopflex_agent.so.0+0x1fb49b)
Previous read of size 8 at 0x7b1000044688 by thread T8 (mutexes: write M1365):
#0 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::data() const /usr/include/c++/9/bits/basic_string.h:2313 (libopflex_agent_renderer_openvswitch.so+0x169c83)
#1 __gnu_cxx::__enable_if<std::__is_char<char>::__value, bool>::__type std::operator==<char>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<cha
#2 std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::operator()(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, st
#3 std::__detail::_Equal_helper<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexag
#4 std::__detail::_Hashtable_base<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflex
#5 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint
#6 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint
#7 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint
#8 std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<opflexagent::Endpoint const>, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<cha
#9 opflexagent::IntFlowManager::updateSvcNodeStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool const&, bool const&) ovs/IntFlowManager.cpp:3520 (libopflex_agent_renderer_openvswitc
#10 opflexagent::IntFlowManager::updateSvcNodeStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool const&, bool const&) ovs/IntFlowManager.cpp:3567 (libopflex_agent_renderer_openvswit
#11 opflexagent::IntFlowManager::handleUpdateSvcStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ovs/IntFlowManager.cpp:3028 (libopflex_agent_renderer_openvswitch.so+0x142453)
#12 operator() ovs/IntFlowManager.cpp:3080 (libopflex_agent_renderer_openvswitch.so+0x142852)
Pattern3:
#0 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint
#1 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint
[2020-Dec-08 19:51:10.927044] [error] [lib/FSEndpointSource.cpp:527:updated] Could not load endpoint from: "/usr/local/var/lib/opflex-agent-ovs/endpoints/h1.ep": /usr/local/var/lib/opflex-agent-ovs/endpoints/h1.ep: cannot open file
#2 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint
#3 std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<opflexagent::Endpoint const>, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<cha
#4 opflexagent::IntFlowManager::updateSvcTgtStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool const&, bool const&) ovs/IntFlowManager.cpp:3774 (libopflex_agent_renderer_openvswitch
#5 opflexagent::IntFlowManager::updateSvcTgtStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool const&, bool const&) ovs/IntFlowManager.cpp:3821 (libopflex_agent_renderer_openvswitch
#6 opflexagent::IntFlowManager::handleUpdateSvcStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ovs/IntFlowManager.cpp:3027 (libopflex_agent_renderer_openvswitch.so+0x142442)
#7 operator() ovs/IntFlowManager.cpp:3080 (libopflex_agent_renderer_openvswitch.so+0x142852)
Previous write of size 8 at 0x7b0c0000fdf0 by thread T12:
[failed to restore the stack]
Mutex M1365 (0x7ba400000920) created at:
[2020-Dec-08 19:51:10.927490] [debug] [ovs/TableState.cpp:489:apply] DEL|cookie=0x0, duration=0s, table=5, n_packets=0, n_bytes=0, idle_age=0, priority=20,icmp6,reg4=0x1,reg6=0x1,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00,icmp_type=135,ic
#0 pthread_mutex_lock <null> (libtsan.so.0+0x41d5b)
#1 __gthread_mutex_lock /usr/include/x86_64-linux-gnu/c++/9/bits/gthr-default.h:749 (libopflex_agent_renderer_openvswitch.so+0x141ae4)
#2 std::mutex::lock() /usr/include/c++/9/bits/std_mutex.h:100 (libopflex_agent_renderer_openvswitch.so+0x141ae4)
#3 std::lock_guard<std::mutex>::lock_guard(std::mutex&) /usr/include/c++/9/bits/std_mutex.h:159 (libopflex_agent_renderer_openvswitch.so+0x141ae4)
#4 opflexagent::IntFlowManager::handleUpdateSvcStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ovs/IntFlowManager.cpp:3015 (libopflex_agent_renderer_openvswitch.so+0x141ae4)
#5 operator() ovs/IntFlowManager.cpp:3080 (libopflex_agent_renderer_openvswitch.so+0x142852)
Pattern4:
WARNING: ThreadSanitizer: heap-use-after-free (pid=10699)
Read of size 8 at 0x7b6400021e88 by thread T8 (mutexes: write M1379, write M182):
#0 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std:
#1 std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::hash<std::__cxx11::basic_string<char, std::ch
#2 opflexagent::AgentPrometheusManager::createLabelMapFromSvcTargetAttr(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<ch
#3 opflexagent::AgentPrometheusManager::createDynamicGaugeSvcTarget(opflexagent::AgentPrometheusManager::SVC_TARGET_METRICS, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_s
#4 opflexagent::AgentPrometheusManager::addNUpdateSvcTargetCounter(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >
#5 opflexagent::IntFlowManager::updateSvcTgtStatsCounters(unsigned long const&, bool const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned long const&, unsigned long const&, std::unor
#6 operator() ovs/IntFlowManager.cpp:3703 (libopflex_agent_renderer_openvswitch.so+0x14127f)
#7 opflexagent::IntFlowManager::updateSvcTgtStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool const&, bool const&) ovs/IntFlowManager.cpp:3776 (libopflex_agent_renderer_openvswitch
#8 opflexagent::IntFlowManager::updateSvcTgtStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool const&, bool const&) ovs/IntFlowManager.cpp:3821 (libopflex_agent_renderer_openvswitch
#9 opflexagent::IntFlowManager::handleUpdateSvcStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ovs/IntFlowManager.cpp:3027 (libopflex_agent_renderer_openvswitch.so+0x142a72)
#10 operator() ovs/IntFlowManager.cpp:3080 (libopflex_agent_renderer_openvswitch.so+0x142e82)
Previous write of size 8 at 0x7b6400021e88 by thread T12:
#0 operator delete(void*) <null> (libtsan.so.0+0x7747e)
#1 std::_Sp_counted_ptr_inplace<opflexagent::Endpoint const, std::allocator<opflexagent::Endpoint>, (__gnu_cxx::_Lock_policy)2>::_M_destroy() /usr/include/c++/9/ext/new_allocator.h:128 (libopflex_agent.so.0+0x1c2f75)
#2 std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() /usr/include/c++/9/bits/shared_ptr_base.h:171 (libopflex_agent_renderer_openvswitch.so+0x1904e8)
#3 std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() /usr/include/c++/9/bits/shared_ptr_base.h:148 (libopflex_agent_renderer_openvswitch.so+0x1904e8)
#4 std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() /usr/include/c++/9/bits/shared_ptr_base.h:730 (libopflex_agent_renderer_openvswitch.so+0x1904e8)
#5 std::__shared_ptr<opflexagent::Endpoint const, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() /usr/include/c++/9/bits/shared_ptr_base.h:1169 (libopflex_agent_renderer_openvswitch.so+0x1904e8)
#6 std::shared_ptr<opflexagent::Endpoint const>::~shared_ptr() /usr/include/c++/9/bits/shared_ptr.h:103 (libopflex_agent_renderer_openvswitch.so+0x1904e8)
#7 opflexagent::AccessFlowManager::handleEndpointUpdate(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ovs/AccessFlowManager.cpp:411 (libopflex_agent_renderer_openvswitch.so+0x1904e8)
Signed-off-by: Gautam Venkataramanan <gautam.chennai@gmail.com>
gautvenk
added a commit
that referenced
this pull request
Dec 14, 2020
WARNING: ThreadSanitizer: data race (pid=29448)
Write of size 8 at 0x7b4400040100 by thread T16 (mutexes: write M1154):
#0 operator delete(void*) <null> (libtsan.so.0+0x75b31)
#1 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, opflexagent::EndpointManager::EndpointState>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, opflexagent::EndpointManager::EndpointState> >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_rehash(unsigned long, unsigned long const&) <null> (libopflex_agent.so.0+0x1b7d2f)
#2 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, opflexagent::EndpointManager::EndpointState>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, opflexagent::EndpointManager::EndpointState> >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_insert_unique_node(unsigned long, unsigned long, std::__detail::_Hash_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, opflexagent::EndpointManager::EndpointState>, true>*, unsigned long) <null> (libopflex_agent.so.0+0x1bd4d6)
#3 std::__detail::_Map_base<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, opflexagent::EndpointManager::EndpointState>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, opflexagent::EndpointManager::EndpointState> >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true>, true>::operator[](std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) <null> (libopflex_agent.so.0+0x1bd77f)
#4 opflexagent::EndpointManager::updateEndpoint(opflexagent::Endpoint const&) <null> (libopflex_agent.so.0+0x1a6a48)
#5 opflexagent::EndpointSource::updateEndpoint(opflexagent::Endpoint const&) <null> (libopflex_agent.so.0+0x1d9f2a)
#6 opflexagent::FSEndpointSource::updated(boost::filesystem::path const&) <null> (libopflex_agent.so.0+0x1e2467)
Previous read of size 8 at 0x7b4400040100 by thread T12 (mutexes: write M402926635977816456):
#2 opflexagent::EndpointManager::updateEndpointCounters(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, opflexagent::EpCounters&) <null> (libopflex_agent.so.0+0x19b594)
#3 opflexagent::InterfaceStatsManager::updateEndpointCounters(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, opflexagent::SwitchConnection*, opflexagent::EpCounters&) <null> (libopflex_agent_renderer_openvswitch.so+0x1c99eb)
#4 opflexagent::InterfaceStatsManager::Handle(opflexagent::SwitchConnection*, int, ofpbuf*, ofputil_flow_removed*) <null> (libopflex_agent_renderer_openvswitch.so+0x1c9fd5)
#5 opflexagent::SwitchConnection::receiveOFMessage() <null> (libopflex_agent_renderer_openvswitch.so+0x1a121b)
#6 opflexagent::SwitchConnection::Monitor() <null> (libopflex_agent_renderer_openvswitch.so+0x1a40d8)
#7 opflexagent::SwitchConnection::operator()() <null> (libopflex_agent_renderer_openvswitch.so+0x1a5bf5)
Signed-off-by: Gautam Venkataramanan <gautam.chennai@gmail.com>
tomflynn
pushed a commit
that referenced
this pull request
Dec 14, 2020
WARNING: ThreadSanitizer: data race (pid=29448)
Write of size 8 at 0x7b4400040100 by thread T16 (mutexes: write M1154):
#0 operator delete(void*) <null> (libtsan.so.0+0x75b31)
#1 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, opflexagent::EndpointManager::EndpointState>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, opflexagent::EndpointManager::EndpointState> >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_rehash(unsigned long, unsigned long const&) <null> (libopflex_agent.so.0+0x1b7d2f)
#2 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, opflexagent::EndpointManager::EndpointState>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, opflexagent::EndpointManager::EndpointState> >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_insert_unique_node(unsigned long, unsigned long, std::__detail::_Hash_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, opflexagent::EndpointManager::EndpointState>, true>*, unsigned long) <null> (libopflex_agent.so.0+0x1bd4d6)
#3 std::__detail::_Map_base<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, opflexagent::EndpointManager::EndpointState>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, opflexagent::EndpointManager::EndpointState> >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true>, true>::operator[](std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) <null> (libopflex_agent.so.0+0x1bd77f)
#4 opflexagent::EndpointManager::updateEndpoint(opflexagent::Endpoint const&) <null> (libopflex_agent.so.0+0x1a6a48)
#5 opflexagent::EndpointSource::updateEndpoint(opflexagent::Endpoint const&) <null> (libopflex_agent.so.0+0x1d9f2a)
#6 opflexagent::FSEndpointSource::updated(boost::filesystem::path const&) <null> (libopflex_agent.so.0+0x1e2467)
Previous read of size 8 at 0x7b4400040100 by thread T12 (mutexes: write M402926635977816456):
#2 opflexagent::EndpointManager::updateEndpointCounters(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, opflexagent::EpCounters&) <null> (libopflex_agent.so.0+0x19b594)
#3 opflexagent::InterfaceStatsManager::updateEndpointCounters(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, opflexagent::SwitchConnection*, opflexagent::EpCounters&) <null> (libopflex_agent_renderer_openvswitch.so+0x1c99eb)
#4 opflexagent::InterfaceStatsManager::Handle(opflexagent::SwitchConnection*, int, ofpbuf*, ofputil_flow_removed*) <null> (libopflex_agent_renderer_openvswitch.so+0x1c9fd5)
#5 opflexagent::SwitchConnection::receiveOFMessage() <null> (libopflex_agent_renderer_openvswitch.so+0x1a121b)
#6 opflexagent::SwitchConnection::Monitor() <null> (libopflex_agent_renderer_openvswitch.so+0x1a40d8)
#7 opflexagent::SwitchConnection::operator()() <null> (libopflex_agent_renderer_openvswitch.so+0x1a5bf5)
Signed-off-by: Gautam Venkataramanan <gautam.chennai@gmail.com>
ansao-aci
pushed a commit
to ansao-aci/opflex
that referenced
this pull request
Feb 10, 2021
…onetworks#305) Pattern1: WARNING: ThreadSanitizer: data race (pid=23522) Write of size 8 at 0x7fff79e44c50 by thread T13 (mutexes: write M1028): #0 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint noironetworks#1 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint noironetworks#2 std::__detail::_Map_base<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent: noironetworks#3 std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<opflexagent::Endpoint const>, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<cha noironetworks#4 opflexagent::EndpointManager::updateEndpointLocal(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, boost::optional<std::set<opflex::modb::URI, std::less<opflex::modb::URI>, std::allocator<opfl noironetworks#5 opflexagent::EndpointManager::updateEndpoint(opflexagent::Endpoint const&) lib/EndpointManager.cpp:311 (libopflex_agent.so.0+0x1bf259) noironetworks#6 opflexagent::EndpointSource::updateEndpoint(opflexagent::Endpoint const&) lib/EndpointSource.cpp:23 (libopflex_agent.so.0+0x1fae1a) noironetworks#7 opflexagent::FSEndpointSource::updated(boost::filesystem::path const&) lib/FSEndpointSource.cpp:521 (libopflex_agent.so.0+0x204af0) noironetworks#8 opflexagent::FSWatcher::operator()() lib/FSWatcher.cpp:215 (libopflex_agent.so.0+0x19e743) Previous read of size 8 at 0x7fff79e44c50 by thread T8 (mutexes: write M1365): #0 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint noironetworks#1 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint noironetworks#2 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint noironetworks#3 std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<opflexagent::Endpoint const>, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<cha noironetworks#4 opflexagent::IntFlowManager::updateSvcTgtStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool const&, bool const&) ovs/IntFlowManager.cpp:3774 (libopflex_agent_renderer_openvswitch noironetworks#5 opflexagent::IntFlowManager::updateSvcTgtStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool const&, bool const&) ovs/IntFlowManager.cpp:3821 (libopflex_agent_renderer_openvswitch noironetworks#6 opflexagent::IntFlowManager::handleUpdateSvcStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ovs/IntFlowManager.cpp:3027 (libopflex_agent_renderer_openvswitch.so+0x142442) noironetworks#7 operator() ovs/IntFlowManager.cpp:3080 (libopflex_agent_renderer_openvswitch.so+0x142852) Pattern2: WARNING: ThreadSanitizer: data race (pid=23522) Write of size 8 at 0x7b1000044688 by thread T13 (mutexes: write M1028): #0 operator delete(void*) <null> (libtsan.so.0+0x7747e) noironetworks#1 __gnu_cxx::new_allocator<std::__detail::_Hash_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint const> >, true> >::deallocate(std::__detail:: noironetworks#2 std::allocator_traits<std::allocator<std::__detail::_Hash_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint const> >, true> > >::deallocate(s noironetworks#3 std::__detail::_Hashtable_alloc<std::allocator<std::__detail::_Hash_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint const> >, true> > >::_M noironetworks#4 std::__detail::_Hashtable_alloc<std::allocator<std::__detail::_Hash_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint const> >, true> > >::_M noironetworks#5 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint noironetworks#6 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint noironetworks#7 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint noironetworks#8 std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<opflexagent::Endpoint const>, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<cha noironetworks#9 opflexagent::EndpointManager::removeEndpoint(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) lib/EndpointManager.cpp:353 (libopflex_agent.so.0+0x1c3972) noironetworks#10 opflexagent::EndpointSource::removeEndpoint(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) lib/EndpointSource.cpp:27 (libopflex_agent.so.0+0x1fae5a) noironetworks#11 opflexagent::FSEndpointSource::deleted(boost::filesystem::path const&) lib/FSEndpointSource.cpp:544 (libopflex_agent.so.0+0x1fb49b) Previous read of size 8 at 0x7b1000044688 by thread T8 (mutexes: write M1365): #0 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::data() const /usr/include/c++/9/bits/basic_string.h:2313 (libopflex_agent_renderer_openvswitch.so+0x169c83) noironetworks#1 __gnu_cxx::__enable_if<std::__is_char<char>::__value, bool>::__type std::operator==<char>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<cha noironetworks#2 std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::operator()(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, st noironetworks#3 std::__detail::_Equal_helper<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexag noironetworks#4 std::__detail::_Hashtable_base<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflex noironetworks#5 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint noironetworks#6 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint noironetworks#7 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint noironetworks#8 std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<opflexagent::Endpoint const>, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<cha noironetworks#9 opflexagent::IntFlowManager::updateSvcNodeStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool const&, bool const&) ovs/IntFlowManager.cpp:3520 (libopflex_agent_renderer_openvswitc noironetworks#10 opflexagent::IntFlowManager::updateSvcNodeStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool const&, bool const&) ovs/IntFlowManager.cpp:3567 (libopflex_agent_renderer_openvswit noironetworks#11 opflexagent::IntFlowManager::handleUpdateSvcStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ovs/IntFlowManager.cpp:3028 (libopflex_agent_renderer_openvswitch.so+0x142453) noironetworks#12 operator() ovs/IntFlowManager.cpp:3080 (libopflex_agent_renderer_openvswitch.so+0x142852) Pattern3: #0 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint noironetworks#1 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint [2020-Dec-08 19:51:10.927044] [error] [lib/FSEndpointSource.cpp:527:updated] Could not load endpoint from: "/usr/local/var/lib/opflex-agent-ovs/endpoints/h1.ep": /usr/local/var/lib/opflex-agent-ovs/endpoints/h1.ep: cannot open file noironetworks#2 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<opflexagent::Endpoint noironetworks#3 std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<opflexagent::Endpoint const>, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<cha noironetworks#4 opflexagent::IntFlowManager::updateSvcTgtStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool const&, bool const&) ovs/IntFlowManager.cpp:3774 (libopflex_agent_renderer_openvswitch noironetworks#5 opflexagent::IntFlowManager::updateSvcTgtStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool const&, bool const&) ovs/IntFlowManager.cpp:3821 (libopflex_agent_renderer_openvswitch noironetworks#6 opflexagent::IntFlowManager::handleUpdateSvcStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ovs/IntFlowManager.cpp:3027 (libopflex_agent_renderer_openvswitch.so+0x142442) noironetworks#7 operator() ovs/IntFlowManager.cpp:3080 (libopflex_agent_renderer_openvswitch.so+0x142852) Previous write of size 8 at 0x7b0c0000fdf0 by thread T12: [failed to restore the stack] Mutex M1365 (0x7ba400000920) created at: [2020-Dec-08 19:51:10.927490] [debug] [ovs/TableState.cpp:489:apply] DEL|cookie=0x0, duration=0s, table=5, n_packets=0, n_bytes=0, idle_age=0, priority=20,icmp6,reg4=0x1,reg6=0x1,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00,icmp_type=135,ic #0 pthread_mutex_lock <null> (libtsan.so.0+0x41d5b) noironetworks#1 __gthread_mutex_lock /usr/include/x86_64-linux-gnu/c++/9/bits/gthr-default.h:749 (libopflex_agent_renderer_openvswitch.so+0x141ae4) noironetworks#2 std::mutex::lock() /usr/include/c++/9/bits/std_mutex.h:100 (libopflex_agent_renderer_openvswitch.so+0x141ae4) noironetworks#3 std::lock_guard<std::mutex>::lock_guard(std::mutex&) /usr/include/c++/9/bits/std_mutex.h:159 (libopflex_agent_renderer_openvswitch.so+0x141ae4) noironetworks#4 opflexagent::IntFlowManager::handleUpdateSvcStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ovs/IntFlowManager.cpp:3015 (libopflex_agent_renderer_openvswitch.so+0x141ae4) noironetworks#5 operator() ovs/IntFlowManager.cpp:3080 (libopflex_agent_renderer_openvswitch.so+0x142852) Pattern4: WARNING: ThreadSanitizer: heap-use-after-free (pid=10699) Read of size 8 at 0x7b6400021e88 by thread T8 (mutexes: write M1379, write M182): #0 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std: noironetworks#1 std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::hash<std::__cxx11::basic_string<char, std::ch noironetworks#2 opflexagent::AgentPrometheusManager::createLabelMapFromSvcTargetAttr(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<ch noironetworks#3 opflexagent::AgentPrometheusManager::createDynamicGaugeSvcTarget(opflexagent::AgentPrometheusManager::SVC_TARGET_METRICS, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_s noironetworks#4 opflexagent::AgentPrometheusManager::addNUpdateSvcTargetCounter(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > noironetworks#5 opflexagent::IntFlowManager::updateSvcTgtStatsCounters(unsigned long const&, bool const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned long const&, unsigned long const&, std::unor noironetworks#6 operator() ovs/IntFlowManager.cpp:3703 (libopflex_agent_renderer_openvswitch.so+0x14127f) noironetworks#7 opflexagent::IntFlowManager::updateSvcTgtStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool const&, bool const&) ovs/IntFlowManager.cpp:3776 (libopflex_agent_renderer_openvswitch noironetworks#8 opflexagent::IntFlowManager::updateSvcTgtStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool const&, bool const&) ovs/IntFlowManager.cpp:3821 (libopflex_agent_renderer_openvswitch noironetworks#9 opflexagent::IntFlowManager::handleUpdateSvcStatsFlows(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ovs/IntFlowManager.cpp:3027 (libopflex_agent_renderer_openvswitch.so+0x142a72) noironetworks#10 operator() ovs/IntFlowManager.cpp:3080 (libopflex_agent_renderer_openvswitch.so+0x142e82) Previous write of size 8 at 0x7b6400021e88 by thread T12: #0 operator delete(void*) <null> (libtsan.so.0+0x7747e) noironetworks#1 std::_Sp_counted_ptr_inplace<opflexagent::Endpoint const, std::allocator<opflexagent::Endpoint>, (__gnu_cxx::_Lock_policy)2>::_M_destroy() /usr/include/c++/9/ext/new_allocator.h:128 (libopflex_agent.so.0+0x1c2f75) noironetworks#2 std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() /usr/include/c++/9/bits/shared_ptr_base.h:171 (libopflex_agent_renderer_openvswitch.so+0x1904e8) noironetworks#3 std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() /usr/include/c++/9/bits/shared_ptr_base.h:148 (libopflex_agent_renderer_openvswitch.so+0x1904e8) noironetworks#4 std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() /usr/include/c++/9/bits/shared_ptr_base.h:730 (libopflex_agent_renderer_openvswitch.so+0x1904e8) noironetworks#5 std::__shared_ptr<opflexagent::Endpoint const, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() /usr/include/c++/9/bits/shared_ptr_base.h:1169 (libopflex_agent_renderer_openvswitch.so+0x1904e8) noironetworks#6 std::shared_ptr<opflexagent::Endpoint const>::~shared_ptr() /usr/include/c++/9/bits/shared_ptr.h:103 (libopflex_agent_renderer_openvswitch.so+0x1904e8) noironetworks#7 opflexagent::AccessFlowManager::handleEndpointUpdate(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ovs/AccessFlowManager.cpp:411 (libopflex_agent_renderer_openvswitch.so+0x1904e8) Signed-off-by: Gautam Venkataramanan <gautam.chennai@gmail.com>
ansao-aci
pushed a commit
to ansao-aci/opflex
that referenced
this pull request
Feb 10, 2021
WARNING: ThreadSanitizer: data race (pid=29448)
Write of size 8 at 0x7b4400040100 by thread T16 (mutexes: write M1154):
#0 operator delete(void*) <null> (libtsan.so.0+0x75b31)
noironetworks#1 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, opflexagent::EndpointManager::EndpointState>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, opflexagent::EndpointManager::EndpointState> >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_rehash(unsigned long, unsigned long const&) <null> (libopflex_agent.so.0+0x1b7d2f)
noironetworks#2 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, opflexagent::EndpointManager::EndpointState>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, opflexagent::EndpointManager::EndpointState> >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_insert_unique_node(unsigned long, unsigned long, std::__detail::_Hash_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, opflexagent::EndpointManager::EndpointState>, true>*, unsigned long) <null> (libopflex_agent.so.0+0x1bd4d6)
noironetworks#3 std::__detail::_Map_base<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, opflexagent::EndpointManager::EndpointState>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, opflexagent::EndpointManager::EndpointState> >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true>, true>::operator[](std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) <null> (libopflex_agent.so.0+0x1bd77f)
noironetworks#4 opflexagent::EndpointManager::updateEndpoint(opflexagent::Endpoint const&) <null> (libopflex_agent.so.0+0x1a6a48)
noironetworks#5 opflexagent::EndpointSource::updateEndpoint(opflexagent::Endpoint const&) <null> (libopflex_agent.so.0+0x1d9f2a)
noironetworks#6 opflexagent::FSEndpointSource::updated(boost::filesystem::path const&) <null> (libopflex_agent.so.0+0x1e2467)
Previous read of size 8 at 0x7b4400040100 by thread T12 (mutexes: write M402926635977816456):
noironetworks#2 opflexagent::EndpointManager::updateEndpointCounters(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, opflexagent::EpCounters&) <null> (libopflex_agent.so.0+0x19b594)
noironetworks#3 opflexagent::InterfaceStatsManager::updateEndpointCounters(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, opflexagent::SwitchConnection*, opflexagent::EpCounters&) <null> (libopflex_agent_renderer_openvswitch.so+0x1c99eb)
noironetworks#4 opflexagent::InterfaceStatsManager::Handle(opflexagent::SwitchConnection*, int, ofpbuf*, ofputil_flow_removed*) <null> (libopflex_agent_renderer_openvswitch.so+0x1c9fd5)
noironetworks#5 opflexagent::SwitchConnection::receiveOFMessage() <null> (libopflex_agent_renderer_openvswitch.so+0x1a121b)
noironetworks#6 opflexagent::SwitchConnection::Monitor() <null> (libopflex_agent_renderer_openvswitch.so+0x1a40d8)
noironetworks#7 opflexagent::SwitchConnection::operator()() <null> (libopflex_agent_renderer_openvswitch.so+0x1a5bf5)
Signed-off-by: Gautam Venkataramanan <gautam.chennai@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This code gets triggered on handlePortStatusUpdate and would result in the
opflex agent getting stuck in an infinite loop because the iterator is
not being incremented in the while loop.
Simple way to repro the issue is to add or del the patch ports on br-fabric
example:
ovs-vsctl del-port br-fabric qpi2b4b79dd-6e
ovs-vsctl add-port br-fabric qpi2b4b79dd-6e -- set interface qpi2b4b79dd-6e type=patch options:peer=qpf2b4b79dd-6e
Or
ovs-vsctl del-port br-fabric patch-fabric-ex
ovs-vsctl add-port br-fabric patch-fabric-ex -- set interface patch-fabric-ex type=patch options:peer=patch-ex-fabric
Verified we come out of the loop with the fix.
Change-Id: Ia330e40942083e9d289e256b951cc803a35e4864
Signed-off-by: Madhu Challa challa@gmail.com
(cherry picked from commit f03095e)