Skip to content

2.25.2.0-b127

2.25.2.0-b127: [#26320] docdb: Fix data race setting up TServerSharedData
@es1024 es1024 tagged this 10 Mar 14:44 
Summary:
After f85bbca, TServerSharedData is now set up as the prepare user data of the shared
memory allocator. TServerSharedData is initially mapped to a temporary address before address
negotiation runs, then remapped to the negotiated address once postmaster starts and we negotiate
addresses. But other code may access TServerSharedData both before and after we remap, and thus
there is a data race on the pointer to TServerSharedData between code using TServerSharedData and
code remapping it.

Additionally, it is possible for a pointer to the temporary address to be retrieved, then shared
memory remapped, then pointer to temporary address used, resulting in a SIGSEGV.

This diff puts access to the TServerSharedData pointer behind URCU (ConcurrentValue) and
additionally delays unmapping the temporary address until we have reached a quiescent state after
updating the TServerSharedData pointer.

This fixes TSAN failures from the above data race, with the following trace:
```
[ts-2] WARNING: ThreadSanitizer: data race (pid=10713)
[ts-2]   Write of size 8 at 0x721c00001db8 by thread T48:
[ts-2]     #0 yb::tserver::SharedMemoryManager::InitializeParentAllocatorsAndObjects() ${YB_SRC_ROOT}/src/yb/tserver/tserver_shared_mem.cc:328:9 (libtserver_shared.so+0x11f66)
[ts-2]     #1 yb::tserver::SharedMemoryManager::ExecuteParentNegotiator(std::shared_ptr<yb::AddressSegmentNegotiator> const&) ${YB_SRC_ROOT}/src/yb/tserver/tserver_shared_mem.cc:318:3 (libtserver_shared.so+0x11f66)

[ts-2]   Previous read of size 8 at 0x721c00001db8 by thread T41 (mutexes: write M0):
[ts-2]     #0 yb::tserver::SharedMemoryManager::SharedData() const ${YB_SRC_ROOT}/src/yb/tserver/tserver_shared_mem.h:182:13 (libtserver.so+0x26623d)
[ts-2]     #1 yb::tserver::DbServerBase::shared_object() const ${YB_SRC_ROOT}/src/yb/tserver/db_server_base.cc:136:31 (libtserver.so+0x26623d)
[ts-2]     #2 yb::tserver::TabletServer::SetYsqlDBCatalogVersions(yb::master::DBCatalogVersionDataPB const&) ${YB_SRC_ROOT}/src/yb/tserver/tablet_server.cc:1126:7 (libtserver.so+0x3fafc8)
[ts-2]     #3 yb::tserver::HeartbeatPoller::TryHeartbeat() ${YB_SRC_ROOT}/src/yb/tserver/heartbeater.cc:499:15 (libtserver.so+0x278e87)
[ts-2]     #4 yb::tserver::HeartbeatPoller::Poll() ${YB_SRC_ROOT}/src/yb/tserver/heartbeater.cc:273:19 (libtserver.so+0x276143)
[ts-2]     #5 yb::tserver::MasterLeaderPollScheduler::Impl::Run() ${YB_SRC_ROOT}/src/yb/tserver/master_leader_poller.cc:133:25 (libtserver.so+0x27ec33)
```
Jira: DB-15671

Test Plan: Jenkins

Reviewers: sergei

Reviewed By: sergei

Subscribers: zdrudi, ybase, yql

Differential Revision: https://phorge.dev.yugabyte.com/D42395
Assets 2
Loading