Skip to content

Commit

Permalink
[upstream] Disable collect own gtids for slave threads
Browse files Browse the repository at this point in the history
Summary:
This is an upstream bug: https://bugs.mysql.com/bug.php?id=92964

The issue is that if slave instance use sysvar_session_track_gtids == OWN_GTID and also enable MTS, the slave will lag behind master and its performance degrades over time.

The reason for this lag/perf degrade is caused by each slave worker keep adding its own gtid into its own Session_consistency_gtids_ctx.m_gtid_set during each transaction commit. Overtime, some of Session_consistency_gtids_ctx.m_gtid_set may have millions gno_interval and it will take long time to insert 1 gtid into Session_consistency_gtids_ctx.m_gtid_set(time complexity O(n)), see [Gtid_set::add_gno_interval](https://github.com/facebook/mysql-5.6/blob/fb-mysql-8.0.13/sql/rpl_gtid_set.cc#L323).

There are couple related code to this issue:
- During slave worker thread start, there is a [check](https://github.com/facebook/mysql-5.6/blob/fb-mysql-8.0.13/sql/session_tracker.cc#L1560) to not enable tracker for system threads but that check doesn't work---The THD system_thread field is initialized with default value(NON_SYSTEM_THREAD) during THD::THD.
```
  m_enabled = thd->variables.session_track_gtids != OFF &&
              /* No need to track GTIDs for system threads. */
              thd->system_thread == NON_SYSTEM_THREAD;
```
```
Call stack:
```
- in [Session_gtids_track::Update](https://github.com/facebook/mysql-5.6/blob/fb-mysql-8.0.13/sql/session_tracker.cc#L1581), the else code doesn't unregister listener due to mysql/mysql-server@1e0844c try to fix another issue.

Solutions:
1.  [Current]add system threads check during collecting data, since THD will be fully initialized at that time.
 -Pro: simple change(easy to port) and perf is ok
2. add `thd->session_track_gtids=OFF; thd->session_tracker.get_tracker(SESSION_GTIDS_TRACKER)->update(thd);` after [init_slave_thread()](https://github.com/facebook/mysql-5.6/blob/fb-mysql-8.0.13/sql/rpl_slave.cc#L6788)
-Pro: During init_slave_thread(), thd->system_thread will be set correctly value.
-Con: it need to add couple places, such as handle_slave_io, handle_slave_sql, handle_slave_worker
3. add `thd->session_track_gtids=OFF;thd->session_tracker.get_tracker(SESSION_GTIDS_TRACKER)->update(thd);` inside [init_slave_thread()](https://github.com/facebook/mysql-5.6/blob/fb-mysql-8.0.13/sql/rpl_slave.cc#L6788)
   -Pro: only add once for slave/slave worker/IO threads
   -Con: [should_collect](https://github.com/facebook/mysql-5.6/blob/fb-mysql-8.0.13/sql/rpl_context.cc#L53) check still return true, thus it is less perf than #1
4. add `thd->session_track_gtids=OFF;thd->rpl_thd_ctx.session_gtids_ctx().unregister_ctx_change_listener(thd->session_tracker.get_tracker(SESSION_GTIDS_TRACKER));` inside init_slave_thread()
   -Pro: perf is better than #3, on par as #1
   -Con: if there are other non-system threads(except IO/sql threads) do transactions, these threads will still collect its own gtids.
5. add another THD overload constructor which takes 'enum_thread_type' as a parameter whose default value is NON_SYSTEM_THREAD
 -Pro: it will initialize correct value with THD and won't enable those session_tracker
 -Con: similar to #2/#4, need to replace THD() with THD(enum_thread_type) couples places.

Differential Revision: D18072672
  • Loading branch information
luqun authored and inikep committed Apr 15, 2024
1 parent 2e0e6a8 commit e7d9343
Showing 1 changed file with 3 additions and 0 deletions.
3 changes: 3 additions & 0 deletions sql/rpl_context.cc
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,9 @@ inline bool Session_consistency_gtids_ctx::shall_collect(const THD *thd) {
m_curr_session_track_gtids == SESSION_TRACK_GTIDS_ALL_GTIDS) &&
/* if there is no listener/tracker, then there is no reason to collect */
m_listener != nullptr &&
/* No need to track GTIDs for system threads to avoid performance issues
like replication lag */
thd->system_thread == NON_SYSTEM_THREAD &&
/* ROLLBACK statements may end up calling trans_commit_stmt */
thd->lex->sql_command != SQLCOM_ROLLBACK &&
thd->lex->sql_command != SQLCOM_ROLLBACK_TO_SAVEPOINT;
Expand Down

0 comments on commit e7d9343

Please sign in to comment.