You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This PR refactors the AutoIncrementTracker to hold a separate mutex for each table instead of a single mutex for the entire database.
Why are we doing this?
This relates to supporting the innodb_autoinc_lock_mode variable, described here: #7634
Currently, the engine acquires the AutoIncrementTracker lock once per inserted row and holds it only while that row is being inserted. However, when innodb_autoinc_lock_mode is set to any value other than its default, the engine must instead hold the lock for the duration of the insert statement.
Unfortunately, there is a specific corner case that causes innodb_autoinc_lock_mode to interact poorly with a single global lock: if a table has a trigger, and that trigger attempts to insert into a different table, then a second insert statement begins executing before the first insert statement completes. In this case, the engine may attempt to acquire the global lock while it's already being held.
There are four possible ways to fix this:
1 - Use reentrant/recursive locks
2 - Analyze the entire execution plan to determine whether it will require the lock, and then acquire the lock at most once.
3 - Add additional logic to determine whether a session already holds the lock and avoid reacquiring it
4 - Use per-table locks
Option 1 is controversial: golang does not support recursive mutexes by design, and their use is discouraged by many. Option 2 will hold the lock for longer than is required, which would negatively impact performance. Option 3 is error prone and may result in deadlocks if implemented incorrectly.
Thus, option 4 is the best option.
What is the impact of this PR?
Concurrent inserts into two different tables with auto-increment columns should be faster because of reduced lock contention, at the cost of the additional overhead of managing multiple mutexes when the inserts are not concurrent. The magnitude of this impact is difficult to predict or measure, but is likely overall positive.
Why are there no tests?
This change should have no observable effect on query results. Our existing integration test suite is comprehensive enough that if no existing integration tests break, this should be safe.
We achieve per-table locking by managing a map from table names to mutexes, as well as using a synchronized map for storing the auto-increment counters. Both of these are necessary:
We require a synchronized map because concurrent accesses to normal golang maps are not thread-safe, since they may trigger a re-allocation.
We require the mutex map because some of the operations we perform on the counter map are not atomic. Thus, before any operation that reads or writes to the auto-increment value of a table, we acquire the lock corresponding to that table.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR refactors the AutoIncrementTracker to hold a separate mutex for each table instead of a single mutex for the entire database.
Why are we doing this?
This relates to supporting the
innodb_autoinc_lock_mode
variable, described here: #7634Currently, the engine acquires the AutoIncrementTracker lock once per inserted row and holds it only while that row is being inserted. However, when
innodb_autoinc_lock_mode
is set to any value other than its default, the engine must instead hold the lock for the duration of the insert statement.Unfortunately, there is a specific corner case that causes
innodb_autoinc_lock_mode
to interact poorly with a single global lock: if a table has a trigger, and that trigger attempts to insert into a different table, then a second insert statement begins executing before the first insert statement completes. In this case, the engine may attempt to acquire the global lock while it's already being held.There are four possible ways to fix this:
1 - Use reentrant/recursive locks
2 - Analyze the entire execution plan to determine whether it will require the lock, and then acquire the lock at most once.
3 - Add additional logic to determine whether a session already holds the lock and avoid reacquiring it
4 - Use per-table locks
Option 1 is controversial: golang does not support recursive mutexes by design, and their use is discouraged by many. Option 2 will hold the lock for longer than is required, which would negatively impact performance. Option 3 is error prone and may result in deadlocks if implemented incorrectly.
Thus, option 4 is the best option.
What is the impact of this PR?
Concurrent inserts into two different tables with auto-increment columns should be faster because of reduced lock contention, at the cost of the additional overhead of managing multiple mutexes when the inserts are not concurrent. The magnitude of this impact is difficult to predict or measure, but is likely overall positive.
Why are there no tests?
This change should have no observable effect on query results. Our existing integration test suite is comprehensive enough that if no existing integration tests break, this should be safe.