MM-23369: Allow mysql to choose a better index #14119
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
When the ORDER BY clause contains a column which is in the WHERE clause and also part of an index, mysql tries to use that specific index to avoid sorting. This is inspite of the fact that there may be other indices which are better for scanning the table and then doing a sort. Essentially, mysql becomes dumb and scans a lot of rows to avoid sorting. Whereas, it could have scanned a lot less rows and do the sorting in no time. To fix this, we use the other columns in the ORDER BY clause as well which are part of the index. This causes no change in the results because the other columns are an EQUAL condition check, but this lets mysql use the right index. Because now mysql sees that it has to order by other columns too, so it better use the other index to scan and then do the sorting. This does not affect tables of smaller size because the LIMIT of rows is always 1. And mysql will stop sorting the moment it gets the first row. So sorting is not the overhead at all. Therefore, this seems like an optimal fix. References: https://dev.mysql.com/doc/refman/5.7/en/table-scan-avoidance.html https://code.openark.org/blog/mysql/7-ways-to-convince-mysql-to-use-the-right-index https://dev.mysql.com/doc/refman/5.7/en/limit-optimization.html
agnivade
added
2: Dev Review
|
@lieut-data - Thought you would be interested in this one. |
agnivade
added
the
Do Not Merge/Awaiting Loadtest
lieut-data
approved these changes
Mar 23, 2020
store/sqlstore/post_store.go
Outdated
| // Adding ChannelId and DeleteAt order columns | ||
| // to let mysql choose the "idx_posts_channel_id_delete_at_create_at" index always. | ||
| // See MM-23369. | ||
| OrderBy("ChannelId, DeleteAt, CreateAt " + sort). |
There was a problem hiding this comment.
nit: might we write this as follows to minimize the "manual SQL" and maximize Squirrel's flexibility?
Suggested change
| OrderBy("ChannelId, DeleteAt, CreateAt " + sort). | |
| OrderBy("ChannelId", "DeleteAt", "CreateAt " + sort). |
lindy65
added
QA/Review Done
jespino
added
the
4: Reviews Complete
|
Load test done. No difference found: |
agnivade
removed
the
Do Not Merge/Awaiting Loadtest
amyblais
added
the
CherryPick/Approved
|
/cherry-pick release-5.23 |
mattermod
added
the
CherryPick/Done
mattermod
removed
the
CherryPick/Approved
agnivade
added a commit
that referenced
this pull request
Apr 20, 2020
* MM-23369: Allow mysql to choose a better index When the ORDER BY clause contains a column which is in the WHERE clause and also part of an index, mysql tries to use that specific index to avoid sorting. This is inspite of the fact that there may be other indices which are better for scanning the table and then doing a sort. Essentially, mysql becomes dumb and scans a lot of rows to avoid sorting. Whereas, it could have scanned a lot less rows and do the sorting in no time. To fix this, we use the other columns in the ORDER BY clause as well which are part of the index. This causes no change in the results because the other columns are an EQUAL condition check, but this lets mysql use the right index. Because now mysql sees that it has to order by other columns too, so it better use the other index to scan and then do the sorting. This does not affect tables of smaller size because the LIMIT of rows is always 1. And mysql will stop sorting the moment it gets the first row. So sorting is not the overhead at all. Therefore, this seems like an optimal fix. References: https://dev.mysql.com/doc/refman/5.7/en/table-scan-avoidance.html https://code.openark.org/blog/mysql/7-ways-to-convince-mysql-to-use-the-right-index https://dev.mysql.com/doc/refman/5.7/en/limit-optimization.html * Added a comment to clarify things in code * Incorporating review comments Co-authored-by: Agniva De Sarker <agnivade@yahoo.co.in>
amyblais
added
Changelog/Not Needed
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
When the ORDER BY clause contains a column which is in the WHERE
clause and also part of an index, mysql tries to use that specific
index to avoid sorting. This is inspite of the fact that
there may be other indices which are better for scanning the table
and then doing a sort.
Essentially, mysql becomes dumb and scans a lot of rows to avoid
sorting. Whereas, it could have scanned a lot less rows and do the
sorting in no time.
To fix this, we use the other columns in the ORDER BY clause as well
which are part of the index. This causes no change in the results
because the other columns are an EQUAL condition check, but this
lets mysql use the right index. Because now mysql sees that it has
to order by other columns too, so it better use the other index
to scan and then do the sorting.
This does not affect tables of smaller size because the LIMIT of
rows is always 1. And mysql will stop sorting the moment it gets
the first row. So sorting is not the overhead at all.
Therefore, this seems like an optimal fix.
References:
https://dev.mysql.com/doc/refman/5.7/en/table-scan-avoidance.html
https://code.openark.org/blog/mysql/7-ways-to-convince-mysql-to-use-the-right-index
https://dev.mysql.com/doc/refman/5.7/en/limit-optimization.html
Ticket Link
https://mattermost.atlassian.net/browse/MM-23369