Disable index hints on spatial indices #428
Conversation
…h in explain statements
Description
===========
When a spatial key is used as a hint in an explain query, the MySQL 5.7
engine crashes in the function "Explain_table_base::explain_key_and_len_index"
in the file sql/opt_explain.cc. This crash is due to MAX_INDEX = 64
being used as the key value in
"Explain_table_base::explain_key_and_len_index", which causes an
incorrect key_len value to be assigned leading to buffer overflow and
memory corruption in aforementioned function.
(gdb) bt
\#0 Explain_table_base::explain_key_and_len_index (this=0x7f3dde5f0260,
key=64, key_length=3315985466, key_parts=4) at /MySQL/sql/opt_explain.cc:903
Sometimes too setting the key to MAX_INDEX = 64 causes the
mysqld process to attempt to access inaccessible memory, leading
to a crash. Although the function "Explain_table_base::explain_key_and_len_index"
appears to have an assert to check that MAX_INDEX != 64, the assert does not fire
in the release build.
The reason key is set to MAX_INDEX = 64 is because the function
"TABLE_LIST::process_index_hints" in sql/table.cc sets the spatial key
from the index hint as the only usable key. This spatial key is then
rejected in the function "find_shortest_key" in sql/sql_optimizer.cc
because it is a spatial key. Instead "find_shortest_key" sets
MAX_INDEX = 64 as the best key to use, and eventually when
"Explain_table_base::explain_key_and_len_index" tries to use this best
key, a crash happens. See the excerpt below from "find_shortest_key":
if (key_ref.key_length < min_length && !(key_ref.flags & HA_SPATIAL)) {
min_length = key_ref.key_length;
best = nr;
}
To solve this problem we disable using spatial indices as hints in joins
from within the function "TABLE_LIST::process_index_hints" in
sql/table.cc.
Testing
=======
To test this change we added a new test case that provides a index hint on a join
to use a spatial index in an explain query. In the result, the spatial key is
ignored and instead a different key is selected to scan the table.
License
=======
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
|
Hi, thank you for submitting this pull request. In order to consider your code we need you to sign the Oracle Contribution Agreement (OCA). Please review the details and follow the instructions at https://oca.opensource.oracle.com/ |
|
I will not sign Oracle Contribution Agreement (OCA) on behalf of myself or my employer. Please accept this contribution without the OCA. |
|
Hi @lottaquestions Thanks |
|
Hi, there was no response to our request to sign an OCA or confirm the code is submitted under the terms of the OCA. As such this request will be closed. |
|
Reopening for continued processing |
|
Hi, thank you for your contribution. Please confirm this code is submitted under the terms of the OCA (Oracle's Contribution Agreement) you have previously signed by cutting and pasting the following text as a comment: |
|
This contribution is under the OCA signed by Amazon and covering submissions to the MySQL project. |
|
Hi, thank you for your contribution. Your code has been assigned to an internal queue. Please follow |
Description
When a spatial key is used as a hint in an explain query, the MySQL 5.7
engine crashes in the function "Explain_table_base::explain_key_and_len_index"
in the file sql/opt_explain.cc. This crash is due to MAX_INDEX = 64
being used as the key value in
"Explain_table_base::explain_key_and_len_index", which causes an
incorrect key_len value to be assigned leading to buffer overflow and
memory corruption in aforementioned function.
Sometimes too setting the key to MAX_INDEX = 64 causes the
mysqld process to attempt to access inaccessible memory, leading
to a crash. Although the function "Explain_table_base::explain_key_and_len_index"
appears to have an assert to check that MAX_INDEX != 64, the assert does not fire
in the release build.
The reason key is set to MAX_INDEX = 64 is because the function
"TABLE_LIST::process_index_hints" in sql/table.cc sets the spatial key
from the index hint as the only usable key. This spatial key is then
rejected in the function "find_shortest_key" in sql/sql_optimizer.cc
because it is a spatial key. Instead "find_shortest_key" sets
MAX_INDEX = 64 as the best key to use, and eventually when
"Explain_table_base::explain_key_and_len_index" tries to use this best
key, a crash happens. See the excerpt below from "find_shortest_key":
To solve this problem we disable using spatial indices as hints in joins
from within the function "TABLE_LIST::process_index_hints" in
sql/table.cc.
Testing
To test this change we added a new test case that provides a index hint on a join
to use a spatial index in an explain query. In the result, the spatial key is
ignored and instead a different key is selected to scan the table.