Merge 5.6.29 #380
Merged
Merge 5.6.29 #380
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Adds wily (15.10) packaging
…E_SESSION_CONNECT.CC:254 Before this fix: BACKGROUND threads are not supposed to have connection attributes and therefore an ASSERT was in place to make sure this code point is not reached from a BACKGROUND thread. But when server is started with --thread-handling=no-threads, no FOREGROUND thread is created for client connection and this code point is reached when a query to table session_connect_attrs is made, and an ASSERT was seen. After this fix: Instead of an ASSERT, silently return from the function.
DOING BAD DDL IN PREPARED STATEMENT Analysis ======== A repeat execution of the prepared statement 'ALTER TABLE v1 CHECK PARTITION' where v1 is a view leads to server exit. ALTER TABLE ... CHECK PARTITION is not applicable for views and check for the same check is missing. This leads to further execution and creation of derived table for the view (Allocated under temp_table mem_root). Any reference to open view or related pointers from second execution leads to server exit as the same was freed at previous execution closure. Fix: ====== Added check for view in mysql_admin_table() on PARTITION operation. This will prevent mysql_admin_table() from going ahead and creating temp table and related issues. Changed message on admin table view operation error to be more appropriate.
HA_MYISAMMRG.CC:631 Analysis ======== Any attempt to open a temporary MyISAM merge table consisting of a view in its list of tables (not the last table in the list) under LOCK TABLES causes the server to exit. Current implementation doesn't perform sanity checks during merge table creation. This allows merge table to be created with incompatible tables (table with non-myisam engine), views or even with table doesn't exist in the system. During view open, check to verify whether requested view is part of a merge table is missing under LOCK TABLES path in open_table(). This leads to opening of underlying table with parent_l having NULL value. Later when attaching child tables to parent, this hits an ASSERT as all child tables should have parent_l pointing to merge parent. If the operation does not happen under LOCK TABLES mode, open_table() checks for view's parent_l and returns error. Fix: ====== Check added before opening view Under LOCK TABLES in open_table() to verify whether it is part of merge table. Error is returned if the view is part of a merge table.
SMALLER ON SLAVE FOR 3 NODE TOPOLOGY Bug#21053163: MIXED BASED REPLICATION LOOSES EVENTS WHEN RELAY_LOG_INFO_REPOSITORY=TABLE Problem: ======= 2 level replication M1 -> S1 ->S2 ( S1 is slave of M1; S2 is slave of S1) replicating a non-transactional storage engine table (e.g.MyISAM) when set relay_log_info_repository=TABLE; and binlog rotation occurs in the middle of statement that was translated to multiple rows, then you loose part of that events. When binlog rotation occurs, on S1 not all rows are written to it's binlog, therefore S2 seamlessly looses part of rows that were translated from one statement to several rows. Fix: === The above mentioned bugs got fixed as part of BUG#16418100 fix. Adding additional test cases to improve test coverage.
INCONSISTENCY Problem: ======= If generating a GTID for a transaction fails, the transaction is not written to the binary log but still gets committed, which potentially leads to master/slave data inconsistency. Analysis: ======== Running out of GTID numbers is a very rare scenario, but if that happens that will be a fatal error and we advise users to 'Restart the server with a new server_uuid'. GTID's are generated at the time of flushing binlog cache to actual binary log. In the existing code if generation of GTID fails, client will get an appropriate error message and the action specified as per the binlog_error_action will be taking place as this is a flush stage error. i.e if the user chose binlog_error_action to be 'IGNORE_ERROR', then binlog will be disabled and transactions will continue to get committed on master and this will lead to an inconsistent slave. If the user chose binlog_error_action to be 'ABORT_SERVER' then server will abort without commiting the transaction on master so it will be consistent with slave. This behavior is the most appropriate one. At present while displaying the error message in error log, it is displayed as 'sync' stage error but it should be a 'flush' stage error. Fix: === During flush stage if generation of GTID fails then we set thd->commit_error= THD::CE_FLUSH_ERROR, so that the error message is accurate.
… 5.6 PB2. Issue: ====== Change in join order in multi update statement can produce different results. main reason bug 16767011 and 18449085. But these fixes are only made on 5.7 back-porting same thing on 5.6 is little risky as considering to use temp table for first table in join order is done before conds are rearranged and pre-processed in make_join_select. Where as in 5.7+ such decision happens after make_join_select. This particular change from 5.7 to 5.6 appears risky hence not back-porting the same. Solution: ========= Re-writing the tests in 5.6 so it produces consistent plan (join orders). This solves sporadic failures.
Problem: SP local variables that are used in ALTER EVENT
are not replicated properly.
Analysis: CALL statements are not written into binary log. Instead
each statement executed in SP is binlogged separately with the exception
that we modify query string: we replace uses of SP local variables with
NAME_CONST('spvar_name', <spvar-value>) calls. The code was written
under the assumption that this replacement was not required for at all
for all the queries in 'row' based format. But it can happen that DDLs
(which are always binlogged in statement mode irrespective of binlog
format) can also use SP local variables and they suffer due to the
above assumption. 'ALTER EVENT' in this bug case is one such cases
Fix: Any SP local variables used in a query should
be replaced with NAME_CONST(...) except for the case when it is DML
query and binlog format is 'ROW'.
…NFORMATION Removed two files that contained duplicate information: INSTALL-WIN-SOURCE and BUILD-CMAKE. Renamed INSTALL-SOURCE to INSTALL. Updated the information with correct links for different versions. Approved-by: Jon Hauglid <jon.hauglid@oracle.com> Approved-by: Terje Rosten <terje.rosten@oracle.com>
FAILURE WITH VALGRIND FOR RELEASE BUILD Issue: ------ Initialization of variable with UNINIT_VAR is flagged by valgrind 3.11. SOLUTION: --------- Initialize the variable to 0. This is a backport of Bug# 14580121.
Problem & Analysis: If DML invokes a trigger or a
stored function that inserts into an AUTO_INCREMENT column,
that DML has to be marked as 'unsafe' statement. If the
tables are locked in the transaction prior to DML statement
(using LOCK TABLES), then the same statement is not marked as
'unsafe' statement. The logic of checking whether unsafeness
is protected with if (!thd->locked_tables_mode). Hence if
we lock the tables prior to DML statement, it is *not* entering
into this if condition. Hence the statement is not marked
as unsafe statement.
Fix: Irrespective of locked_tables_mode value, the unsafeness
check should be done. Now with this patch, the code is moved
out to 'decide_logging_format()' function where all these checks
are happening and also with out 'if(!thd->locked_tables_mode)'.
Along with the specified test case in the bug scenario
(BINLOG_STMT_UNSAFE_AUTOINC_COLUMNS), we also identified that
other cases BINLOG_STMT_UNSAFE_AUTOINC_NOT_FIRST,
BINLOG_STMT_UNSAFE_WRITE_AUTOINC_SELECT, BINLOG_STMT_UNSAFE_INSERT_TWO_KEYS
are also protected with thd->locked_tables_mode which is not right. All
of those checks also moved to 'decide_logging_format()' function.
WHERE INNODB WRITES TEMP FILES Problem: ======== InnoDB creates temporary files for online ALTER statements in the tmpdir. In some cases, the tmpdir is too small, or for other reasons, not the best choice. Solution: ========= Create a new dynamic session variable "innodb_tmpdir" that would determine where the temp files should create during alter operation. Behaviour of innodb_tmpdir : =========================== 1) Default value is NULL. 2) Valid inputs are any path other than mysql data directory path. 3) Directory Permission and existence checked as a part of validation for innodb_tmpdir. 4) If value is set to NULL, then temporary file created in the location of mysql server variable(--tmpdir). 5) User should have file privilege to set the variable. 6) If user provides a path which is symlink, then we resolve and store absolute path in innodb_tmpdir. 7) Path should not exceed 512 bytes. 8) Path should be a directory. Reviewed-by: Marko Mäkelä<marko.makela@oracle.com> Reviewed-by: Harin Vadodaria<harin.vadodaria@oracle.com> Reviewed-by: Jon Olav Hauglid<jon.hauglid@oracle.com> RB: 7628
Query below produces valgrind warning: select a1,a2,b,min(c) from t2 where ((a1 > 'a') or (a1 < '9')) and (a2 >= 'b') and (b = 'a') and (c < 'h112') group by a1,a2,b; Problematic expession is (c < 'h112'). SEL_ARG::min_range is is_null_string which has length=2, memcmp is executed with the length which is bigger than is_null_string. The fix: Do not perform comparison if one of the argument is NULL value.
…YSQL_OPT_READ_TIMEOUT Description: SSL connection not considering the value specified by MYSQL_OPT_READ_TIMEOUT. Connect the client to server with MYSQL_OPT_READ_TIMEOUT lesser (i.e like 5 seconds) then the mysql_query() execution time(i.e in mysql_query() have query as "SELECT SLEEP(10)"). For the above scenario we you get "Lost connection to MySQL server during query" error. And this is happening in non-ssl case. But in SSL connection case we are getting the query execution is successful. Analysis: When we set the MYSQL_OPT_READ_TIMEOUT as '5' under SSL connection, we call my_net_set_read_timeout() in mysql_real_connect(i.e CLI_MYSQL_REAL_CONNECT) and converts the vio->read_timeout to milliseconds and further we call run_plugin_auth() and further so on to ssl_do(), there we will make new vio and copy the timeout values to it. But here we have the timeouts already in milliseconds. And we are passing those milliseconds(as seconds) to vio_timeout(). Thus further those will be converted to incorrect milliseconds which leads to having 15000000 milliseconds instead of 15000 milliseconds in case of SSL connections. Hence the read_timeout for the query is more and we are getting the results successfully. Fix: Since the updated value of vio->read_timeout and vio->write_timeout value always be in milliseconds, we are converting the milliseconds value of vio->read_timeout in vio_reset to seconds and calling vio_timeout().
WITH NEW VALGRIND Issue: ------ Function signature in valgrind.supp requires a change with valgrind 3.11. Static function is replaced with wild card.
FAILS USING 5.6 GTID
Problem: When there are more than one drop temp table in stored function
or trigger, replication is failing when GTIDs are enabled.
Analysis: In ROW based replication format, even though CREATE TEMPORARY
TABLE query is not replicated, DROP TEMPORARY TABLE queries
are replicated to achieve proper clean up on Slave end (CREATE
TEMPORARY TABLE query would have executed and replicated when
the replication format is STATEMENT) by adding 'IF EXISTS'
clause. When DROP TEMPORARY TABLE query is in a stored function
along with some DML statements, the binlog equivalent query
for that function execution will look like
BEGIN
DROP TEMP TABLE ...
ROW EVENT FOR DML 1
ROW EVENT FOR DML 2
END
But when GTIDs are enabled, it is documented that CREATE/DROP
TEMPORARY TABLE queries are not allowed in Multi Statement
Transactions because half executed gtid transactions (rolled
back of these transactions) can leave these temporary tables
in a bad state.
In the old code, one DROP TEMPORARY TABLE in a function is
working fine because the 'DROP TEMP TABLE' is going into
STMT_CACHE (which does not be wrapped with BEGIN/COMMIT).
//STMT_CACHE
GTID_EVENT
DROP TEMP TABLE ...
//TRANS_CACHE
GTID_EVENT
BEGIN
ROW EVENT FOR DML 1
ROW EVENT FOR DML 2
END
But if the function contains two 'DROP TEMP TABLE's, both
of them are going into 'STMT_CACHE' (which does not be wrapped
with BEGIN/COMMIT) and STMT_CACHE with one gtid_event cannot
accommodate two separate DROP TEMP TABLE queries. And with above
Multi Statement Transactions + GTID restriction, we cannot
add 'BEGIN/COMMIT'.
Fix: Stored functions and Triggers are also considered as another form of
Multi Statement Transactions across the server. To maintain gtid
consistency and to avoid the problems that are mentioned in this bug
scenario, CREATE/DROP temp tables are disallowed from stored functions
and triggers also just like how they were restricted in Multi Statement
Transactions. Now function execution that has CREATE/DROP TEMP TABLES
will throw ER_GTID_UNSAFE_CREATE_DROP_TEMPORARY_TABLE_IN_TRANSACTION.
("When @@GLOBAL.ENFORCE_GTID_CONSISTENCY = 1, the statements CREATE
TEMPORARY TABLE and DROP TEMPORARY TABLE can be executed in a
non-transactional context only, and require that AUTOCOMMIT = 1. These
statements are not allowed in Functions or Triggers also as they are also
considered as Multi Statement transaction.)
…SQL/SQL_OPTIMIZER.CC:1657 Problem: At the end of first execution select_lex->prep_where is pointing to a runtime created object (temporary table field). As a result server exits trying to access a invalid pointer during second execution. Analysis: While optimizing the join conditions for the query, after the permanent transformation, optimizer makes a copy of the new where conditions in select_lex->prep_where. "prep_where" is what is used as the "where condition" for the query at the start of execution. W.r.t the query in question, "where" condition is actually pointing to a field in the temporary table. As a result, for the second execution the pointer is no more valid resulting in server exit. Fix: At the end of the first execution, select_lex->where will have the original item of the where condition. Make prep_where the new place where the original item of select->where has to be rolled back. Fixed in 5.7 with the wl#7082 - Move permanent transformations from JOIN::optimize to JOIN::prepare Patch for 5.5 includes the following backports from 5.6: Bugfix for Bug12603141 - This makes the first execute statement in the testcase pass in 5.5 However it was noted later in in Bug16163596 that the above bugfix needed to be modified. Although Bug16163596 is reproducible only with changes done for Bug12582849, we have decided include the fix. Considering that Bug12582849 is related to Bug12603141, the fix is also included here. However this results in Bug16317817, Bug16317685, Bug16739050. So fix for the above three bugs is also part of this patch.
WHERE INNODB WRITES TEMP FILES - Post push fix to address pb2 failure.
OPEN SSL TRACE Open ssl has valgrind issues. The relevant stacktraces have been added to valgrind.supp. This is a backport from 5.7: 2ad2964fe0c254ee77c402b674befaa216f6cf20
"MALFORMED COMMUNICATION PACKET" ERROR Post push to fix valgrind test case failure in pb2.
Problem Statement ========= Fix various issues when building MySQL with Visual Studio 2015. Fix: ======= - Visual Studio 2015 adds support for timespec. Add check and related code to use this and only use our replacement if timespec is not defined. - Rename lfind/lsearch to my* to avoid redefinition problems. - Set default value for TMPDIR to "" on Windows as P_tmpdir no longer exists. - using VS definition of snprintf if available - tzname are now renamed to _tzname.
Problem Statement ========= Fix various issues when building MySQL with Visual Studio 2015. Fix: ======= - Visual Studio 2015 adds support for timespec. Add check for this and only use our replacement if timespec is not defined. - Rename lfind/lsearch to my* to avoid redefinition problems. - Set default value for TMPDIR to "" on Windows as P_tmpdir no longer exists. - using VS definition of snprintf if available - tzname are now renamed to _tzname. - This patch raises the minimum version required of WiX Toolkit to 3.8, which is required to make MSI packages with Visual Studio 2015
PROCEDURE RESULTS IN GARBAGE BYTES Issue: ----- This problem occurs under the following conditions: a) Stored procedure has a variable is declared as TEXT/BLOB. b) Data is copied into the the variable using the SELECT...INTO syntax from a TEXT/BLOB column. Data corruption can occur in such cases. SOLUTION: --------- The blob type does not allocate space for the string to be stored. Instead it contains a pointer to the source string. Since the source is deallocated immediately after the select statement, this can cause data corruption. As part of the fix for Bug #21143080, when the source was part of the table's write-set, blob would allocate the neccessary space. But this fix missed the possibility that, as in the above case, the target might be a variable. The fix will add the copy_blobs check that was removed by the earlier fix.
…BROKEN Our binaries depend on libstlport, which is not part of Solaris by default, so we ship it as part of mysql packages. The problem was that this dependency was not noted in the client library. Fix: bacport more from: Bug#16555106 FIX BROKEN BUILD WITH SOLARIS/GCC 64BIT MODE Specifically: add -R$ORIGIN/../lib when linking the client library.
Conflicts: sql-common/client.c
Test fix for 5.5 and 5.6
we can see from the hang stacktrace, srv_monitor_thread is blocked when getting log_sys::mutex, so that sync_arr_wake_threads_if_sema_free cannot get a change to break the mutex deadlock. The fix is simply removing any mutex wait in srv_monitor_thread. Patch is reviewed by Sunny over IM.
|
Same comments re. bug 1521120 and the merge blueprint as in the 5.5 merge. One more comment, as https://blueprints.launchpad.net/percona-server/+spec/merge-5.6.29 says, it should be checked whether testcase for http://bugs.mysql.com/bug.php?id=78292 is there. I am unable to find it (grepping by "no-threads"), if this is correct, can you please add it? |
|
Man pages not imported? |
vlad-lesin
force-pushed
the
merge-5.6.29
branch
2 times, most recently
from
ddfc1ab
to
c5203c5
Compare
Done.
Yes, you are right, I missed it. I added the test in perfschema.no_threads.
Imported now. |
|
threadpool_common.cc:threadpool_remove_connection: should dec_connection_count call be moved after the remove_global_thread call, for 3b633ba ? |
|
Vlad, did you test perfschema.no_threads? I have added your snipped to the 5.7.11 merge in progress and get --- /Users/laurynas/percona/mysql-server/mysql-test/suite/perfschema/r/no_threads.result 2016-03-01 15:08:55.000000000 +0300 Looking at the other testcases in perfschema that query this table I see that they never record its exact contents. |
The tests for 74670, 76277 bug fixes are added. Bug lp:1521120 is also fixed. The test for bug 78292 is added in perfschema.no_threads. log_slow_applicable() and query_response_time_audit_notify() are fixed to avoid error of casting from Query_arena* to sp_lex_instr*. The fix is in adding thd->sp_runtime_ctx check before the casting to be sure we are inside of SP and thd->stmt_arena can be casted to sp_lex_instr*. Some replication test results were fixed due to this commit: 907a7cc - Bug#21317739: APPLYING CREATE TEMPORARY TABLE SQL ON A SLAVE WITH REPLICATE-REWRITE-DB FAILScommit. Conflicts: README VERSION mysql-test/r/mysql_config_editor.result mysql-test/suite/rpl/r/rpl_mixed_implicit_commit_binlog.result mysql-test/suite/rpl/r/rpl_temp_table_mix_row.result mysql-test/valgrind.supp scripts/mysql_config.sh sql/mysqld.cc sql/mysqld.h sql/scheduler.cc sql/sql_base.cc storage/innobase/handler/ha_innodb.cc
|
New testing: http://jenkins.percona.com/view/PS%205.6/job/percona-server-5.6-param/1083/
Yes, also condition "extra_connections_count > 0" is added in close_connections() for waiting for all connections to be closed loop.
Yes, I have tested, but the query returned empty result. Anyway, the result of the query does not matter, the fact of the query execution without crash does matter. That's why I just added "--disable-result-log" command in the test code. |
|
LGTM |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
http://jenkins.percona.com/view/PS%205.6/job/percona-server-5.6-param/1082/