2.3 #9

akopytov · 2015-02-20T07:56:38Z

No description provided.

Removed files erroneously resurrected on git import.

Release 2.2.9

2.3

…QL_CMD)-> GET_XA_OPT() Bug#22273964 INNODB: FAILING ASSERTION: TOTAL_TRX >= TRX_SYS->N_PREPARED_TRX **Problem description** An assertion of Bug#21942487 static_cast<Sql_cmd_xa_commit*>(thd->lex->m_sql_cmd)-> get_xa_opt() == XA_ONE_PHASE #8 0x0000000000e3a5e1 in ha_commit_low #9 0x00000000015158e6 in TC_LOG_DUMMY::commit #10 0x0000000001529bc3 in Sql_cmd_xa_commit::trans_xa_commit was caused by incorrect assumption by XA binlogging of wl6860 in that @@session.pseudo_slave_mode can be only set 1 through binlog. If fact the var can be set so manually. Another assert in Bug#22273964 takes place for the very same reason. **Solution** The most reliable way to identify that the executing thread is a binlog applier must include both the checking of rli_fake and @@session.pseudo_slave_mode. The former merely checking of the session variable as atttemp to identify the binlog applier is replaced by checking the conjuction of the two properties. Notice that load containting SET @@session.pseudo_slave_mode=1 and BINLOG '' pseudo-queries is not necessary authentic to mysqlbinlog output. Nevertheless it must be processable when it's manually engineered such way. A test is included to prove that as well. **Note** Beware of Bug #19502202 SERVER SHUTDOWN HANG SEEN IN SOME INNODB & RPL TESTS at testing with binlog.binlog_xa_prepared_disconnect that still fails sporadically.

Fixing backup dirs inside data dirs in testcases

…E TO A SERVER Problem ======================================================================== Running the GCS tests with ASAN seldomly reports a user-after-free of the server reference that the acceptor_learner_task uses. Here is an excerpt of ASAN's output: ==43936==ERROR: AddressSanitizer: heap-use-after-free on address 0x63100021c840 at pc 0x000000530ff8 bp 0x7fc0427e8530 sp 0x7fc0427e8520 WRITE of size 8 at 0x63100021c840 thread T3 #0 0x530ff7 in server_detected /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/xcom_transport.c:962 #1 0x533814 in buffered_read_bytes /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/xcom_transport.c:1249 #2 0x5481af in buffered_read_msg /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/xcom_transport.c:1399 #3 0x51e171 in acceptor_learner_task /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/xcom_base.c:4690 #4 0x562357 in task_loop /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/task.c:1140 #5 0x5003b2 in xcom_taskmain2 /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/xcom_base.c:1324 #6 0x6a278a in Gcs_xcom_proxy_impl::xcom_init(unsigned short, node_address*) /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/gcs_xcom_proxy.cc:164 #7 0x59b3c1 in xcom_taskmain_startup /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/gcs_xcom_control_interface.cc:107 #8 0x7fc04a2e4dd4 in start_thread (/lib64/libpthread.so.0+0x7dd4) #9 0x7fc047ff2bfc in __clone (/lib64/libc.so.6+0xfebfc) 0x63100021c840 is located 64 bytes inside of 65688-byte region [0x63100021c800,0x63100022c898) freed by thread T3 here: #0 0x7fc04a5d7508 in __interceptor_free (/lib64/libasan.so.4+0xde508) #1 0x52cf86 in freesrv /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/xcom_transport.c:836 #2 0x52ea78 in srv_unref /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/xcom_transport.c:868 #3 0x524c30 in reply_handler_task /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/xcom_base.c:4914 #4 0x562357 in task_loop /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/task.c:1140 #5 0x5003b2 in xcom_taskmain2 /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/xcom_base.c:1324 #6 0x6a278a in Gcs_xcom_proxy_impl::xcom_init(unsigned short, node_address*) /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/gcs_xcom_proxy.cc:164 #7 0x59b3c1 in xcom_taskmain_startup /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/gcs_xcom_control_interface.cc:107 #8 0x7fc04a2e4dd4 in start_thread (/lib64/libpthread.so.0+0x7dd4) previously allocated by thread T3 here: #0 0x7fc04a5d7a88 in __interceptor_calloc (/lib64/libasan.so.4+0xdea88) #1 0x543604 in mksrv /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/xcom_transport.c:721 #2 0x543b4c in addsrv /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/xcom_transport.c:755 #3 0x54af61 in update_servers /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/xcom_transport.c:1747 #4 0x501082 in site_install_action /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/xcom_base.c:1572 #5 0x55447c in import_config /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/site_def.c:486 #6 0x506dfc in handle_x_snapshot /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/xcom_base.c:5257 #7 0x50c444 in xcom_fsm /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/xcom_base.c:5325 #8 0x516c36 in dispatch_op /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/xcom_base.c:4510 #9 0x521997 in acceptor_learner_task /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/xcom_base.c:4772 #10 0x562357 in task_loop /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/task.c:1140 #11 0x5003b2 in xcom_taskmain2 /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/xcom/xcom_base.c:1324 #12 0x6a278a in Gcs_xcom_proxy_impl::xcom_init(unsigned short, node_address*) /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/gcs_xcom_proxy.cc:164 #13 0x59b3c1 in xcom_taskmain_startup /home/tvale/mysql/plugin/group_replication/libmysqlgcs/src/bindings/xcom/gcs_xcom_control_interface.cc:107 #14 0x7fc04a2e4dd4 in start_thread (/lib64/libpthread.so.0+0x7dd4) Analysis ======================================================================== The server structure is reference counted by the associated sender_task and reply_handler_task. When they finish, they unreference the server, which leads to its memory being freed. However, the acceptor_learner_task keeps a "naked" reference to the server structure. Under the right ordering of operations, i.e. the sender_task and reply_handler_task terminating after the acceptor_learner_task acquires, but before it uses, the reference to the server structure, leads to the acceptor_learner_task accessing the server structure after it has been freed. Solution ======================================================================== Let the acceptor_learner_task also reference count the server structure so it is not freed while still in use. Reviewed-by: André Negrão <andre.negrao@oracle.com> Reviewed-by: Venkatesh Venugopal <venkatesh.venugopal@oracle.com> RB: 21209

…er [percona#9] [noclose] SEVERAL COMPILE WARNINGS FOR MYSQL CLUSTER ON WINDOWS WITH VS 2019 storage\ndb\plugin\ha_ndbcluster_binlog.cc(291,1): warning C4302: 'type cast': truncation from 'uchar *' to 'long' storage\ndb\plugin\ha_ndbcluster_binlog.cc(291,1): warning C4311: 'type cast': pointer truncation from 'uchar *' to 'long' Change-Id: I40e43518511138f8d606011f9a5e3fc06b670fd0

Use the m_table_info_instance and m_table_info direct instead of via local reference variable. Remove redundant use of struct keyword. Remove misplaced comment. Change-Id: I375f82897c8fe9767e19141af6ad4448213b7412

…ct_server() Change-Id: I1bcb50507deccf7ebd17678c333cf90b7822f9cf

# This is the 1st commit message: WL#15280: HEATWAVE SUPPORT FOR MDS HA Problem Statement ----------------- Currently customers cannot enable heatwave analytics service to their HA DBSystem or enable HA if they are using Heatwave enabled DBSystem. In this change, we attempt to remove this limitation and provide failover support of heatwave in an HA enabled DBSystem. High Level Overview ------------------- To support heatwave with HA, we extended the existing feature of auto- reloading of tables to heatwave on MySQL server restart (WL-14396). To provide seamless failover functionality to tables loaded to heatwave, each node in the HA cluster (group replication) must have the latest view of tables which are currently loaded to heatwave cluster attached to the primary, i.e., the secondary_load flag should be in-sync always. To achieve this, we made following changes - 1. replicate secondary load/unload DDL statements to all the active secondary nodes by writing the DDL into the binlog, and 2. Control how secondary load/unload is executed when heatwave cluster is not attached to node executing the command Implementation Details ---------------------- Current implementation depends on two key assumptions - 1. All MDS DBSystems will have RAPID plugin installed. 2. No non-MDS system will have the RAPID plugin installed. Based on these assumptions, we made certain changes w.r.t. how server handles execution of secondary load/unload statements. 1. If secondary load/unload command is executed from a mysql client session on a system without RAPID plugin installed (i.e., non-MDS), instead of an error, a warning message will be shown to the user, and the DDL is allowed to commit. 2. If secondary load/unload command is executed from a replication connection on an MDS system without heatwave cluster attached, instead of throwing an error, the DDL is allowed to commit. 3. If no error is thrown from secondary engine, then the DDL will update the secondary_load metadata and write a binlog entry. Writing to binlog implies that all the consumer of binlog now need to handle this DDL gracefully. This has an adverse effect on Point-in-time Recovery. If the PITR backup is taken from a DBSystem with heatwave, it may contain traces of secondary load/unload statements in its binlog. If such a backup is used to restore a new DBSystem, it will cause failure while trying to execute statements from its binlog because a) DBSystem will not heatwave cluster attached at this time, and b) Statements from binlog are executed from standard mysql client connection, thus making them indistinguishable from user executed command. Customers will be prevented (by control plane) from using PITR functionality on a heatwave enabled DBSystem until there is a solution for this. Testing ------- This commit changes the behavior of secondary load/unload statements, so it - adjusts existing tests' expectations, and - adds a new test validating new DDL behavior under different scenarios Change-Id: Ief7e9b3d4878748b832c366da02892917dc47d83 # This is the commit message percona#2: WL#15280: HEATWAVE SUPPORT FOR MDS HA (PITR SUPPORT) Problem ------- A PITR backup taken from a heatwave enabled system could have traces of secondary load or unload statements in binlog. When such a backup is used to restore another system, it can cause failure because of following two reasons: 1. Currently, even if the target system is heatwave enabled, heatwave cluster is attached only after PITR restore phase completes. 2. When entries from binlogs are applied, a standard mysql client connection is used. This makes it indistinguishable from other user session. Since secondary load (or unload) statements are meant to throw error when they are executed by user in the absence of a healthy heatwave cluster, PITR restore workflow will fail if binlogs from the backup have any secondary load (or unload) statements in them. Solution -------- To avoid PITR failure, we are introducing a new system variable rapid_enable_delayed_secondary_ops. It controls how load or unload commands are to be processed by rapid plugin. - When turned ON, the plugin silently skips the secondary engine operation (load/unload) and returns success to the caller. This allows secondary load (or unload) statements to be executed by the server in the absence of any heatwave cluster. - When turned OFF, it follows the existing behavior. - The default value is OFF. - The value can only be changed when rapid_bootstrap is IDLE or OFF. - This variable cannot be persisted. In PITR workflow, Control Plane would set the variable at the start of PITR restore and then reset it at the end of workflow. This allows the workflow to complete without failure even when heatwave cluster is not attached. Since metadata is always updated when secondary load/unload DDLs are executed, when heatwave cluster is attached at a later point in time, the respective tables get reloaded to heatwave automatically. Change-Id: I42e984910da23a0e416edb09d3949989159ef707 # This is the commit message percona#3: WL#15280: HEATWAVE SUPPORT FOR MDS HA (TEST CHANGES) This commit adds new functional tests for the MDS HA + HW integration. Change-Id: Ic818331a4ca04b16998155efd77ac95da08deaa1 # This is the commit message percona#4: WL#15280: HEATWAVE SUPPORT FOR MDS HA BUG#34776485: RESTRICT DEFAULT VALUE FOR rapid_enable_delayed_secondary_ops This commit does two things: 1. Add a basic test for newly introduced system variable rapid_enable_delayed_secondary_ops, which controls the behavior of alter table secondary load/unload ddl statements when rapid cluster is not available. 2. It also restricts the DEFAULT value setting for the system variable So, following is not allowed: SET GLOBAL rapid_enable_delayed_secondary_ops = default This variable is to be used in restricted scenarios and control plane only sets it to ON/OFF before and after PITR apply. Allowing set to default has no practical use. Change-Id: I85c84dfaa0f868dbfc7b1a88792a89ffd2e81da2 # This is the commit message percona#5: Bug#34726490: ADD DIAGNOSTICS FOR SECONDARY LOAD / UNLOAD DDL Problem: -------- If secondary load or unload DDL gets rolled back due to some error after it had loaded / unloaded the table in heatwave cluster, there is no undo of the secondary engine action. Only secondary_load flag update is reverted and binlog is not written. From User's perspective, the table is loaded and can be seen on performance_schema. There are also no error messages printed to notify that the ddl didn't commit. This creates a problem to debug any issue in this area. Solution: --------- The partial undo of secondary load/unload ddl will be handled in bug#34592922. In this commit, we add diagnostics to reveal if the ddl failed to commit, and from what stage. Change-Id: I46c04dd5dbc07fc17beb8aa2a8d0b15ddfa171af # This is the commit message percona#6: WL#15280: HEATWAVE SUPPORT FOR MDS HA (TEST FIX) Since ALTER TABLE SECONDARY LOAD / UNLOAD DDL statements now write to binlog, from Heatwave's perspective, SCN is bumped up. In this commit, we are adjusting expected SCN values in certain tests which does secondary load/unload and expects SCN to match. Change-Id: I9635b3cd588d01148d763d703c72cf50a0c0bb98 # This is the commit message percona#7: Adding MTR tests for ML in rapid group_replication suite Added MTR tests with Heatwave ML queries with in an HA setup. Change-Id: I386a3530b5bbe6aea551610b6e739ab1cf366439 # This is the commit message percona#8: WL#15280: HEATWAVE SUPPORT FOR MDS HA (MTR TEST ADJUSTMENT) In this commit we have adjusted the existing test to work with the new MTR test infrastructure which extends the functionalities to HA landscape. With this change, a lot of mannual settings have now become redundant and thus removed in this commit. Change-Id: Ie1f4fcfdf047bfe8638feaa9f54313d509cbad7e # This is the commit message percona#9: WL#15280: HEATWAVE SUPPORT FOR MDS HA (CLANG-TIDY FIX) Fix clang-tidy warnings found in previous change#16530, patch#20 Change-Id: I15d25df135694c2f6a3a9146feebe2b981637662 Change-Id: I3f3223a85bb52343a4619b0c2387856b09438265

If a data node starts up with RequireCertificate, it must have a valid TLS certificate. This adds three new tests: testMgmd -n NdbdWithoutCertificate testMgmd -n NdbdWithCertificate testMgmd -n NdbdWithExpiredCertificate Change-Id: I23cb96e14136c78cc295d586b044a99b13ec50bc

…ndb_mgmd When a node starts up, it might connect to a management server to obtain the cluster configuration. The component that does this is known as ConfigRetriever. This patch adds an instance of TlsKeyManager to ConfigRetriever for use in MGM TLS by both sorts of ndbd processes: the data node and the angel process. The data node (but not the angel) also has a TlsKeyManager in the global TransporterRegistry. In the data node, these two TlsKeyManagers are initialized with the same node type and TLS search path, so they will both use the same certificate. In a data node, ConfigRetriever's MGM connection is the one that will get turned in to a transporter. In the existing MTR test tls_off_certs, the behavior changes so that the data nodes now have TLS connections to the MGM node. An NDB management server might also be a client of some other NDB management server. This patch adds support for the --ndb-mgm-tls option to ndb_mgmd, for setting its MGM TLS requirement level when it acts as a client. This patch extends the test testMgmd -n NdbdWithCertificate to run with [MGM]RequireTls set to true. Change-Id: I9ab68bef61b80a18edbf1375366c81ed874d7026

Post push fix. Tests testMgmd -n NdbdWithoutCertificate and testMgmd -n NdbdWithExpiredCertificate wait 100ms for data node to start and stop due to missing respectively invalid certiticates. In some environments 100ms is not enough, extending wait time to 1s. Change-Id: I964b4bd4b0b2a8cd89998a26034fffc5ef32e99b

…ces as function arguments WL#13520 "Transform correlated scalar subqueries" has a requirement (percona#9), which states that the equality expression operands must be simple column references, e.g. in the below transformation example from WL#13520, t2.a and t1.a — the operands of the WHERE clause equality predicate — are simple references. This WL lifts this restriction and allows the column references to be arguments of functions. SELECT * FROM t1 WHERE (SELECT a FROM t2 WHERE t2.a=t1.a) > 0; -> (with non-strict grouping and a built-in assert to signal any cardinality error): SELECT t1.a AS a FROM t1 JOIN (SELECT t2.a AS a, COUNT(0) AS cnt FROM t2 WHERE (t2.a > 0) GROUP BY t2.a) derived WHERE t1.a = derived.a AND reject_if(derived.cnt > 1) The outer reference, t1.a, can already be embedded as a function argument, but the non-outer (inner for short) column that we place in the GROUP BY above, t2.a, cannot. With this WL, we will also be able to transform queries like SELECT * FROM t1 WHERE (SELECT func(t2.a) FROM t2 WHERE func(t2.a) = t1.a ) > 0; -> SELECT t1.* FROM t1 JOIN ( SELECT func(t2.a) AS a, COUNT(0) AS cnt FROM t2 WHERE (t2.a > 0) GROUP BY func(t2.a)) derived WHERE derived.a = t1.a AND reject_if(derived.cnt > 1) This transformation is needed by Heatwave/RAPID, which doesn't support subqueries in the WHERE clause. Limitations: due to a phase problem in the resolver, functional dependencies are not always handled correctly: if the transformed subquery has explicit grouping, functional dependency analysis may be too pessimistic and give an error when the transform is used (but not without). See worklog for details. A work-around is to wrap the offending column in ANY_VALUE. Change-Id: I06fc0bbf8931f87ca964dc784329f4b2e24e048f

This reverts commit 0a661dc0d6dd8b498b0ed9f59207b9814d380fbe. Some mtr tests have leaks originating from percona#7 0x7f579a2a28cc (/lib/x86_64-linux-gnu/libprotobuf-lite.so.23+0x4d8cc) (BuildId: 1af51cdba58393c136bc541bbfc26ee7568c8c78) percona#8 0x7f579a2a44cc in google::protobuf::internal::InitSCCImpl(google::protobuf::internal::SCCInfoBase*) (/lib/x86_64-linux-gnu/libprotobuf-lite.so.23+0x4f4cc) (BuildId: 1af51cdba58393c136bc541bbfc26ee7568c8c78) percona#9 0x56352824ad0d in google::protobuf::internal::InitSCC(google::protobuf::internal::SCCInfoBase*) /usr/include/google/protobuf/generated_message_util.h:240:5 Change-Id: I52ea02cee282a9521a2b153b4d883314ed15eb5c

akopytov and others added 5 commits February 17, 2015 13:43

Merge pull request percona#5 from akopytov/2.2

86ab9b2

Removed files erroneously resurrected on git import.

Version bump

9033b0b

Merge pull request percona#7 from tplavcic/release-2.2.9

bcfbdf4

Release 2.2.9

Updated PS versions in bootstrap.sh to latest 5.5 and 5.6.

18b4159

Merge branch '2.2' into 2.3

00e61a0

akopytov added a commit that referenced this pull request Feb 20, 2015

Merge pull request #9 from akopytov/2.3

8459631

2.3

akopytov merged commit 8459631 into percona:2.3 Feb 20, 2015

dutow pushed a commit to dutow/percona-xtrabackup that referenced this pull request Aug 13, 2018

Merge pull request percona#9 from dutow/fix-incorrect-dirs

74d2cab

Fixing backup dirs inside data dirs in testcases

altmannmarcelo pushed a commit to altmannmarcelo/percona-xtrabackup that referenced this pull request May 4, 2023

WL#15130 Socket-level TLS patch percona#9: TransporterRegistry::conne…

02b1449

…ct_server() Change-Id: I1bcb50507deccf7ebd17678c333cf90b7822f9cf

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

2.3 #9

2.3 #9

akopytov commented Feb 20, 2015

2.3 #9

2.3 #9

Conversation

akopytov commented Feb 20, 2015