-
Notifications
You must be signed in to change notification settings - Fork 120
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
TRUE & FALSE not defined #1
Comments
+1 |
1 similar comment
+1 |
We have fixed the build error, reopen it if any problem again. |
xiewajueji
pushed a commit
that referenced
this issue
May 5, 2024
A heap-buffer-overflow in libmyqlxclient when - auth-method is MYSQL41 - the "server" sends a nonce that is shortert than 20 bytes. ==2466857==ERROR: AddressSanitizer: heap-buffer-overflow on address #0 0x4a7b76 in memcpy (routertest_component_routing_splicer+0x4a7b76) #1 0x7fd3a1d89052 in SHA1_Update (/libcrypto.so.1.1+0x1c2052) #2 0x63409c in compute_mysql41_hash_multi(unsigned char*, char const*, unsigned int, char const*, unsigned int) ... RB: 25305 Reviewed-by: Lukasz Kotula <lukasz.kotula@oracle.com>
xiewajueji
pushed a commit
that referenced
this issue
May 5, 2024
…TH VS 2019 [#1] [noclose] storage\ndb\include\util\Bitmask.hpp(388,23): warning C4146: unary minus operator applied to unsigned type, result still unsigned Change-Id: I657966b790b96356d987767302cfceb446e29a98
xiewajueji
pushed a commit
that referenced
this issue
May 5, 2024
Add --cluster-config-suffix to ndb_mgmd and ndb_config. Change-Id: I68592847019e855ef4ae837a1a568ac517d0aa75
xiewajueji
pushed a commit
that referenced
this issue
May 5, 2024
…lize subqueries] For IN subqueries that have not been converted to semijoin, consider materializing them. To do this, we need to add two interrelated steps: 1. Every subquery that has gone through IN-to-EXISTS needs to be planned twice; once as is (for direct execution), and once without the added IN-to-EXISTS conditions (for materialization, since the extra conditions are dependent on the outer query block). There is some overlap between these two plans, in particular if the subquery involves many tables. At one point, we had a prototype that did both at the same time, tagging access paths with whether they included such conditions or not (and not combining incompatible alternatives), but it ended up becoming very intrusive, so instead, we simply plan from scratch twice. Most subqueries are cheap to plan anyway. 2. Whenever we see a filter with a subquery, we propose two alternatives, one as-is and one where the subquery is materialized. (These are the two plans generated by #1.) The latter will have high init_once_cost but usually much lower cost, so the planner will make a cost-based decision largely depending on the number of joined rows. This is fairly similar to what the old optimizer does, except that the old one seems to ignore some of the costs (it doesn't plan both alternatives before making the decisions; it only replans if it actually chooses materialization). Change-Id: I6947419e1f9d4ec0f03f7ba214f46daf2f690c4c
xiewajueji
pushed a commit
that referenced
this issue
May 5, 2024
…ING TABLESPACES The occurrence of this message is a minor issue fixed by change #1 below. But during testing, I found that if mysqld is restarted while remote and local tablespaces are discarded, especially if the tablespaces to be imported are already in place at startup, then many things can go wrong. There were various asserts that occurred depending on timing. During all the testing and debugging, the following changes were made. 1. Prevent the stats thread from complaining about a missing tablespace. See dict_stats_update(). 2. Prevent a discarded tablespace from being opened at startup, even if the table to be imported is already in place. See Validate_files::check(). 3. dd_tablespace_get_state_enum() was refactored to separate the normal way to do it in v8.0, which is to use "state" key in dd::tablespaces::se_private_date, from the non-standard way which is to check undo::spaces or look for the old key value pair of "discarded=true". This allowed the new call to this routine by the change in fix #2 above. 4. Change thd_tablespace_op() in sql/sql_thd_api.cc such that instead of returning 1 if the DDL requires an implicit tablespace, it returns the DDL operation flag. This can still be interpreted as a boolean, but it can also be used to determine if the op is an IMPORT or a DISCARD. 5. With that change, the annoying message that a space is discarded can be avoided during an import when it needs to be discarded. 6. Several test cases were corrected now that the useless "is discarded" warning is no longer being written. 7. Two places where dd_tablespace_set_state() was called to set the state to either "discard" or "normal" were consolidated to a new version of dd_tablespace_set_state(thd, dd_space_id, space_name, dd_state). 8. This new version of dd_tablespace_set_state() was used in dd_commit_inplace_alter_table() to make sure that in all three places the dd is changed to identify a discarded tablesapace, it is identified in dd:Tablespace::se_private_data as well as dd:Table::se_private_data or dd::Partition::se_private_data. The reason it is necessary to record this in dd::Tablespace is that during startup, boot_tablespaces() and Validate::files::check() are only traversing dd::Tablespace. And that is where fix #2 is done! 9. One of the asserts that occurred was during IMPORT TABLESPACE after a restart that found a discarded 5.7 tablespace in the v8.0 discarded location. This assert occurred in Fil_shard::get_file_size() just after ER_IB_MSG_272. The 5.7 file did not have the SDI flag, but the v8.0 space that was discarded did have that flag. So the flags did not match. That crash was fixed by setting the fil_space_t::flags to what it is in the tablespace header page. A descriptive comment was added. 10. There was a section in fil_ibd_open() that checked `if (space != nullptr) {` and if true, it would close and free stuff then immediately crash. I think I remember many years ago adding that assert because I did not think it actually occurred. Well it did occur during my testing before I added fix #2 above. This made fil_ibd_open() assume that the file was NOT already open. So fil_ibd_open() is now changed to allow for that possibility by adding `if (space != nullptr) {return DB_SUCCESS}` further down. Since fil_ibd_open() can be called with a `validate` boolean, the routine now attempts to do all the validation whether or not the tablespace is already open. The following are non-functional changes; - Many code documentation lines were added or improved. - dict_sys_t::s_space_id renamed to dict_sys_t::s_dict_space_id in order to clarify better which space_id it referred to. - For the same reason, change s_dd_space_id to s_dd_dict_space_id. - Replaced `table->flags2 & DICT_TF2_DISCARDED` with `dict_table_is_discarded(table)` in dict0load.cc - A redundant call to ibuf_delete_for_discarded_space(space_id) was deleted from fil_discard_tablespace() because it is also called higher up in the call stack in row_import_for_mysql(). - Deleted the declaration to `row_import_update_discarded_flag()` since the definition no longer exists. It was deleted when we switched from `discarded=true` to 'state=discarded' in dd::Tablespace::se_private_data early in v8.0 developement. Approved by Mateusz in RB#26077
xiewajueji
pushed a commit
that referenced
this issue
May 5, 2024
Memory leaks detected when running testMgm with ASAN build. bld_asan$> mtr test_mgm Direct leak of 8 byte(s) in 1 object(s) allocated from: #0 0x3004ed in malloc (trunk/bld_asan/runtime_output_directory/testMgm+0x3004ed) #1 0x7f794d6b0b46 in ndb_mgm_create_logevent_handle trunk/bld_asan/../storage/ndb/src/mgmapi/ndb_logevent.cpp:85:24 #2 0x335b4b in runTestMgmApiReadErrorRestart(NDBT_Context*, NDBT_Step*) trunk/bld_asan/../storage/ndb/test/ndbapi/testMgm.cpp:652:32 Add support for using unique_ptr for all functions in mgmapi that return pointer to something that need to be released. Move existing functionality for ndb_mgm_configuration to same new file. Use ndb_mgm namespace for new functions and remove implementation details from both the new and old functionality Use new functionality to properly release allocated memory. Change-Id: Id455234077c4ed6756e93bf7f40a1e93179af1a0
xiewajueji
pushed a commit
that referenced
this issue
May 5, 2024
Remove already implemented TODO for WL#9019 Change-Id: Id5ed723cad060029dc80fd0905485bcf5dc61c2a
xiewajueji
pushed a commit
that referenced
this issue
May 5, 2024
…DISPLAYED AS COMPILED WHEN IT IS PERSISTED RB#26664 Removed the break; statement as it turns our multiple-prefixes are valid
xiewajueji
pushed a commit
that referenced
this issue
May 5, 2024
Patch #1: Support view references in generated column substitution. REF_ITEM was not considered in Item::can_be_substituted_for_gc() and get_gc_for_expr(), so optimizer failed to use multi-valued index from a view. The patch is also applicable to view references in general, thus not restricted to multi-valued indexes. This is a contribution by Yubao Liu. Change-Id: Iaa82a1245c80641fab0ed3a2d0f459e4e9bc26c1
xiewajueji
pushed a commit
that referenced
this issue
May 5, 2024
This error happens for queries such as: SELECT ( SELECT 1 FROM t1 ) AS a, ( SELECT a FROM ( SELECT x FROM t1 ORDER BY a ) AS d1 ); Query_block::prepare() for query block #4 (corresponding to the 4th SELECT in the query above) calls setup_order() which again calls find_order_in_list(). That function replaces an Item_ident for 'a' in Query_block.order_list with an Item_ref pointing to query block #2. Then Query_block::merge_derived() merges query block #4 into query block #3. The Item_ref mentioned above is then moved to the order_list of query block #3. In the next step, find_order_in_list() is called for query block #3. At this point, 'a' in the select list has been resolved to another Item_ref, also pointing to query block #2. find_order_in_list() detects that the Item_ref in the order_list is equivalent to the Item_ref in the select list, and therefore decides to replace the former with the latter. Then find_order_in_list() calls Item::clean_up_after_removal() recursively (via Item::walk()) for the order_list Item_ref (since that is no longer needed). When calling clean_up_after_removal(), no Cleanup_after_removal_context object is passed. This is the actual error, as there should be a context pointing to query block #3 that ensures that clean_up_after_removal() only purge Item_subselect.unit if both of the following conditions hold: 1) The Item_subselect should not be in any of the Item trees in the select list of query block #3. 2) Item_subselect.unit should be a descendant of query block #3. These conditions ensure that we only purge Item_subselect.unit if we are sure that it is not needed elsewhere. But without the right context, query block #2 gets purged even if it is used in the select lists of query blocks #1 and #3. The fix is to pass a context (for query block #3) to clean_up_after_removal(). Both of the above conditions then become false, and Item_subselect.unit is not purged. As an additional shortcut, find_order_in_list() will not call clean_up_after_removal() if real_item() of the order item and the select list item are identical. In addition, this commit changes clean_up_after_removal() so that it requires the context to be non-null, to prevent similar errors. It also simplifies Item_sum::clean_up_after_removal() by removing window functions unconditionally (and adds a corresponding test case). Change-Id: I449be15d369dba97b23900d1a9742e9f6bad4355
xiewajueji
pushed a commit
that referenced
this issue
May 5, 2024
…nt [#1] Problem ======= When the coordinator receives a stale schema event. It crashes due to assert failure. Description =========== After bug#32593352 fix, client/user thread can now detect schema distribution timeout by itself and can free the schema object. So, if a stale schema event reaches the coordinator after the client/user thread have freed the schema object, then the coordinator will try to get the schema object and will hit the assert failure. prior to bug#32593352, the schema distribution timeout can be detected only by the coordinator. So, it is assumed that the schema object should be always valid inside coordinator. As, there exists a valid scenario where schema object can be invalid the assert check now is not useful and can be removed. Fix === Fixed by removing the assert check. Change-Id: I0482ccc940505e83d66cbf2258528fbac6951599
xiewajueji
pushed a commit
that referenced
this issue
May 5, 2024
…NSHIP WITH THE BUFFER SIZE Bug #33501541: Unmanageable Sort Buffer Behavior in 8.0.20+ Implement direct disk-to-disk copies of large packed addons during the filesort merge phase; if a single row is so large that its addons do not fit into its slice of the sort buffer during merging (even after emptying that slice of all other rows), but the sort key _does_ fit, simply sort the truncated row as usual, and then copy the rest of the addon incrementally from the input to the output, 4 kB at a time, when the row is to be written to the merge output. This is possible because the addon itself doesn't need to be in RAM for the row to be compared against other rows; only the sort key must. This greatly relaxes the sort buffer requirements for successful merging, especially when it comes to JSON rows or small blobs (which are typically used as packed addons, not sort keys). The rules used to be: 1. During initial chunk generation: The sort buffer must be at least as large as the largest row to be sorted. 2. During merging: Merging is guaranteed to pass if the sort buffer is at least 15 times as large as the largest row (sort key + addons), but one may be lucky and pass with only the demands from #1. Now, for sorts implemented using packed addons (which is the common case for small blobs and JSON), the new rules are: 1. Unchanged from #1 above. 2. During merging: Merging is guaranteed to pass if the sort buffer is at least 15 times are large as the largest _sort key_ (plus 4-byte length marker), but one may be lucky and pass with only the demands from #1. In practice, this means that filesort merging will almost never fail due to insufficient buffer space anymore; the query will either fail because a single row is too large in the sort step, or it will pass nearly all of the time. However, do note that while such merges will work, they will not always be very performant, as having lots of 1-row merge chunks will mean many merge passes and little work being done during the initial in-memory sort. Thus, the main use of this functionality is to be able to do sorts where there are a few rows with large JSON values or similar, but where most fit comfortably into the buffer. Also note that since requirement #1 is unchanged, one still cannot sort e.g. 500 kB JSON values using the default 256 kB sort buffer. Older recommendations to keep sort buffers small at nearly any cost are no longer valid, and have not been for a while. Sort buffers should be sized to as much RAM as one can afford without interfering with other tasks (such as the buffer pool, join buffers, or other concurrent sorts), and small sorts are not affected by the maximum sort buffer size being set to a larger value, as the sort buffer is incrementally allocated. Change-Id: I85745cd513402a42ed5fc4f5b7ddcf13c5793100
xiewajueji
pushed a commit
that referenced
this issue
May 5, 2024
…NSHIP WITH THE BUFFER SIZE Bug #33501541: Unmanageable Sort Buffer Behavior in 8.0.20+ Implement direct disk-to-disk copies of large packed addons during the filesort merge phase; if a single row is so large that its addons do not fit into its slice of the sort buffer during merging (even after emptying that slice of all other rows), but the sort key _does_ fit, simply sort the truncated row as usual, and then copy the rest of the addon incrementally from the input to the output, 4 kB at a time, when the row is to be written to the merge output. This is possible because the addon itself doesn't need to be in RAM for the row to be compared against other rows; only the sort key must. This greatly relaxes the sort buffer requirements for successful merging, especially when it comes to JSON rows or small blobs (which are typically used as packed addons, not sort keys). The rules used to be: 1. During initial chunk generation: The sort buffer must be at least as large as the largest row to be sorted. 2. During merging: Merging is guaranteed to pass if the sort buffer is at least 15 times as large as the largest row (sort key + addons), but one may be lucky and pass with only the demands from #1. Now, for sorts implemented using packed addons (which is the common case for small blobs and JSON), the new rules are: 1. Unchanged from #1 above. 2. During merging: Merging is guaranteed to pass if the sort buffer is at least 15 times are large as the largest _sort key_ (plus 4-byte length marker), but one may be lucky and pass with only the demands from #1. In practice, this means that filesort merging will almost never fail due to insufficient buffer space anymore; the query will either fail because a single row is too large in the sort step, or it will pass nearly all of the time. However, do note that while such merges will work, they will not always be very performant, as having lots of 1-row merge chunks will mean many merge passes and little work being done during the initial in-memory sort. Thus, the main use of this functionality is to be able to do sorts where there are a few rows with large JSON values or similar, but where most fit comfortably into the buffer. Also note that since requirement #1 is unchanged, one still cannot sort e.g. 500 kB JSON values using the default 256 kB sort buffer. Older recommendations to keep sort buffers small at nearly any cost are no longer valid, and have not been for a while. Sort buffers should be sized to as much RAM as one can afford without interfering with other tasks (such as the buffer pool, join buffers, or other concurrent sorts), and small sorts are not affected by the maximum sort buffer size being set to a larger value, as the sort buffer is incrementally allocated. Change-Id: I85745cd513402a42ed5fc4f5b7ddcf13c5793100
xiewajueji
pushed a commit
that referenced
this issue
May 5, 2024
…NSHIP WITH THE BUFFER SIZE Bug #33501541: Unmanageable Sort Buffer Behavior in 8.0.20+ Implement direct disk-to-disk copies of large packed addons during the filesort merge phase; if a single row is so large that its addons do not fit into its slice of the sort buffer during merging (even after emptying that slice of all other rows), but the sort key _does_ fit, simply sort the truncated row as usual, and then copy the rest of the addon incrementally from the input to the output, 4 kB at a time, when the row is to be written to the merge output. This is possible because the addon itself doesn't need to be in RAM for the row to be compared against other rows; only the sort key must. This greatly relaxes the sort buffer requirements for successful merging, especially when it comes to JSON rows or small blobs (which are typically used as packed addons, not sort keys). The rules used to be: 1. During initial chunk generation: The sort buffer must be at least as large as the largest row to be sorted. 2. During merging: Merging is guaranteed to pass if the sort buffer is at least 15 times as large as the largest row (sort key + addons), but one may be lucky and pass with only the demands from #1. Now, for sorts implemented using packed addons (which is the common case for small blobs and JSON), the new rules are: 1. Unchanged from #1 above. 2. During merging: Merging is guaranteed to pass if the sort buffer is at least 15 times are large as the largest _sort key_ (plus 4-byte length marker), but one may be lucky and pass with only the demands from #1. In practice, this means that filesort merging will almost never fail due to insufficient buffer space anymore; the query will either fail because a single row is too large in the sort step, or it will pass nearly all of the time. However, do note that while such merges will work, they will not always be very performant, as having lots of 1-row merge chunks will mean many merge passes and little work being done during the initial in-memory sort. Thus, the main use of this functionality is to be able to do sorts where there are a few rows with large JSON values or similar, but where most fit comfortably into the buffer. Also note that since requirement #1 is unchanged, one still cannot sort e.g. 500 kB JSON values using the default 256 kB sort buffer. Older recommendations to keep sort buffers small at nearly any cost are no longer valid, and have not been for a while. Sort buffers should be sized to as much RAM as one can afford without interfering with other tasks (such as the buffer pool, join buffers, or other concurrent sorts), and small sorts are not affected by the maximum sort buffer size being set to a larger value, as the sort buffer is incrementally allocated. Change-Id: I85745cd513402a42ed5fc4f5b7ddcf13c5793100
xiewajueji
pushed a commit
that referenced
this issue
May 5, 2024
This reduces the number of issues of type "Single - argument constructor may inadvertently be used as a type conversion constructor" as reported by Flint++ tool. Some of the reported issues were not addressed: - lines explicitly marked with NOLINT (like plugin/x/src/prepare_param_handler.h) - false positives (tool reports alignas() calls as constructors) - issues where objects were intended to be used with type conversion - constructors with "const std::string &" param, to accept literal strings as well Change-Id: I7eb6fabea7137fc7e81143f06ec7636b22f6ea97
xiewajueji
pushed a commit
that referenced
this issue
May 5, 2024
Includes a partial implementation of span from C++20. Change-Id: Ibae9a4aeed95135f4ef35a7ce7b095e6930c1d66
xiewajueji
pushed a commit
that referenced
this issue
May 5, 2024
Post push fix. Remove include of unused header files "ndb_global.h" and <cassert>. Use std::size_t instead of size_t. Change-Id: I2c718d0889965ce5967d575172da8df4aa55b1d7
xiewajueji
pushed a commit
that referenced
this issue
May 5, 2024
Patch #1 caused several problems in mysql-trunk related to ndbinfo initialization and upgrade, including the failure of the test ndb_76_inplace_upgrade and the failure of all NDB MTR tests in Pushbuild on Windows. This patch fixes these issues, including fixes for bug#33726826 and bug#33730799. In ndbinfo, revert the removal of ndb$blocks and ndb$index_stats and the change of blocks and index_stats from views to tables. Improve the ndbinfo schema upgrade & initialization logic to better handle such a change in the future. This logic now runs in two passes: first it drops the known tables and views from current and previous versions, then it creates the tables and views for the current version. Add a new class method NdbDictionary::printColumnTypeDescription(). This is needed for the ndbinfo.columns table in patch #2 but was missing from patch #1. Add boilerplate index lookup initialization code that was also missing. Fix ndbinfo prefix determination on Windows. Change-Id: I422856bcad4baf5ae9b14c1e3a1f2871bd6c5f59
xiewajueji
pushed a commit
that referenced
this issue
May 5, 2024
* PROBLEM The test "ndb.ndb_bug17624736" was constantly failing in [daily|weekly]-8.0-cluster branches in PB2, whether on `ndb-ps` or `ndb-default-big` profile test runs. The high-level reason for the failure was the installation of a duplicate entry in the Data Dictionary in respect to the `engine`-`se_private_id` pair, even when the previous table definition should have been dropped. * LOW-LEVEL EXPLANATION NDB reuses the least available ID for the dictionary table ID. The ID is then used by the NDB plugin to install as SE private ID field of the MySQL Data Dictionary table definition. If a problem occurs during the synchronization of NDB table definitions in the Data Dictionary (i.e., a previous definition was not successfully removed), then an attempt to install a table using an already installed SE private ID can occur. If that ID was inadvertedly cached as `missing`, then the function `acquire_uncached_table_by_se_private_id` will return fast without retrieving the table definition. Therefore, the old table definition on that ID cannot be retrieved ever for that Data Dictionary client instance, the new one won't be installed, and errors will be raised. * SOLUTION For NDB plugin to query a table definition, using the SE private ID (without causing a missing entry to be cached forever for that client instance), this patch adds a flag argument to the function to allow the caller to request it to skip the fast cache. Change-Id: I45eef594ee544000fe6b30b86977e5e91155dc80
xiewajueji
pushed a commit
that referenced
this issue
May 5, 2024
-- Patch #1: Persist secondary load information -- Problem: We need a way of knowing which tables were loaded to HeatWave after MySQL restarts due to a crash or a planned shutdown. Solution: Add a new "secondary_load" flag to the `options` column of mysql.tables. This flag is toggled after a successful secondary load or unload. The information about this flag is also reflected in INFORMATION_SCHEMA.TABLES.CREATE_OPTIONS. -- Patch #2 -- The second patch in this worklog triggers the table reload from InnoDB after MySQL restart. The recovery framework recognizes that the system restarted by checking whether tables are present in the Global State. If there are no tables present, the framework will access the Data Dictionary and find which tables were loaded before the restart. This patch introduces the "Data Dictionary Worker" - a MySQL service recovery worker whose task is to query the INFORMATION_SCHEMA.TABLES table from a separate thread and find all tables whose secondary_load flag is set to 1. All tables that were found in the Data Dictionary will be appended to the list of tables that have to be reloaded by the framework from InnoDB. If an error occurs during restart recovery we will not mark the recovery as failed. This is done because the types of failures that can occur when the tables are reloaded after a restart are less critical compared to previously existing recovery situations. Additionally, this code will soon have to be adapted for the next worklog in this area so we are proceeding with the simplest solution that makes sense. A Global Context variable m_globalStateEmpty is added which indicates whether the Global State should be recovered from an external source. -- Patch #3 -- This patch adds the "rapid_reload_on_restart" system variable. This variable is used to control whether tables should be reloaded after a restart of mysqld or the HeatWave plugin. This variable is persistable (i.e., SET PERSIST RAPID_RELOAD_ON_RESTART = TRUE/FALSE). The default value of this variable is set to false. The variable can be modified in OFF, IDLE, and SUSPENDED states. -- Patch #4 -- This patch refactors the recovery code by removing all recovery-related code from ha_rpd.cc and moving it to separate files: - ha_rpd_session_factory.h/cc: These files contain the MySQLAdminSessionFactory class, which is used to create admin sessions in separate threads that can be used to issue SQL queries. - ha_rpd_recovery.h/cc: These files contain the MySQLServiceRecoveryWorker, MySQLServiceRecoveryJob and ObjectStoreRecoveryJob classes which were previously defined in ha_rpd.cc. This file also contains a function that creates the RecoveryWorkerFactory object. This object is passed to the constructor of the Recovery Framework and is used to communicate with the other section of the code located in rpdrecoveryfwk.h/cc. This patch also renames rpdrecvryfwk to rpdrecoveryfwk for better readability. The include relationship between the files is shown on the following diagram: rpdrecoveryfwk.h◄──────────────rpdrecoveryfwk.cc ▲ ▲ │ │ │ │ │ └──────────────────────────┐ │ │ ha_rpd_recovery.h◄─────────────ha_rpd_recovery.cc──┐ ▲ │ │ │ │ │ │ │ │ │ ▼ │ ha_rpd.cc───────────────────────►ha_rpd.h │ ▲ │ │ │ ┌───────────────────────────────┘ │ │ ▼ ha_rpd_session_factory.cc──────►ha_rpd_session_factory.h Other changes: - In agreement with Control Plane, the external Global State is now invalidated during recovery framework startup if: 1) Recovery framework recognizes that it should load the Global State from an external source AND, 2) rapid_reload_on_restart is set to OFF. - Addressed review comments for Patch #3, rapid_reload_on_restart is now also settable while plugin is ON. - Provide a single entry point for processing external Global State before starting the recovery framework loop. - Change when the Data Dictionary is read. Now we will no longer wait for the HeatWave nodes to connect before querying the Data Dictionary. We will query it when the recovery framework starts, before accepting any actions in the recovery loop. - Change the reload flow by inserting fake global state entries for tables that need to be reloaded instead of manually adding them to a list of tables scheduled for reload. This method will be used for the next phase where we will recover from Object Storage so both recovery methods will now follow the same flow. - Update secondary_load_dd_flag added in Patch #1. - Increase timeout in wait_for_server_bootup to 300s to account for long MySQL version upgrades. - Add reload_on_restart and reload_on_restart_dbg tests to the rapid suite. - Add PLUGIN_VAR_PERSIST_AS_READ_ONLY flag to "rapid_net_orma_port" and "rapid_reload_on_restart" definitions, enabling their initialization from persisted values along with "rapid_bootstrap" when it is persisted as ON. - Fix numerous clang-tidy warnings in recovery code. - Prevent suspended_basic and secondary_load_dd_flag tests to run on ASAN builds due to an existing issue when reinstalling the RAPID plugin. -- Bug#33752387 -- Problem: A shutdown of MySQL causes a crash in queries fired by DD worker. Solution: Prevent MySQL from killing DD worker's queries by instantiating a DD_kill_immunizer before the queries are fired. -- Patch #5 -- Problem: A table can be loaded before the DD Worker queries the Data Dictionary. This means that table will be wrongly processed as part of the external global state. Solution: If the table is present in the current in-memory global state we will not consider it as part of the external global state and we will not process it by the recovery framework. -- Bug#34197659 -- Problem: If a table reload after restart causes OOM the cluster will go into RECOVERYFAILED state. Solution: Recognize when the tables are being reloaded after restart and do not move the cluster into RECOVERYFAILED. In that case only the current reload will fail and the reload for other tables will be attempted. Change-Id: Ic0c2a763bc338ea1ae6a7121ff3d55b456271bf0
xiewajueji
pushed a commit
that referenced
this issue
May 5, 2024
Enh#34350907 - [Nvidia] Allow DDLs when tables are loaded to HeatWave Bug#34433145 - WL#15129: mysqld crash Assertion `column_count == static_cast<int64_t>(cp_table- Bug#34446287 - WL#15129: mysqld crash at rapid::data::RapidNetChunkCtx::consolidateEncodingsDic Bug#34520634 - MYSQLD CRASH : Sql_cmd_secondary_load_unload::mysql_secondary_load_or_unload Bug#34520630 - Failed Condition: "table_id != InvalidTableId" Currently, DDL statements such as ALTER TABLE*, RENAME TABLE, and TRUNCATE TABLE are not allowed if a table has a secondary engine defined. The statements fail with the following error: "DDLs on a table with a secondary engine defined are not allowed." This worklog lifts this restriction for tables whose secondary engine is RAPID. A secondary engine hook is called in the beginning (pre-hook) and in the end (post-hook) of a DDL statement execution. If the DDL statement succeeds, the post-hook will direct the recovery framework to reload the table in order to reflect that change in HeatWave. Currently all DDL statements that were previously disallowed will trigger a reload. This can be improved in the future by checking whether the DDL operation has an impact on HeatWave or not. However detecting all edge-cases in this behavior is not straightforward so this improvement has been left as a future improvement. Additionally, if a DDL modifies the table schema in a way that makes it incompatible with HeatWave (e.g., dropping a primary key column) the reload will fail silently. There is no easy way to recognize whether the table schema will become incompatible with HeatWave in a pre-hook. List of changes: 1) [MySQL] Add new HTON_SECONDARY_ENGINE_SUPPORTS_DDL flag to indicate whether a secondary engine supports DDLs. 2) [MySQL] Add RAII hooks for RENAME TABLE and TRUNCATE TABLE, modeled on the ALTER TABLE hook. 3) Define HeatWave hooks for ALTER TABLE, RENAME TABLE, and TRUNCATE TABLE statements. 4) If a table reload is necessary, trigger it by marking the table as stale (WL#14914). 4) Move all change propagation & DDL hooks to ha_rpd_hooks.cc. 5) Adjust existing tests to support table reload upon DDL execution. 6) Extract code related to RapidOpSyncCtx in ha_rpd_sync_ctx.cc, and the PluginState enum to ha_rpd_fsm.h. * Note: ALTER TABLE statements related to secondary engine setting and loading were allowed before: - ALTER TABLE <TABLE> SECONDARY_UNLOAD, - ALTER TABLE SECONDARY_ENGINE = NULL. -- Bug#34433145 -- -- Bug#34446287 -- --Problem #1-- Crashes in Change Propagation when the CP thread tries to apply DMLs of tables with new schema to the not-yet-reloaded table in HeatWave. --Solution #1-- Remove table from Change Propagation before marking it as stale and revert the original change from rpd_binlog_parser.cc where we were checking if the table was stale before continuing with binlog parsing. The original change is no longer necessary since the table is removed from CP before being marked as stale. --Problem #2-- In case of a failed reload, tables are not removed from Global State. --Solution #2-- Keep track of whether the table was reloaded because it was marked as STALE. In that case we do not want the Recovery Framework to retry the reload and therefore we can remove the table from the Global State. -- Bug#34520634 -- Problem: Allowing the change of primary engine for tables with a defined secondary engine hits an assertion in mysql_secondary_load_or_unload(). Example: CREATE TABLE t1 (col1 INT PRIMARY KEY) SECONDARY_ENGINE = RAPID; ALTER TABLE t1 ENGINE = BLACKHOLE; ALTER TABLE t1 SECONDARY_LOAD; <- assertion hit here Solution: Disallow changing the primary engine for tables with a defined secondary engine. -- Bug#34520630 -- Problem: A debug assert is being hit in rapid_gs_is_table_reloading_from_stale because the table was dropped in the meantime. Solution: Instead of asserting, just return false if table is not present in the Global State. This patch also changes rapid_gs_is_table_reloading_from_stale to a more specific check (inlined the logic in load_table()). This check now also covers the case when a table was dropped/unloaded before the Recovery Framework marked it as INRECOVERY. In that case, if the reload fails we should not have an entry for that table in the Global State. The patch also adjusts dict_types MTR test, where we no longer expect for tables to be in UNAVAIL state after a failed reload. Additionally, recovery2_ddls.test is adjusted to not try to offload queries running on Performance Schema. Change-Id: I6ee390b1f418120925f5359d5e9365f0a6a415ee
xiewajueji
pushed a commit
that referenced
this issue
May 5, 2024
… Signal (get_store_key at sql/sql_select.cc:2383) These are two related but distinct problems manifested in the shrinkage of key definitions for derived tables or common table expressions, implemented in JOIN::finalize_derived_keys(). The problem in Bug#34572040 is that we have two references to one CTE, each with a valid key definition. The function will first loop over the first reference (cte_a) and move its used key from position 0 to position 1. Next, it will attempt to move the key for the second reference (cte_b) from position 4 to position 2. However, for each iteration, the function will calculate used key information. On the first iteration, the values are correct, but since key value #1 has been moved into position #0, the old information is invalid and provides wrong information. The problem is thus that for subsequent iterations we read data that has been invalidated by earlier key moves. The best solution to the problem is to move the keys for all references to the CTE in one operation. This way, we can calculate used keys information safely, before any move operation has been performed. The problem in Bug#34634469 is also related to having more than one reference to a CTE, but in this case the first reference (ref_3) has a key in position 5 which is moved to position 0, and the second reference (ref_4) has a key in position 3 that is moved to position 1. However, the key parts of the first key will overlap with the key parts of the second key after the first move, thus invalidating the key structure during the copy. The actual problem is that we move a higher-numbered key (5) before a lower-numbered key (3), which in this case makes it impossible to find an empty space for the moved key. The solution to this problem is to ensure that keys are moved in increasing key order. The patch changes the algorithm as follows: - When identifying a derived table/common table expression, ensure to move all its keys in one operation (at least those references from the same query block). - First, collect information about all key uses: hash key, unique index keys and actual key references. For the key references, also populate a mapping array that enumerates table references with key references in order of increasing key number. Also clear used key information for references that do not use keys. - For each table reference with a key reference in increasing key order, move the used key into the lowest available position. This will ensure that used entries are never overwritten. - When all table references have been processed, remove unused key definitions. Change-Id: I938099284e34a81886621f6a389f34abc51e78ba
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
You have lost you TRUE AND FALSE defined in my_global.h file
#ifndef MY_GLOBAL_INCLUDED
#define MY_GLOBAL_INCLUDED
/* Define some general constants /
#ifndef TRUE
#define TRUE (1) / Logical true /
#define FALSE (0) / Logical false */
#endif
#endif // MY_GLOBAL_INCLUDED
The text was updated successfully, but these errors were encountered: