Skip to content
Permalink
Branch: 10.5
Commits on Oct 12, 2019
  1. Merge 10.4 into 10.5

    dr-m committed Oct 12, 2019
  2. Merge 10.3 into 10.4

    dr-m committed Oct 12, 2019
  3. MDEV-20814 Assertion index->is_instant() failed on trivial upgrade fr…

    dr-m committed Oct 12, 2019
    …om 10.1
    
    rec_init_offsets(): Relax the assertion that was added in
    commit 01f45be
    to catch ROW_FORMAT=REDUNDANT records that have fewer fields
    than expected.
    
    This assertion would fail when accessing the records of the
    built-in InnoDB table SYS_INDEXES. The column MERGE_THRESHOLD
    had been effectively instantly added in MariaDB Server 10.2
    (and MySQL 5.7), but is_instant() does not hold for that index.
    Relax the assertion, so that it will not fail in this case.
  4. Null-merge 10.3 into 10.4

    dr-m committed Oct 12, 2019
  5. Merge 10.2 into 10.3

    dr-m committed Oct 12, 2019
  6. MDEV-20813 Assertion failure in buf_flush_init_for_writing() for inno…

    dr-m committed Oct 12, 2019
    …db_immediate_scrub_data_uncompressed=ON
    
    The assertion that was added in
    commit c0c003b
    to augment the fix of MDEV-20805 turns out to be invalid when
    innodb_immediate_scrub_data_uncompressed is enabled.
    In this mode, fsp_init_file_page() will be invoked on data pages
    that have been freed, causing writes of almost-all-zero pages.
    
    btr_page_free(): Adjust the comment.
    
    buf_flush_init_for_writing(): Disable the assertion with a note
    that it should be re-enabled in MDEV-15528.
  7. Fix -Wunused-variable

    dr-m committed Oct 12, 2019
  8. Merge 10.3 into 10.4

    dr-m committed Oct 12, 2019
  9. Merge 10.2 into 10.3

    dr-m committed Oct 12, 2019
  10. Merge 10.1 into 10.2

    dr-m committed Oct 12, 2019
Commits on Oct 11, 2019
  1. Fix -std=c++98 -Wzero-length-array

    dr-m committed Oct 11, 2019
    This is another follow-up fix to
    commit b393e2c
    which turned out to be still broken.
    
    Replace the C++11 keyword 'constexpr' with #define.
    
    debug_sync_t::str: Remove the zero-length array.
    Replace sync->str with reinterpret_cast<char*>(&sync[1]).
  2. After-merge fix: Correct an assertion

    dr-m committed Oct 11, 2019
    During IMPORT TABLESPACE, we do invoke
    buf_flush_init_for_writing() with block==NULL and newest_lsn!=0.
  3. Merge 10.1 into 10.2

    dr-m committed Oct 11, 2019
  4. Fix CMAKE_BUILD_TYPE=Debug

    dr-m committed Oct 11, 2019
    Remove unused variables and type mismatch that was introduced
    in commit b393e2c
    
    Also, fix a typo in the documentation of the parameter, and
    update the test.
  5. MDEV-19514 Defer change buffer merge until pages are requested

    dr-m committed Oct 11, 2019
    We will remove the InnoDB background operation of merging buffered
    changes to secondary index leaf pages. Changes will only be merged as a
    result of an operation that accesses a secondary index leaf page,
    such as a SQL statement that performs a lookup via that index,
    or is modifying the index. Also ROLLBACK and some background operations,
    such as purging the history of committed transactions, or computing
    index cardinality statistics, can cause change buffer merge.
    Encryption key rotation will not perform change buffer merge.
    
    The motivation of this change is to simplify the I/O logic and to
    allow crash recovery to happen in the background (MDEV-14481).
    We also hope that this will reduce the number of "mystery" crashes
    due to corrupted data. Because change buffer merge will typically
    take place as a result of executing SQL statements, there should be
    a clearer connection between the crash and the SQL statements that
    were executed when the server crashed.
    
    In many cases, a slight performance improvement was observed.
    
    This is joint work with Thirunarayanan Balathandayuthapani
    and was tested by Axel Schwenke and Matthias Leich.
    
    The InnoDB monitor counter innodb_ibuf_merge_usec will be removed.
    
    On slow shutdown (innodb_fast_shutdown=0), we will continue to
    merge all buffered changes (and purge all undo log history).
    
    Two InnoDB configuration parameters will be changed as follows:
    
    innodb_disable_background_merge: Removed.
    This parameter existed only in debug builds.
    All change buffer merges will use synchronous reads.
    
    innodb_force_recovery will be changed as follows:
    * innodb_force_recovery=4 will be the same as innodb_force_recovery=3
    (the change buffer merge cannot be disabled; it can only happen as
    a result of an operation that accesses a secondary index leaf page).
    The option used to be capable of corrupting secondary index leaf pages.
    Now that capability is removed, and innodb_force_recovery=4 becomes 'safe'.
    * innodb_force_recovery=5 (which essentially hard-wires
    SET GLOBAL TRANSACTION ISOLATION LEVEL READ UNCOMMITTED)
    becomes safe to use. Bogus data can be returned to SQL, but
    persistent InnoDB data files will not be corrupted further.
    * innodb_force_recovery=6 (ignore the redo log files)
    will be the only option that can potentially cause
    persistent corruption of InnoDB data files.
    
    Code changes:
    
    buf_page_t::ibuf_exist: New flag, to indicate whether buffered
    changes exist for a buffer pool page. Pages with pending changes
    can be returned by buf_page_get_gen(). Previously, the changes
    were always merged inside buf_page_get_gen() if needed.
    
    ibuf_page_exists(const buf_page_t&): Check if a buffered changes
    exist for an X-latched or read-fixed page.
    
    buf_page_get_gen(): Add the parameter allow_ibuf_merge=false.
    All callers that know that they may be accessing a secondary index
    leaf page must pass this parameter as allow_ibuf_merge=true,
    unless it does not matter for that caller whether all buffered
    changes have been applied. Assert that whenever allow_ibuf_merge
    holds, the page actually is a leaf page. Attempt change buffer
    merge only to secondary B-tree index leaf pages.
    
    btr_block_get(): Add parameter 'bool merge'.
    All callers of btr_block_get() should know whether the page could be
    a secondary index leaf page. If it is not, we should avoid consulting
    the change buffer bitmap to even consider a merge. This is the main
    interface to requesting index pages from the buffer pool.
    
    ibuf_merge_or_delete_for_page(), recv_recover_page(): Replace
    buf_page_get_known_nowait() with much simpler logic, because
    it is now guaranteed that that the block is x-latched or read-fixed.
    
    mlog_init_t::mark_ibuf_exist(): Renamed from mlog_init_t::ibuf_merge().
    On crash recovery, we will no longer merge any buffered changes
    for the pages that we read into the buffer pool during the last batch
    of applying log records.
    
    buf_page_get_gen_known_nowait(), BUF_MAKE_YOUNG, BUF_KEEP_OLD: Remove.
    
    btr_search_guess_on_hash(): Merge buf_page_get_gen_known_nowait()
    to its only remaining caller.
    
    buf_page_make_young_if_needed(): Define as an inline function.
    Add the parameter buf_pool.
    
    buf_page_peek_if_young(), buf_page_peek_if_too_old(): Add the
    parameter buf_pool.
    
    fil_space_validate_for_mtr_commit(): Remove a bogus comment
    about background merge of the change buffer.
    
    btr_cur_open_at_rnd_pos_func(), btr_cur_search_to_nth_level_func(),
    btr_cur_open_at_index_side_func(): Use narrower data types and scopes.
    
    ibuf_read_merge_pages(): Replaces buf_read_ibuf_merge_pages().
    Merge the change buffer by invoking buf_page_get_gen().
  6. MDEV-20805 follow-up: Catch writes of bogus pages

    dr-m committed Oct 11, 2019
    buf_flush_init_for_writing(): Assert that FIL_PAGE_TYPE is set
    except when creating a new data file with a dummy first page.
    
    buf_dblwr_create(): Ensure that FIL_PAGE_TYPE on all pages
    will be initialized. Reset buf_dblwr_being_created at the end.
  7. Merge 5.5 into 10.1

    dr-m committed Oct 11, 2019
  8. MDEV-20805 ibuf_add_free_page() is not initializing FIL_PAGE_TYPE first

    dr-m committed Oct 11, 2019
    In the function recv_parse_or_apply_log_rec_body() there are debug checks
    for validating the state of the page when redo log records are being
    applied. Most notably, FIL_PAGE_TYPE should be set before anything else
    is being written to the page.
    
    ibuf_add_free_page(): Set FIL_PAGE_TYPE before performing any other changes.
  9. MDEV-19455: Avoid SET DEBUG_DBUG='-d,...' construct

    dr-m committed Oct 11, 2019
    Apply the correct pattern for debug instrumentation:
    
    SET @save_dbug=@@debug_dbug;
    SET debug_dbug='+d,...';
    ...
    SET debug_dbug=@save_dbug;
    
    Numerous tests use statements of the form
    
    SET debug_dbug='-d,...';
    
    which will inadvertently enable all DBUG tracing output,
    causing unnecessary waste of resources.
  10. MDEV-20804 Speed up main.index_merge_innodb

    dr-m committed Oct 11, 2019
    The test main.index_merge_innodb is taking very much time,
    especially on later versions (10.2 and 10.3).
    
    Some of this could be attributed to the use of INSERT...SELECT,
    which is time-consumingly creating explicit record locks in InnoDB
    for the locking read in the SELECT part.
    
    In 10.3 and later, some slowness can be attributed to MDEV-12288,
    which makes the InnoDB purge thread spend time to reset transaction
    identifiers in the inserted records. If we prevent purge from
    running before all tables are dropped, the test seems to be
    10% faster on an unoptimized debug build on 10.5. (A proper fix
    would be to implement MDEV-515 and stop writing row-level undo log
    records for inserts into an empty table or partition.)
    
    At the same time, it should not hurt to make main.index_merge_myisam
    to use the sequence engine. Not only could it be a little faster,
    but the test would be slightly more readable.
  11. Merge 10.4 into 10.5

    dr-m committed Oct 11, 2019
  12. Add encryption.innodb-redo-badkey,strict_full_crc32

    dr-m committed Oct 11, 2019
    In commit 0f7732d
    we introduced a innodb_checksum_algorithm=full_crc32 combination
    to a number of encryption tests, and also fixed the code accordingly.
    
    The default in MariaDB 10.5 is innodb_checksum_algorithm=full_crc32.
    In a test merge to 10.5, the test encryption.innodb-redo-badkey failed
    once due to a message that had been added in that commit.
    
    Let us introduce a full_crc32 option to that test.
    And let us use strict_crc32 and strict_full_crc32 instead of the
    non-strict variants, for the previously augmented tests, to be in
    line with the earlier tests encryption.corrupted_during_recovery and
    encryption.innodb_encrypt_temporary_tables.
Commits on Oct 10, 2019
  1. Merge 10.3 into 10.4

    dr-m committed Oct 10, 2019
  2. MDEV-19783: Relax an assertion

    dr-m committed Oct 10, 2019
    btr_page_get_split_rec_to_left(): Assert that in the leftmost leaf page,
    if the metadata record exists, index->is_instant() must hold.
    The assertion of commit 01f45be
    could fail during innobase_instant_try().
  3. MDEV-19783: Add more assertions

    dr-m committed Oct 10, 2019
    btr_page_get_split_rec_to_left(): Assert that in the leftmost leaf page,
    the metadata record exists if and only if index->is_instant().
    
    page_validate(): Correct the wording of a message.
    
    rec_init_offsets(): Assert that whenever a record is in "instant ALTER"
    format, index->is_instant() must hold.
  4. Merge 10.2 into 10.3

    dr-m committed Oct 10, 2019
  5. MDEV-20788: Bogus assertion failure for PAGE_FREE list

    dr-m committed Oct 10, 2019
    In MDEV-11369 (instant ADD COLUMN) in MariaDB Server 10.3,
    we introduced the hidden metadata record that must be the
    first record in the clustered index if and only if
    index->is_instant() holds.
    
    To catch MDEV-19783, in
    commit ed0793e and
    commit 99dc40d
    we added some assertions to find cases where
    the metadata record is missing while it should not be, or a
    record exists when it should not. Those assertions were invalid
    when traversing the PAGE_FREE list. That list can contain anything;
    we must only be able to determine the successor and the size of
    each garbage record in it.
    
    page_validate(), page_simple_validate_old(), page_simple_validate_new():
    Do not invoke page_rec_get_next_const() for traversing the PAGE_FREE
    list, but instead use a lower-level accessor that does not attempt to
    validate the REC_INFO_MIN_REC_FLAG.
    
    page_copy_rec_list_end_no_locks(),
    page_copy_rec_list_start(), page_delete_rec_list_start():
    Add assertions.
    
    btr_page_get_split_rec_to_left(): Remove a redundant return value,
    and make the output parameter the return value.
    
    btr_page_get_split_rec_to_right(), btr_page_split_and_insert(): Clean up.
  6. MDEV-19335 adjustment for innodb_checksum_algorithm=full_crc32

    dr-m committed Oct 10, 2019
    When MDEV-12026 introduced innodb_checksum_algorithm=full_crc32 in
    MariaDB 10.4, it accidentally added a dependency on buf_page_t::encrypted.
    Now that the flag has been removed, we must adjust the page-read routine.
    
    buf_page_io_complete(): When the full_crc32 page checksum matches but the
    tablespace ID in the page does not match after decrypting, we should
    declare it a decryption failure and suppress the page dump output and
    any attempts to re-read the page.
  7. MDEV-274: Fix unresolved link-time references

    dr-m committed Oct 10, 2019
    Field::marked_for_read() and Field::marked_for_write_or_computed()
    are being called from plugin/type_inet/sql_type_inet.cc ever since
    commit 6ea5c2b
    and thus they cannot be declared inline any more.
  8. Merge 10.3 into 10.4

    dr-m committed Oct 10, 2019
Commits on Oct 9, 2019
  1. Rename log_make_checkpoint_at() to log_make_checkpoint()

    dr-m committed Oct 9, 2019
    The function was always called with lsn=LSN_MAX.
    Remove that redundant parameter.
    
    Spotted by Thirunarayanan Balathandayuthapani.
  2. Merge 10.2 into 10.3

    dr-m committed Oct 9, 2019
  3. MDEV-20591: Follow-up fix

    dr-m committed Oct 9, 2019
    calc_field_event_length(): For type=MYSQL_TYPE_BLOB and meta==0,
    return 0 instead of *ptr+1. This was noted by -Wimplicit-fallthrough.
  4. Merge 10.1 into 10.2

    dr-m committed Oct 9, 2019
  5. Add page_has_prev(), page_has_next(), page_has_siblings()

    dr-m committed Oct 8, 2019
    Until now, InnoDB inefficiently compared the aligned fields
    FIL_PAGE_PREV, FIL_PAGE_NEXT to the byte-order-agnostic value FIL_NULL.
    
    This is a backport of 32170f8
    from MariaDB Server 10.3.
Older
You can’t perform that action at this time.