Skip to content

Commit

Permalink
Implement parallel doublewrite
Browse files Browse the repository at this point in the history
Blueprint:
https://blueprints.launchpad.net/percona-server/+spec/parallel-doublewrite

New data structure for a single doublewrite partition: struct
parallel_dlbwr_shard_t, and a new structure for the whole doublewrite:
struct parallel_dblwr_t.

Remove batch doublewrite fields from struct buf_dblwr_t: first_free,
b_reserved, b_event, and batch_running.

New methods to support parallel doublewrite:
buf_parallel_dblwr_make_path, buf_parallel_dblwr_close,
buf_parallel_dblwr_delete, buf_parallel_dblwr_finish_recovery,
buf_parallel_dblwr_file_create, buf_parallel_dblwr_create,
buf_parallel_dblwr_destroy, buf_parallel_dblwr_partition,
buf_parallel_dblwr_shard_num.  New global variable
parallel_dblwr_buf.

Make buf_dblwr_init_or_load_pages to read any pages from the parallel
doublewrite file.

Delete the old one if existing and re-create the doublewrite buffer at
the end of buf_dblwr_process.

Make buf_dblwr_update to decrement atomically the remaining to process
page count for the batch flushing modes, and signal the doublewrite
partition event on the last one.

Make buf_dblwr_flush_buffered_writes take a doublewrite partition
index argument, remove the logic to sync with an already running
batch.

Make buf_dblwr_add_to_batch take a flush type argument in order to
find the correct doublewrite shard for the page being added. Remove
all logic to sync with an already running batch.

In buf_dblwr_update and buf_dblwr_flush_single_page, allow single page
flushes to take any slot in the legacy doublewrite buffer.  Simplify
the latter function.

Remove any synchronous flushing code so that no thread except from the
cleaner/LRU manager threads can do any kind of flushing by
itself. Instead, any other thread has to wait for the flusher threads
to do the required flushing. This affects buf_flush_sync_all_pools,
and log_preflush_pool_modified_pages.

Do not make log_checkpoint_at flush at the very end of server shutdown
when neither the page cleaner is running nor there can be dirty pages
in the buffer pool.

Destroy (and delete its file) the parallel doublewrite at the end of
logs_empty_and_mark_files_at_shutdown.

Delete the existing doublewrite buffer, if any, in
recv_recovery_from_checkpoint_start, if found that the instance does
not need crash recovery.

Create the doublewrite buffer in
innobase_start_or_create_for_mysql. Add an assert that any pages from
the old buffer have been used or it was impossible to use them due to
some condition (an error, high innodb_force_recovery value, etc).

Introduce new server variable innodb_parallel_doublewrite_path.

Register the parallel doublewrite file with Performance Schema.

Fix bug 1549301 / http://bugs.mysql.com/bug.php?id=80496
(buf_dblwr_init_or_load_pages now returns an error code, but caller
not updated) as the function in question now can return many more
errors related to the parallel doublewrite file I/O.

Add new possible flag, OS_FILE_O_SYNC, for os_file_create_simple_func.

Move suite/innodb/t/doublewrite.test to
include/doublewrite.inc. Create new innodb.doublewrite testcase that
simply includes that file and so has the same effect. Also create
innodb.doublewrite_multiple_pools that includes the same, but has two
buffer pool instances so that the tested data pages fall into
different doublewrite shards.

Create new testcase innodb.percona_doublewrite that tests the
doublewrite file handling, path setting, and error detection.

Create new testcase innodb.percona_doublewrite_single_page_flush that
tests the single page flush code path of the legacy doublewrite.

Update perfschema.show_sanity and create
sys_vars.innodb_percona_doublewrite_path testcase for the new
variable.

Make include/start_mysqld.inc and restart_mysqld.inc accept a one-shot
option $restart_hide_args, that would not print the actual server args
after the "# restart" string. This is done in order to test server
restart where the passed args should not be recorded in the output,
such as temp file paths.

Remove xb_doublewrite file in selected testcases manually as needed.
  • Loading branch information
laurynas-biveinis committed Mar 7, 2016
1 parent 43e28f9 commit 4596aaa
Show file tree
Hide file tree
Showing 37 changed files with 1,713 additions and 638 deletions.
10 changes: 9 additions & 1 deletion mysql-test/include/restart_mysqld.inc
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,15 @@ if (!$restart_parameters)

# We use --exec echo instead of simple "--echo" because
# replace_result works on --exec
--exec echo "# $restart_parameters"
if (!$restart_hide_args)
{
--exec echo "# $restart_parameters"
}
if ($restart_hide_args)
{
--echo # restart:<hidden args>
let $restart_hide_args=;
}

# Write file to make mysql-test-run.pl expect the "crash", but don't start
# it until it's told to
Expand Down
10 changes: 9 additions & 1 deletion mysql-test/include/start_mysqld.inc
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,15 @@ if (!$restart_parameters)

# We use --exec echo instead of simple "--echo" because
# replace_result works on --exec
--exec echo "# $restart_parameters"
if (!$restart_hide_args)
{
--exec echo "# $restart_parameters"
}
if ($restart_hide_args)
{
--echo # restart:<hidden args>
let $restart_hide_args=;
}

# Include this script only after using shutdown_mysqld.inc
# or kill_mysqld.inc or expect_crash.inc
Expand Down
Loading

0 comments on commit 4596aaa

Please sign in to comment.