Read Free Replication with TokuDB
When row based replication is used, an image of the row is written into the binary log for write, delete, and update operations. The slave can use the row image from the binary log to avoid reading the row from the table. This read free replication design can reduce the I/O load on the slave significantly.
When handling write rows, delete rows, and update rows log events, the MySQL replication slave uses extra queries to verify various properties of the slave's version of the table. For the write rows event, the table's storage engine does hidden queries to verify the uniqueness properties of the indexes on the table. For the delete rows and update rows events, the MySQL replication slave looks up the row prior to deleting it or updating it, respectively.
The intent of these hidden queries is to detect differences between the master and slave versions of a table. For a TokuDB fractal tree, these hidden queries have a very high cost. If the keys are accessed in a random sequence, then each replication event will result in a point query. For large tables, each point query result in a I/O read stall. These stalls are fatal to the slave's performance, and result in large slave lags. In addition, the reads detract from the I/O capacity of the slave and impact slave query performance.
If the goal of replication is to maintain a version of a table on the slave that is eventually consistent with the master, then one can leverage the fact the binary log contains the entire row image that is necessary to execute the write, delete, or update events. The replication slave can avoid the point queries. There may be a need for a lower frequency check on the consistency between the master and the slave.
Here is how TokuDB uses read free replication.
Write rows log event
- When the master successfully inserts a row into a table, it puts a write rows event into the log.
- There must have been no uniqueness violations on the master, unless the client application takes responsibility by setting 'unique_checks=OFF'.
- The slave will stop replication if there is a uniqueness violation when it tries to write the row.
- The cost of a uniqueness check is a point query into the tree, which can result in a costly I/O read.
- If the slave is read only, then the uniqueness checks can be avoided since only replication events can change a table and the master already ran uniqueness checks.
- TokuDB avoids the unique checks on the primary key and any unique secondary keys for write rows log events. TokuDB handles the write row event by constructing a fractal tree insert message from the binlog write event row images and putting a fractal tree insert message into the fractal tree. This has amortized write cost. The combination of no unique checks on the slave and TokuDB fractal tree messaging results in no reads for write replication events.
Delete rows log event
- When the master successfully deletes a row from the table with row based replication, it puts a delete rows event into the log.
- The replication slave uses the row image from the delete rows event to read the row from the table even though the row image is already in the binlog. If the row does not exist, then slave replication stops.
- TokuDB can tell the replication slave to NOT read the row from the table prior to deleting it. In this case, the slave uses the delete row image in the replication event to delete the row.
Update rows log event
- When the master successfully updates a row from the table with row based replication, it puts an update rows event into the log.
- The replication slave uses the row image from the update rows log event to read the row from the table even though the row image is already in the binlog. If the row does not exist, then slave replication stops.
- TokuDB can tell the replication slave to NOT read the row from the table prior to updating it. In this case, the slave uses the row images in the update replication event to update the row.
- http://www.tokutek.com/2014/09/tokudb-v7-5-read-free-replication-the-benchmark/
- http://www.tokutek.com/2014/09/tokudb-read-free-replication-details-and-use-cases/
Master variables
- server-id=
- log-bin=
- sync_binlog=ON
- binlog_format=ROW (default STATEMENT). Put the row image into the binary log.
- binlog_row_image=FULL (default FULL, MySQL 5.6). Put the full row image into the binary log.
Slave variables
- server-id=
- log-bin=
- read_only=ON (default OFF). Run the slave in read only mode.
- master_info_repository=TABLE (MySQL 5.6)
- sync_master_info=1
- relay_log_info_repository=TABLE (MySQL 5.6)
- sync_relay_log_info=1
- slave_parallel_workers (MySQL 5.6)
- slave_parallel_threads (MariaDB 10)
TokuDB slave variables
- tokudb_rpl_unique_checks=OFF (default ON). Turn off tokudb unique checks for a write row log event or an update row log event.
- tokudb_rpl_unique_checks_delay=MILLISECONDS (default 0). Simulate a long disk read by sleeping for some time prior to the unique checks query.
- tokudb_rpl_lookup_rows=OFF (default ON). Turn off row lookups for a delete row log event or an update row log event.
- tokudb_rpl_lookup_rows_delay=MILLISECONDS (default 0). Simulate a long disk read by sleeping for some time prior to the row lookup query.
- tokudb_rpl_check_readonly=ON (default ON). Check if the slave is read only, and if it is, then do check unique constraints.
- Are all of the key values included in the row images?
- Does the table need a primary key defined?