Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,13 @@ ms.topic: how-to
> [!NOTE]
> This article contains references to the term *slave*, a term that Microsoft no longer uses. When the term is removed from the software, we'll remove it from this article.

You can use methods such as MySQL dump and restore, MySQL Workbench Export and Import, or Azure Database Migration Service to migrate your MySQL databases to Azure Database for MySQL flexible server. You can migrate your workloads with minimum downtime by using a combination of open-source tools such as mysqldump or mydumper and myloader with Data-in Replication.
You can use methods such as MySQL dump and restore, MySQL Workbench Export and Import, or Azure Database Migration Service to migrate your MySQL databases to Azure Database for MySQL Flexible Server. You can migrate your workloads with minimum downtime by using a combination of open-source tools such as mysqldump or mydumper and myloader with Data-in Replication.

Data-in Replication is a technique that replicates data changes from the source server to the destination server based on the binary log file position method. In this scenario, the MySQL instance operating as the source (on which the database changes originate) writes updates and changes as *events* to the binary log. The information in the binary log is stored in different logging formats according to the database changes being recorded. Replicas are configured to read the binary log from the source and execute the events in the binary log on the replica's local database.

Set up [Data-in Replication](../flexible-server/concepts-data-in-replication.md) to synchronize data from a source MySQL server to a target MySQL server. You can do a selective cutover of your applications from the primary (or source database) to the replica (or target database).

In this tutorial, you'll learn how to set up Data-in Replication between a source server that runs Amazon Relational Database Service (RDS) for MySQL and a target server that runs Azure Database for MySQL flexible server.
In this tutorial, you'll learn how to set up Data-in Replication between a source server that runs Amazon Relational Database Service (RDS) for MySQL and a target server that runs Azure Database for MySQL Flexible Server.

## Performance considerations

Expand All @@ -35,7 +35,7 @@ Before you begin this tutorial, consider the performance implications of the loc

Perform dump or restore operations from a client computer that's launched in the same location as the database server:

- For Azure Database for MySQL flexible server instances, the client machine should be in the same virtual network and availability zone as the target database server.
- For Azure Database for MySQL Flexible Server instances, the client machine should be in the same virtual network and availability zone as the target database server.
- For source Amazon RDS database instances, the client instance should exist in the same Amazon Virtual Private Cloud and availability zone as the source database server.
In the preceding case, you can move dump files between client machines by using file transfer protocols like FTP or SFTP or upload them to Azure Blob Storage. To reduce the total migration time, compress files before you transfer them.

Expand All @@ -51,16 +51,16 @@ No matter where the client computer is located, it requires adequate computing,

To complete this tutorial, you need to:

- Install the [mysqlclient](https://dev.mysql.com/downloads/) on your client computer to create a dump, and perform a restore operation on your target Azure Database for MySQL flexible server instance.
- Install the [mysqlclient](https://dev.mysql.com/downloads/) on your client computer to create a dump, and perform a restore operation on your target Azure Database for MySQL Flexible Server instance.
- For larger databases, install [mydumper and myloader](https://centminmod.com/mydumper.html) for parallel dumping and restoring of databases.

> [!NOTE]
> Mydumper can only run on Linux distributions. For more information, see [How to install mydumper](https://github.com/maxbube/mydumper#how-to-install-mydumpermyloader).

- Create an instance of Azure Database for MySQL flexible server that runs version 5.7 or 8.0.
- Create an instance of Azure Database for MySQL Flexible Server that runs version 5.7 or 8.0.

> [!IMPORTANT]
> If your target is Azure Database for MySQL flexible server with zone-redundant high availability (HA), note that Data-in Replication isn't supported for this configuration. As a workaround, during server creation set up zone-redundant HA:
> If your target is Azure Database for MySQL Flexible Server with zone-redundant high availability (HA), note that Data-in Replication isn't supported for this configuration. As a workaround, during server creation set up zone-redundant HA:
>
> 1. Create the server with zone-redundant HA enabled.
> 1. Disable HA.
Expand All @@ -74,43 +74,43 @@ Ensure that several parameters and features are configured and set up properly,
- Have a primary key in each table. A lack of primary keys on tables can slow the replication process.
- Make sure the character set of the source and the target database are the same.
- Set the `wait_timeout` parameter to a reasonable time. The time depends on the amount of data or workload you want to import or migrate.
- Verify that all your tables use InnoDB. Azure Database for MySQL flexible server only supports the InnoDB storage engine.
- Verify that all your tables use InnoDB. Azure Database for MySQL Flexible Server only supports the InnoDB storage engine.
- For tables with many secondary indexes or large tables, performance overhead effects are visible during restore. Modify the dump files so that the `CREATE TABLE` statements don't include secondary key definitions. After you import the data, re-create secondary indexes to avoid the performance penalty during the restore process.

Finally, to prepare for Data-in Replication:

- Verify that the target Azure Database for MySQL flexible server instance can connect to the source Amazon RDS for MySQL server over port 3306.
- Verify that the target Azure Database for MySQL Flexible Server instance can connect to the source Amazon RDS for MySQL server over port 3306.
- Ensure that the source Amazon RDS for MySQL server allows both inbound and outbound traffic on port 3306.
- Make sure you provide [site-to-site connectivity](/azure/vpn-gateway/tutorial-site-to-site-portal) to your source server by using either [Azure ExpressRoute](/azure/expressroute/expressroute-introduction) or [Azure VPN Gateway](/azure/vpn-gateway/vpn-gateway-about-vpngateways). For more information about creating a virtual network, see the [Azure Virtual Network documentation](/azure/virtual-network/). Also see the quickstart articles with step-by-step details.
- Configure your source database server's network security groups to allow the target Azure Database for MySQL flexible server IP address.
- Configure your source database server's network security groups to allow the target Azure Database for MySQL Flexible Server IP address.

> [!IMPORTANT]
> If the source Amazon RDS for MySQL instance has GTID_mode set to ON, the target instance of Azure Database for MySQL flexible server must also have GTID_mode set to ON.
> If the source Amazon RDS for MySQL instance has GTID_mode set to ON, the target instance of Azure Database for MySQL Flexible Server must also have GTID_mode set to ON.

## Configure the target instance of Azure Database for MySQL

To configure the target instance of Azure Database for MySQL flexible server, which is the target for Data-in Replication:
To configure the target instance of Azure Database for MySQL Flexible Server, which is the target for Data-in Replication:

1. Set the `max_allowed_packet` parameter value to the maximum of **1073741824**, which is 1 GB. This value prevents any overflow issues related to long rows.
1. Set the `slow_query_log`, `general_log`, `audit_log_enabled`, and `query_store_capture_mode` parameters to **OFF** during the migration to help eliminate any overhead related to query logging.
1. Scale up the compute size of the target Azure Database for MySQL flexible server instance to the maximum of 64 vCores. This size provides more compute resources when restoring the source server's database dump.
1. Scale up the compute size of the target Azure Database for MySQL Flexible Server instance to the maximum of 64 vCores. This size provides more compute resources when restoring the source server's database dump.

You can always scale back the compute to meet your application demands after the migration is complete.

1. Scale up the storage size to get more IOPS during the migration or increase the maximum IOPS for the migration.

> [!NOTE]
> Available maximum IOPS are determined by compute size. For more information, see the IOPS section in [Compute and storage options in Azure Database for MySQL flexible server](../flexible-server/concepts-compute-storage.md#iops).
> Available maximum IOPS are determined by compute size. For more information, see the IOPS section in [Compute and storage options in Azure Database for MySQL Flexible Server](../flexible-server/concepts-compute-storage.md#iops).

## Configure the source Amazon RDS for MySQL server

To prepare and configure the MySQL server hosted in Amazon RDS, which is the *source* for Data-in Replication:

1. Confirm that binary logging is enabled on the source Amazon RDS for MySQL server. Check that automated backups are enabled, or ensure a read replica exists for the source Amazon RDS for MySQL server.

1. Ensure that the binary log files on the source server are retained until after the changes are applied on the target instance of Azure Database for MySQL flexible server.
1. Ensure that the binary log files on the source server are retained until after the changes are applied on the target instance of Azure Database for MySQL Flexible Server.

With Data-in Replication, Azure Database for MySQL flexible server doesn't manage the replication process.
With Data-in Replication, Azure Database for MySQL Flexible Server doesn't manage the replication process.

1. To check the binary log retention on the source Amazon RDS server to determine the number of hours the binary logs are retained, call the `mysql.rds_show_configuration` stored procedure:

Expand Down Expand Up @@ -145,7 +145,7 @@ There are two ways to capture a dump of data from the source Amazon RDS for MySQ
You can also temporarily set the `read_only` parameter to a value of **1** so that writes aren't processed when you're capturing a dump of data.

1. After you stop the writes on the source server, collect the binary log file name and offset by running the command `Mysql> Show master status;`.
1. Save these values to start replication from your Azure Database for MySQL flexible server instance.
1. Save these values to start replication from your Azure Database for MySQL Flexible Server instance.
1. To create a dump of the data, execute `mysqldump` by running the following command:

```
Expand All @@ -163,15 +163,15 @@ There are two ways to capture a dump of data from the source Amazon RDS for MySQ
```

1. With replication stopped, connect to the replica. Then run the `SHOW SLAVE STATUS` command to retrieve the current binary log file name from the **Relay_Master_Log_File** field and the log file position from the **Exec_Master_Log_Pos** field.
1. Save these values to start replication from your Azure Database for MySQL flexible server instance.
1. Save these values to start replication from your Azure Database for MySQL Flexible Server instance.
1. To create a dump of the data from the Amazon RDS for MySQL read replica, execute `mysqldump` by running the following command:

```
$ mysqldump -h hostname -u username -p –single-transaction –databases dbnames –order-by-primary> dumpname.sql
```

> [!NOTE]
> You can also use mydumper for capturing a parallelized dump of your data from your source Amazon RDS for MySQL database. For more information, see [Migrate large databases to Azure Database for MySQL flexible server using mydumper/myloader](../single-server/concepts-migrate-mydumper-myloader.md).
> You can also use mydumper for capturing a parallelized dump of your data from your source Amazon RDS for MySQL database. For more information, see [Migrate large databases to Azure Database for MySQL Flexible Server using mydumper/myloader](../single-server/concepts-migrate-mydumper-myloader.md).

## Link source and replica servers to start Data-in Replication

Expand All @@ -182,7 +182,7 @@ There are two ways to capture a dump of data from the source Amazon RDS for MySQ
```

> [!NOTE]
> If you're instead using myloader, see [Migrate large databases to Azure Database for MySQL flexible server using mydumper/myloader](../single-server/concepts-migrate-mydumper-myloader.md).
> If you're instead using myloader, see [Migrate large databases to Azure Database for MySQL Flexible Server using mydumper/myloader](../single-server/concepts-migrate-mydumper-myloader.md).

1. Sign in to the source Amazon RDS for MySQL server, and set up a replication user. Then grant the necessary privileges to this user.

Expand All @@ -204,13 +204,13 @@ There are two ways to capture a dump of data from the source Amazon RDS for MySQ

Stored procedures do all Data-in Replication functions. For information about all procedures, see [Data-in Replication stored procedures](../single-server/reference-stored-procedures.md#data-in-replication-stored-procedures). You can run these stored procedures in the MySQL shell or MySQL Workbench.

1. To link the Amazon RDS for MySQL source server and the Azure Database for MySQL flexible server target server, sign in to the target Azure Database for MySQL flexible server instance. Set the Amazon RDS for MySQL server as the source server by running the following command:
1. To link the Amazon RDS for MySQL source server and the Azure Database for MySQL Flexible Server target server, sign in to the target Azure Database for MySQL Flexible Server instance. Set the Amazon RDS for MySQL server as the source server by running the following command:

```
CALL mysql.az_replication_change_master('source_server','replication_user_name','replication_user_password',3306,'<master_bin_log_file>',master_bin_log_position,'<master_ssl_ca>');
```

1. To start replication between the source Amazon RDS for MySQL server and the target Azure Database for MySQL flexible server instance, run the following command:
1. To start replication between the source Amazon RDS for MySQL server and the target Azure Database for MySQL Flexible Server instance, run the following command:

```
Mysql> CALL mysql.az_replication_start;
Expand All @@ -232,16 +232,16 @@ There are two ways to capture a dump of data from the source Amazon RDS for MySQ

To ensure a successful cutover:

1. Configure the appropriate logins and database-level permissions in the target Azure Database for MySQL flexible server instance.
1. Configure the appropriate logins and database-level permissions in the target Azure Database for MySQL Flexible Server instance.
1. Stop writes to the source Amazon RDS for MySQL server.
1. Ensure that the target Azure Database for MySQL flexible server instance has caught up with the source server and that the `Seconds_Behind_Master` value is **0** from `show slave status`.
1. Call the stored procedure `mysql.az_replication_stop` to stop the replication because all changes have been replicated to the target Azure Database for MySQL flexible server instance.
1. Ensure that the target Azure Database for MySQL Flexible Server instance has caught up with the source server and that the `Seconds_Behind_Master` value is **0** from `show slave status`.
1. Call the stored procedure `mysql.az_replication_stop` to stop the replication because all changes have been replicated to the target Azure Database for MySQL Flexible Server instance.
1. Call `mysql.az_replication_remove_master` to remove the Data-in Replication configuration.
1. Redirect clients and client applications to the target Azure Database for MySQL flexible server instance.
1. Redirect clients and client applications to the target Azure Database for MySQL Flexible Server instance.

At this point, the migration is complete. Your applications are connected to the server running Azure Database for MySQL flexible server.
At this point, the migration is complete. Your applications are connected to the server running Azure Database for MySQL Flexible Server.

## Next steps

- For more information about migrating databases to Azure Database for MySQL flexible server, see the [Database Migration Guide](https://github.com/Azure/azure-mysql/tree/master/MigrationGuide).
- View the video [Easily migrate MySQL/PostgreSQL apps to Azure managed service](https://medius.studios.ms/Embed/Video/THR2201?sid=THR2201). It contains a demo that shows how to migrate MySQL apps to Azure Database for MySQL flexible server.
- For more information about migrating databases to Azure Database for MySQL Flexible Server, see the [Database Migration Guide](https://github.com/Azure/azure-mysql/tree/master/MigrationGuide).
- View the video [Easily migrate MySQL/PostgreSQL apps to Azure managed service](https://medius.studios.ms/Embed/Video/THR2201?sid=THR2201). It contains a demo that shows how to migrate MySQL apps to Azure Database for MySQL Flexible Server.