Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change the 16 shards of the original table to 32 shards #31242

Open
ttfont opened this issue May 16, 2024 · 6 comments
Open

Change the 16 shards of the original table to 32 shards #31242

ttfont opened this issue May 16, 2024 · 6 comments

Comments

@ttfont
Copy link

ttfont commented May 16, 2024

Question

May I ask a question, now I want to change the 16 shards of the original table to 32 shards, will the original data be automatically balanced? How to operate?

@TherChenYang
Copy link
Collaborator

@ttfont For this scenario, the first thing you need to do is data migration and rebalance the data. You can use the built-in migration feature of ShardingSphere or tools like dataX to complete the data migration, and then change the corresponding sharding configuration after migration.

@ttfont
Copy link
Author

ttfont commented May 20, 2024

I understand that the task is to migrate data from 16 shards to a new cluster with 32 shards. @TherChenYang

@ttfont For this scenario, the first thing you need to do is data migration and rebalance the data. You can use the built-in migration feature of ShardingSphere or tools like dataX to complete the data migration, and then change the corresponding sharding configuration after migration.

@TherChenYang
Copy link
Collaborator

@ttfont The first thing you need to do is to migrate the data, and modify the ShardingSphere configuration after migration.
Maybe I didn't quite understand the automatically balanced part above, I understand that this isn't handled on the shardingSphere side.

@ttfont
Copy link
Author

ttfont commented May 21, 2024

@ttfont The first thing you need to do is to migrate the data, and modify the ShardingSphere configuration after migration. Maybe I didn't quite understand the automatically balanced part above, I understand that this isn't handled on the shardingSphere side.

After migrating the data, adjust the number of shards in ShardingSphere, and then migrate the data to ShardingSphere. Is that correct? @TherChenYang

@TherChenYang
Copy link
Collaborator

@ttfont Are you using the ShardingSphere-JDBC or ShardingSphere-Proxy? If it is the JDBC module, ShardingSphere is not responsible for data migration. You need to use a data migration tool to complete the migration of your 16 shards to 32 shards. During the migration process, for both old and new clusters, you need to perform dual write operations by reading data from the old cluster (16 shards). Once the overall migration process is completed, change ShardingSphere's configuration so that both read and write operations are routed to the new cluster.

@ttfont
Copy link
Author

ttfont commented May 27, 2024

Are you using the ShardingSphere-JDBC or ShardingSphere-Proxy? If it is the JDBC module, ShardingSphere is not responsible for data migration. You need to use a data migration tool to complete the migration of your 16 shards to 32 shards. During the migration process, for both old and new clusters, you need to perform dual write operations by reading data from the old cluster (16 shards). Once the overall migration process is completed, change ShardingSphere's configuration so that both read and write operations are routed to the new cluster.

@TherChenYang 3Q

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants