Skip to content

[IOTDB-4758]Delete snapshot after region migration#7739

Merged
xingtanzjr merged 4 commits intoapache:masterfrom
suchenglong:fix_mulitleader_spot_clean
Oct 27, 2022
Merged

[IOTDB-4758]Delete snapshot after region migration#7739
xingtanzjr merged 4 commits intoapache:masterfrom
suchenglong:fix_mulitleader_spot_clean

Conversation

@suchenglong
Copy link
Copy Markdown
Contributor

@suchenglong suchenglong commented Oct 26, 2022

Description

Content1 ...

Delete remote snapshot after region migration

test

Test plan:

  1. Enable the cluster 1C3D and two replicas
  2. kiil drops the process where DataNode2 resides
  3. Run the show regions command to view the replication information
  4. Use the remove_node.sh script to remove DataNode2
  5. Check whether snapshots are generated for the snapshot directories in DataNode1 and DataNode3
  6. Wait for a period of time, and check whether the temporary file that completes the replica migration is deleted in DataNode1 and DataNode3. Run the show regions command to check whether the replica migration is correct
  7. Connect to the datanode where the migrated replica resides to query data and verify that data is normal

Result:
image
Before volume reduction:
image
Stop node 2:
image
Removing node 2:
image
image
The data migration is complete and no temporary snapshot file is available:
image
Query migrated replica data on datanoe1:
image


This PR has:

  • been self-reviewed.
    • concurrent read
    • concurrent write
    • concurrent read and write
  • added documentation for new or modified features or behaviors.
  • added Javadocs for most classes and all non-trivial methods.
  • added or updated version, license, or notice information
  • added comments explaining the "why" and the intent of the code wherever would not be obvious
    for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold
    for code coverage.
  • added integration tests.
  • been tested in a test IoTDB cluster.

Key changed/added classes (or packages if there are too many classes) in this PR

impl.activePeer(peer);

// step 7: spot clean
logger.info("[MultiLeaderConsensus] do spot clean...");
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why clean in addPeer()?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After a peer is added, the snapshot replicas need to be transferred. After the snapshot replicas are transferred and loaded, you need to clear the temporary replica files on the remote server to prevent data space waste.

@xingtanzjr xingtanzjr merged commit 0767935 into apache:master Oct 27, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants