Skip to content

HDFS-15737. Don't remove datanodes from outOfServiceNodeBlocks while checking in DatanodeAdminManager #2562

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
NickyYe wants to merge 1 commit into from
Closed

HDFS-15737. Don't remove datanodes from outOfServiceNodeBlocks while checking in DatanodeAdminManager #2562

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 19 additions & 2 deletions ...dfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeAdminManager.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -289,6 +289,21 @@ public void startMaintenance(DatanodeDescriptor node,
*/
@VisibleForTesting
public void stopMaintenance(DatanodeDescriptor node) {
stopMaintenance(node, true);
}

/**
* Stop maintenance of the specified datanode with option to remove it from
* outOfServiceNodeBlocks to avoid ConcurrentModificationException while
* iterating it.
*
* @param node datanode to stop maintenance
* @param removeOutOfServiceNodeBlocks false means the node won't be removed
* from outOfServiceNodeBlocks. Caller
* will do that.
*/
private void stopMaintenance(DatanodeDescriptor node,
boolean removeOutOfServiceNodeBlocks) {
if (node.isMaintenance()) {
// Update DN stats maintained by HeartbeatManager
hbManager.stopMaintenance(node);
Expand Down Expand Up @@ -324,7 +339,9 @@ public void stopMaintenance(DatanodeDescriptor node) {

// Remove from tracking in DatanodeAdminManager
pendingNodes.remove(node);
outOfServiceNodeBlocks.remove(node);
if (removeOutOfServiceNodeBlocks) {
outOfServiceNodeBlocks.remove(node);
}
} else {
LOG.trace("stopMaintenance: Node {} in {}, nothing to do." +
node, node.getAdminState());
Expand Down Expand Up @@ -556,7 +573,7 @@ private void check() {
boolean fullScan = false;
if (dn.isMaintenance() && dn.maintenanceExpired()) {
// If maintenance expires, stop tracking it.
stopMaintenance(dn);
stopMaintenance(dn, false);
toRemove.add(dn);
continue;
}
Expand Down