Skip to content

Conversation

@zz12341
Copy link

@zz12341 zz12341 commented Jan 10, 2024

Description of PR

HDFS-16064 fixed an issue where decommissioning replicas were not counted as usable replicas, which caused decom to stuck forever in the case of small clusters. We are seeing the same issue on 2.10, and thus backporting the changes.

How was this patch tested?

For code changes:

  • [ x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

return minReplication;
}

public short getMinStorageNum(BlockInfo block) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zz12341 Why we want these changes? I don't see it in the original patch here.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was originally trying to make it in sync with what trunk branch is doing: https://github.com/apache/hadoop/pull/4410/files#diff-305ecf45a0f0708849b5e3c0d21a56c681db3a1497e52a19ef24939278dc99feL1922-R1926

Let me revert this change

@zz12341 zz12341 force-pushed the backporting-hdfs-16064-to-2.10.0 branch from 447f90e to 6df043a Compare January 11, 2024 00:01
@zz12341 zz12341 requested a review from shahrs87 January 11, 2024 00:04
@zz12341 zz12341 marked this pull request as draft January 11, 2024 00:27
expectedRedundancies;

boolean minReplicationSatisfied =
numberOfReplicas.liveReplicas() >= minReplication;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
numberOfReplicas.liveReplicas() >= minReplication;
numUsableReplicas >= minReplication;

@shahrs87
Copy link

@zz12341 branch-2.10 is the active branch for 2.10 (not branch-2.10.0). You can see all the active branches here.

@shahrs87
Copy link

Screenshot 2024-01-11 at 9 45 49 AM
@zz12341 This change in BlockManager#addStoredBlock is missing from the original patch? #4410

@shahrs87
Copy link

[HDFS-16064] backporting HDFS-16064. Determine when to invalidate corrupt replicas based on number of usable replicas (#4410) #6437

@zz12341 Keep the title and the commit message same as #4410. We shouldn't add backporting keyword in the title and commit message.

@github-actions
Copy link
Contributor

github-actions bot commented Oct 3, 2025

We're closing this stale PR because it has been open for 100 days with no activity. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you feel like this was a mistake, or you would like to continue working on it, please feel free to re-open it and ask for a committer to remove the stale tag and review again.
Thanks all for your contribution.

@github-actions github-actions bot added the Stale label Oct 3, 2025
@github-actions github-actions bot closed this Oct 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants