HDDS-9674. Read from non-datanode host should consider topology#5610
Merged
adoroszlai merged 2 commits intoapache:masterfrom Nov 28, 2023
Merged
HDDS-9674. Read from non-datanode host should consider topology#5610adoroszlai merged 2 commits intoapache:masterfrom
adoroszlai merged 2 commits intoapache:masterfrom
Conversation
Contributor
|
@tanvipenumudy @vtutrinov @ibrusentsev can you please take a look? |
Contributor
|
@adoroszlai thanks for the PR, LGTM+ |
vtutrinov
approved these changes
Nov 16, 2023
Contributor
|
No objections, LGTM |
adoroszlai
requested review from
GeorgeJahad
and removed request for
GeorgeJahad
tanvipenumudy
approved these changes
Nov 21, 2023
Contributor
tanvipenumudy
left a comment
tanvipenumudy
left a comment
There was a problem hiding this comment.
The changes look good to me +1, thank you.
Contributor
Author
|
@ChenSammi please review |
Contributor
Author
|
Thanks @ibrusentsev, @sodonnel, @tanvipenumudy, @vtutrinov for the review. |
jojochuang
pushed a commit
to jojochuang/ozone
that referenced
this pull request
Feb 1, 2024
…topology (apache#5610) (cherry picked from commit ee6a4ad) Change-Id: I627ec0c82700bdc357ef5c3ae6fa496eaa8826ae
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Client's node (place in the topology) is looked up via
SCMNodeManager.getNodesByAddress. ButSCMNodeManageronly knows about datanodes. If the request comes from a host which does not run Ozone Datanode, then read will not be topology-aware. While read cannot be host-local in this case, it still could benefit from being rack-local.Examples where such non-datanode hosts may occur:
https://issues.apache.org/jira/browse/HDDS-9674
How was this patch tested?
Added unit test.
Manually tested on
ozone-topologyenvironment:s3gcontainer indocker-compose.yamlwith address10.5.0.73scm(inrack1) ands3g(inrack2) in topologynetwork-configtest.shto runbasic/basic.robotands3/objectputget.robotKeyManagerImplOM log shows datanodes in the same rack were sorted to head of list for both
scmands3g:CI:
https://github.com/adoroszlai/ozone/actions/runs/6882001098