HDFS-17001. Support getStatus API in WebHDFS #5628
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
💔 -1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
@ayushtkn @slfan1989 Would you be so kind as to review my pull request, please? Thank you very much! |
ayushtkn
reviewed
May 10, 2023
| @@ -2178,6 +2179,19 @@ HdfsFileStatus decodeResponse(Map<?, ?> json) { | |||
| return status.makeQualified(getUri(), f); | |||
| } | |||
|
|
|||
| @Override | |||
| public FsStatus getStatus(Path f) throws IOException { | |||
Comment on lines
1546
to
1548
| private long getStateAtIndex(long[] states, int index) { | ||
| return states.length > index ? states[index] : -1; | ||
| } |
There was a problem hiding this comment.
The same is defined in DfsClient, can we make the definition over there public static and use it here as well, rather than defining it twice?
| } | ||
|
|
||
| public static Map<String, Object> toJsonMap(FsStatus status) | ||
| throws IOException { |
There was a problem hiding this comment.
this doesn't throw IOE, can be removed, once you remove it from here, I think the throws IOE can be removed from the above method as well
Comment on lines
1207
to
1211
| "FsStatus": { | ||
| "used": 0, | ||
| "remaining": 0, | ||
| "capacity":0 | ||
| } |
There was a problem hiding this comment.
can you try it in an actual cluster, get a better example rather than having all 0
| final Configuration conf = WebHdfsTestUtil.createConf(); | ||
| try { | ||
| cluster = new MiniDFSCluster.Builder(conf) | ||
| .numDataNodes(1) |
There was a problem hiding this comment.
datanodes are 1 by default, this line isn't required
Comment on lines
2280
to
2282
| Assert.assertTrue(fsStatus.getUsed() >= 0); | ||
| Assert.assertTrue(fsStatus.getRemaining() >= 0); | ||
| Assert.assertTrue(fsStatus.getCapacity() >= 0); |
There was a problem hiding this comment.
there is already a static import for these. No need of Assert. prefix.
Rather than just asserting they aren't 0, can you get the values from DistributedFileSystem and validate that they are same
There was a problem hiding this comment.
Thank you for your valuable suggestion. I greatly appreciate it and will promptly make the necessary changes to the code!
This comment was marked as outdated.
This comment was marked as outdated.
|
💔 -1 overall
This message was automatically generated. |
|
@ayushtkn @slfan1989 Could you please help review this PR again? The |
ayushtkn
approved these changes
May 11, 2023
slfan1989
reviewed
May 12, 2023
| dfsFsStatus.getCapacity()); | ||
| } finally { | ||
| cluster.shutdown(); | ||
| } |
There was a problem hiding this comment.
@zhtttylz Thanks for the contribution! should we close the os?
There was a problem hiding this comment.
Thank you for your valuable suggestion. I sincerely appreciate it and will promptly implement the required adjustments to the code!
|
@ayushtkn Thank you very much for your valuable suggestion. We will create a ticket to add this feature to HTTPFs! |
|
💔 -1 overall
This message was automatically generated. |
|
@slfan1989 Would you kindly assist in reviewing this PR once again?The |
slfan1989
approved these changes
May 12, 2023
|
@ayushtkn @slfan1989 Thank you for your assistance in reviewing the code! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
JIRA: HDFS-17001. Support getStatus API in WebHDFS