New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
HDDS-7048. EC: Add debug logging with exception info when stripe write failed #3625
Conversation
hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/io/ECKeyOutputStream.java
Show resolved
Hide resolved
|
||
String failedStreamsString = IntStream.range(1, | ||
numDataBlks + numParityBlks + 1) | ||
.mapToObj(index -> failedStreamIndexMap.containsKey(index) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
change P -> S . meaning "Success"
...t/src/main/java/org/apache/hadoop/ozone/client/io/ECBlockReconstructedStripeInputStream.java
Show resolved
Hide resolved
@@ -203,6 +206,25 @@ private void encodeAndWriteParityCells() throws IOException { | |||
} | |||
} | |||
|
|||
private void logStreamError(List<ECBlockOutputStream> failedStreams, | |||
String operation) { | |||
Set<Integer> failedStreamIndexMap = |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You may want to change this variable name to set.
.mapToObj(index -> failedStreamIndexMap.contains(index) | ||
? "F" : "S") | ||
.collect(Collectors.joining(" ")); | ||
LOG.error("{} failed: {}", operation, failedStreamsString); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In EC we will actually proceed by closing current blockGroup. Should we change the log level to warn instead?
...t/src/main/java/org/apache/hadoop/ozone/client/io/ECBlockReconstructedStripeInputStream.java
Show resolved
Hide resolved
* Get the Replication Index. | ||
* @return replicationIndex | ||
*/ | ||
public int getReplicationIndex() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can this be in default scope?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The class we are accessing from is not in the same package
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Git it. This is ok.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, pending CI
… stripe write failed (apache#3625) Change-Id: I364a38c34e6f554f58cdbbc1e5ffa9c3a19d3497
What changes were proposed in this pull request?
Adding debug Logs for EC Stripe Write Failures
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-7048
How was this patch tested?
Logging Changes