HADOOP-16767 Handle non-IO exceptions in reopen() #2685
Conversation
Contributed by Sergei Poganshev. Catches Exception instead of IOException in closeStream() and so handle exceptions such as SdkClientException by aborting the wrapped stream. This will increase resilience to failures, as any which occuring during stream closure will be caught. Furthermore, because the underlying HTTP connection is aborted, rather than closed, it will not be recycled to cause problems on subsequent operations.
|
Hi @steveloughran , sorry I'm behind getting HADOOP-17338 to 2.10 branch. I have a version ready, but before that, I'd like to get HADOOP-16767 to 2.10. Here are the test results I have with both together (largely some timeout failures). Would you please help getting this one in and I will put HADOOP-17338 on top of it. Thanks a lot. [WARNING] Tests run: 11, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 64.07 s - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractCreate |
|
💔 -1 overall
This message was automatically generated. |
|
Thanks a lot @steveloughran ! |
|
Has yetus started on the PR? as that's what really matters. You can always just link the JIRA to the PR by hand |
|
#2688 Seems the build failed, maybe because I uploaded the diff before your merge thus the base is different. As of now, I don't see your merge of HADOOP-16767. I will rebase to the latest 2.10 branch once your merge shows up and recreate the PR. Thanks Steve. |
Contributed by Sergei Poganshev.
Catches Exception instead of IOException in closeStream()
and so handle exceptions such as SdkClientException by
aborting the wrapped stream. This will increase resilience
to failures, as any which occuring during stream closure
will be caught. Furthermore, because the
underlying HTTP connection is aborted, rather than closed,
it will not be recycled to cause problems on subsequent
operations.
NOTICE
Please create an issue in ASF JIRA before opening a pull request,
and you need to set the title of the pull request which starts with
the corresponding JIRA issue number. (e.g. HADOOP-XXXXX. Fix a typo in YYY.)
For more details, please see https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute