HADOOP-16340. ABFS driver continues to retry on IOException responses from REST operations

[rlevas]

ABFS driver continues to retry (until retry count is exhausted) upon IOException responses from REST operations.

In the exception hander for IOExceptions at

hadoop/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsRestOperation.java

Lines 174 to 197 in 65f60e5

	if (ex instanceof UnknownHostException) {
	LOG.warn(String.format("Unknown host name: %s. Retrying to resolve the host name...", httpOperation.getUrl().getHost()));
	}
	if (LOG.isDebugEnabled()) {
	if (httpOperation != null) {
	LOG.debug("HttpRequestFailure: " + httpOperation.toString(), ex);
	} else {
	LOG.debug("HttpRequestFailure: " + method + "," + url, ex);
	}
	}
	if (!client.getRetryPolicy().shouldRetry(retryCount, -1)) {
	throw new InvalidAbfsRestOperationException(ex);
	}
	// once HttpException is thrown by AzureADAuthenticator,
	// it indicates the policy in AzureADAuthenticator determined
	// retry is not needed
	if (ex instanceof HttpException) {
	throw new AbfsRestOperationException((HttpException) ex);
	}
	return false;

, there is no way exit out of the retry loop by re-throwing an exception unless one of the following conditions have been met:

• The retry limit was hit
• An HttpException was encountered

From an org.apache.hadoop.fs.azurebfs.extensions.CustomTokenProviderAdaptee or org.apache.hadoop.fs.azurebfs.extensions.CustomDelegationTokenManager implementation, there is no way to create an org.apache.hadoop.fs.azurebfs.oauth2.AzureADAuthenticator.HttpException since the constructor is package private.

To solve this issue, access to org.apache.hadoop.fs.azurebfs.oauth2.AzureADAuthenticator.HttpException needs to be set to that custom implementations can use it.

This patch changes the org.apache.hadoop.fs.azurebfs.oauth2.AzureADAuthenticator.HttpException constructor to private rather than package private.

[rlevas]

FYI @steveloughran , @DadanielZ

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	47	Docker mode activated.
		_ Prechecks _
+1	dupname	0	No case conflicting files found.
+1	@author	0	The patch does not contain any @author tags.
-1	test4tests	0	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
		_ trunk Compile Tests _
+1	mvninstall	1147	trunk passed
+1	compile	30	trunk passed
+1	checkstyle	21	trunk passed
+1	mvnsite	32	trunk passed
+1	shadedclient	776	branch has no errors when building and testing our client artifacts.
+1	javadoc	24	trunk passed
0	spotbugs	52	Used deprecated FindBugs config; considering switching to SpotBugs.
+1	findbugs	51	trunk passed
		_ Patch Compile Tests _
+1	mvninstall	27	the patch passed
+1	compile	24	the patch passed
+1	javac	24	the patch passed
+1	checkstyle	16	the patch passed
+1	mvnsite	28	the patch passed
+1	whitespace	0	The patch has no whitespace issues.
+1	shadedclient	819	patch has no errors when building and testing our client artifacts.
+1	javadoc	20	the patch passed
+1	findbugs	56	the patch passed
		_ Other Tests _
+1	unit	71	hadoop-azure in the patch passed.
+1	asflicense	28	The patch does not generate ASF License warnings.
		3314

Subsystem	Report/Notes
Docker	Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-939/1/artifact/out/Dockerfile
GITHUB PR	#939
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux 110ff19fa896 4.4.0-143-generic #169~14.04.2-Ubuntu SMP Wed Feb 13 15:00:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / fcfe7a3
Default Java	1.8.0_212
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-939/1/testReport/
Max. process+thread count	305 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-939/1/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

[rlevas]

No unit tests added because the only change is the protection level on a class's constructor.

[DadanielZ]

LGTM.
Thanks for adding the fix @rlevas I verified this patch with my US west azure storage account, all tests passed:

XNS account:
Tests run: 41, Failures: 0, Errors: 0, Skipped: 0
Tests run: 392, Failures: 0, Errors: 0, Skipped: 23
Tests run: 190, Failures: 0, Errors: 0, Skipped: 23

non-xns account:
Tests run: 41, Failures: 0, Errors: 0, Skipped: 0
Tests run: 190, Failures: 0, Errors: 0, Skipped: 15
Tests run: 392, Failures: 1, Errors: 0, Skipped: 207

[DadanielZ]

@steveloughran @ajfabbri could you help to review this change?

[rlevas]

@DadanielZ , Thanks for the review and help to push this along.

[steveloughran]

committed into trunk, thanks

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	0	Docker mode activated.
-1	patch	14	#939 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.

Subsystem	Report/Notes
GITHUB PR	#939
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-939/2/console
versions	git=1.9.1
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.