HADOOP-18883. [ABFS]: Expect-100 JDK bug resolution: prevent multiple… #6484

saxenapranav · 2024-01-22T13:45:51Z

Backport of 7dc166d
Trunk PR: #6022

Description:

Address JDK bug JDK-8314978 related to handling of HTTP 100 responses.

https://bugs.openjdk.org/browse/JDK-8314978

In the AbfsHttpOperation, after sendRequest() we call processResponse() method from AbfsRestOperation.
Even if the conn.getOutputStream() fails due to expect-100 error, we consume the exception and let the code go ahead. This may call getHeaderField() / getHeaderFields() / getHeaderFieldLong() after getOutputStream() has failed. These invocation all lead to server calls.

This commit aims to prevent this.
If connection.getOutputStream() fails due to an Expect-100 error, the ABFS client does not invoke getHeaderField(), getHeaderFields(), getHeaderFieldLong() or getInputStream().

getResponseCode() is safe as on the failure it sets the responseCode variable in HttpUrlConnection object.

… server calls (apache#6022) Address JDK bug JDK-8314978 related to handling of HTTP 100 responses. https://bugs.openjdk.org/browse/JDK-8314978 In the AbfsHttpOperation, after sendRequest() we call processResponse() method from AbfsRestOperation. Even if the conn.getOutputStream() fails due to expect-100 error, we consume the exception and let the code go ahead. This may call getHeaderField() / getHeaderFields() / getHeaderFieldLong() after getOutputStream() has failed. These invocation all lead to server calls. This commit aims to prevent this. If connection.getOutputStream() fails due to an Expect-100 error, the ABFS client does not invoke getHeaderField(), getHeaderFields(), getHeaderFieldLong() or getInputStream(). getResponseCode() is safe as on the failure it sets the responseCode variable in HttpUrlConnection object. Contributed by Pranav Saxena

hadoop-yetus · 2024-01-22T16:19:10Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	10m 17s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 3 new or modified test files.
			_ branch-3.3 Compile Tests _
+1 💚	mvninstall	50m 35s		branch-3.3 passed
+1 💚	compile	0m 37s		branch-3.3 passed
+1 💚	checkstyle	0m 29s		branch-3.3 passed
+1 💚	mvnsite	0m 42s		branch-3.3 passed
+1 💚	javadoc	0m 33s		branch-3.3 passed
+1 💚	spotbugs	1m 8s		branch-3.3 passed
+1 💚	shadedclient	40m 8s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
-1 ❌	mvninstall	0m 30s	/patch-mvninstall-hadoop-tools_hadoop-azure.txt	hadoop-azure in the patch failed.
-1 ❌	compile	0m 30s	/patch-compile-hadoop-tools_hadoop-azure.txt	hadoop-azure in the patch failed.
-1 ❌	javac	0m 30s	/patch-compile-hadoop-tools_hadoop-azure.txt	hadoop-azure in the patch failed.
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 18s	/results-checkstyle-hadoop-tools_hadoop-azure.txt	hadoop-tools/hadoop-azure: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
-1 ❌	mvnsite	0m 31s	/patch-mvnsite-hadoop-tools_hadoop-azure.txt	hadoop-azure in the patch failed.
+1 💚	javadoc	0m 21s		the patch passed
-1 ❌	spotbugs	0m 29s	/patch-spotbugs-hadoop-tools_hadoop-azure.txt	hadoop-azure in the patch failed.
+1 💚	shadedclient	41m 22s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	0m 34s	/patch-unit-hadoop-tools_hadoop-azure.txt	hadoop-azure in the patch failed.
+1 💚	asflicense	0m 35s		The patch does not generate ASF License warnings.
		151m 26s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6484/1/artifact/out/Dockerfile
GITHUB PR	#6484
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux d97a07336b03 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-3.3 / `eba331a`
Default Java	Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~18.04-b09
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6484/1/testReport/
Max. process+thread count	606 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6484/1/console
versions	git=2.17.1 maven=3.6.0 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

saxenapranav · 2024-01-23T05:13:46Z

:::: AGGREGATED TEST RESULT ::::

HNS-OAuth

[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] TestAccountConfiguration.testConfigPropNotFound:386->testMissingConfigKey:399 Expected a org.apache.hadoop.fs.azurebfs.contracts.exceptions.TokenAccessProviderException to be thrown, but got the result: : "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider"
[INFO]
[ERROR] Tests run: 141, Failures: 1, Errors: 0, Skipped: 5
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemLease.testAcquireRetry:329 » TestTimedOut test timed o...
[INFO]
[ERROR] Tests run: 580, Failures: 0, Errors: 1, Skipped: 26
[INFO] Results:
[INFO]
[WARNING] Tests run: 339, Failures: 0, Errors: 0, Skipped: 41

HNS-SharedKey

[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] TestAccountConfiguration.testConfigPropNotFound:386->testMissingConfigKey:399 Expected a org.apache.hadoop.fs.azurebfs.contracts.exceptions.TokenAccessProviderException to be thrown, but got the result: : "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider"
[INFO]
[ERROR] Tests run: 141, Failures: 1, Errors: 0, Skipped: 5
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemLease.testAcquireRetry:329 » TestTimedOut test timed o...
[INFO]
[ERROR] Tests run: 580, Failures: 0, Errors: 1, Skipped: 26
[INFO] Results:
[INFO]
[WARNING] Tests run: 339, Failures: 0, Errors: 0, Skipped: 41

NonHNS-SharedKey

[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] TestAccountConfiguration.testConfigPropNotFound:386->testMissingConfigKey:399 Expected a org.apache.hadoop.fs.azurebfs.contracts.exceptions.TokenAccessProviderException to be thrown, but got the result: : "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider"
[INFO]
[ERROR] Tests run: 141, Failures: 1, Errors: 0, Skipped: 11
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemLease.testAcquireRetry:338 » TestTimedOut test timed o...
[INFO]
[ERROR] Tests run: 581, Failures: 0, Errors: 1, Skipped: 269
[INFO] Results:
[INFO]
[WARNING] Tests run: 339, Failures: 0, Errors: 0, Skipped: 44

AppendBlob-HNS-OAuth

[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] TestAccountConfiguration.testConfigPropNotFound:386->testMissingConfigKey:399 Expected a org.apache.hadoop.fs.azurebfs.contracts.exceptions.TokenAccessProviderException to be thrown, but got the result: : "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider"
[INFO]
[ERROR] Tests run: 141, Failures: 1, Errors: 0, Skipped: 5
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] ITestAzureBlobFileSystemRandomRead.testSkipBounds:218->Assert.assertTrue:42->Assert.fail:89 There should not be any network I/O (elapsedTimeMs=106).
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemLease.testAcquireRetry:329 » TestTimedOut test timed o...
[INFO]
[ERROR] Tests run: 580, Failures: 1, Errors: 1, Skipped: 26
[INFO] Results:
[INFO]
[WARNING] Tests run: 339, Failures: 0, Errors: 0, Skipped: 41

Time taken: 48 mins 45 secs.
azureuser@Hadoop-VM-EAST2:/hadoop/hadoop-tools/hadoop-azure$
azureuser@Hadoop-VM-EAST2:/hadoop/hadoop-tools/hadoop-azure$
azureuser@Hadoop-VM-EAST2:~/hadoop/hadoop-tools/hadoop-azure$ git log
commit baafc30 (HEAD -> branch-3.3_expect100resolution, origin/branch-3.3_expect100resolution)
Author: Pranav Saxena <>
Date: Mon Jan 22 20:11:33 2024 -0800

compile issue

saxenapranav · 2024-01-23T05:16:32Z

@steveloughran , requesting you to kindly review the backport PR please. Thank you so much.

hadoop-yetus · 2024-01-23T06:36:34Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 44s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 3 new or modified test files.
			_ branch-3.3 Compile Tests _
+1 💚	mvninstall	50m 35s		branch-3.3 passed
+1 💚	compile	0m 36s		branch-3.3 passed
+1 💚	checkstyle	0m 29s		branch-3.3 passed
+1 💚	mvnsite	0m 42s		branch-3.3 passed
+1 💚	javadoc	0m 33s		branch-3.3 passed
+1 💚	spotbugs	1m 10s		branch-3.3 passed
+1 💚	shadedclient	39m 54s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 37s		the patch passed
+1 💚	compile	0m 29s		the patch passed
+1 💚	javac	0m 29s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 18s	/results-checkstyle-hadoop-tools_hadoop-azure.txt	hadoop-tools/hadoop-azure: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
+1 💚	mvnsite	0m 32s		the patch passed
+1 💚	javadoc	0m 22s		the patch passed
+1 💚	spotbugs	1m 6s		the patch passed
+1 💚	shadedclient	39m 39s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	2m 16s		hadoop-azure in the patch passed.
+1 💚	asflicense	0m 34s		The patch does not generate ASF License warnings.
		143m 59s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6484/2/artifact/out/Dockerfile
GITHUB PR	#6484
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 979ed9189574 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-3.3 / `baafc30`
Default Java	Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~18.04-b09
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6484/2/testReport/
Max. process+thread count	527 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6484/2/console
versions	git=2.17.1 maven=3.6.0 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2024-01-23T09:13:49Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 46s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 3 new or modified test files.
			_ branch-3.3 Compile Tests _
-1 ❌	mvninstall	50m 3s	/branch-mvninstall-root.txt	root in branch-3.3 failed.
+1 💚	compile	0m 36s		branch-3.3 passed
+1 💚	checkstyle	0m 29s		branch-3.3 passed
+1 💚	mvnsite	0m 42s		branch-3.3 passed
+1 💚	javadoc	0m 34s		branch-3.3 passed
+1 💚	spotbugs	1m 9s		branch-3.3 passed
+1 💚	shadedclient	40m 17s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 37s		the patch passed
+1 💚	compile	0m 29s		the patch passed
+1 💚	javac	0m 29s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 18s		the patch passed
+1 💚	mvnsite	0m 33s		the patch passed
+1 💚	javadoc	0m 22s		the patch passed
+1 💚	spotbugs	1m 5s		the patch passed
+1 💚	shadedclient	39m 49s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	2m 17s		hadoop-azure in the patch passed.
+1 💚	asflicense	0m 36s		The patch does not generate ASF License warnings.
		144m 15s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6484/3/artifact/out/Dockerfile
GITHUB PR	#6484
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux a673466fa107 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-3.3 / `f29780f`
Default Java	Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~18.04-b09
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6484/3/testReport/
Max. process+thread count	528 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6484/3/console
versions	git=2.17.1 maven=3.6.0 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

steveloughran · 2024-01-25T22:23:07Z

build failures are networking and unrelated to change

[INFO] Finished at: 2024-01-23T07:44:05Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.3.1:exec (check-jar-contents) on project hadoop-client-check-invariants: Execution check-jar-contents of goal org.codehaus.mojo:exec-maven-plugin:1.3.1:exec failed: Plugin org.codehaus.mojo:exec-maven-plugin:1.3.1 or one of its dependencies could not be resolved: Could not transfer artifact asm:asm-tree:jar:3.3.1 from/to central (https://repo.maven.apache.org/maven2): Connection reset -> [Help 1]
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.3.1:exec (check-jar-contents) on project hadoop-client-check-test-invariants: Execution check-jar-contents of goal org.codehaus.mojo:exec-maven-plugin:1.3.1:exec failed: Plugin org.codehaus.mojo:exec-maven-plugin:1.3.1 or one of its dependencies could not be resolved: Failure to transfer asm:asm-tree:jar:3.3.1 from https://repo.maven.apache.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced. Original error: Could not transfer artifact asm:asm-tree:jar:3.3.1 from/to central (https://repo.maven.apache.org/maven2): Connection reset -> [Help 1]
[ERROR]

steveloughran

+1

steveloughran · 2024-01-25T22:24:36Z

Merged. @saxenapranav you need to do a PR and test for branch-3.4 now, so 3.4.1 will have the fix

saxenapranav · 2024-01-29T07:02:12Z

@steveloughran , thank you very much for the review and the merge. I have created a pr #6511 on branch-3.4

Thank you very much.

compile issue

baafc30

saxenapranav marked this pull request as ready for review January 23, 2024 05:14

saxenapranav mentioned this pull request Jan 23, 2024

HADOOP-18883. [ABFS]: Expect-100 JDK bug resolution: prevent multiple server calls #6022

Merged

checkstyles: duplicate import

f29780f

steveloughran approved these changes Jan 25, 2024

View reviewed changes

steveloughran merged commit c728580 into apache:branch-3.3 Jan 25, 2024
0 of 3 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HADOOP-18883. [ABFS]: Expect-100 JDK bug resolution: prevent multiple… #6484

HADOOP-18883. [ABFS]: Expect-100 JDK bug resolution: prevent multiple… #6484

saxenapranav commented Jan 22, 2024

hadoop-yetus commented Jan 22, 2024

saxenapranav commented Jan 23, 2024

saxenapranav commented Jan 23, 2024

hadoop-yetus commented Jan 23, 2024

hadoop-yetus commented Jan 23, 2024

steveloughran commented Jan 25, 2024

steveloughran left a comment

steveloughran commented Jan 25, 2024

saxenapranav commented Jan 29, 2024

HADOOP-18883. [ABFS]: Expect-100 JDK bug resolution: prevent multiple… #6484

HADOOP-18883. [ABFS]: Expect-100 JDK bug resolution: prevent multiple… #6484

Conversation

saxenapranav commented Jan 22, 2024

hadoop-yetus commented Jan 22, 2024

saxenapranav commented Jan 23, 2024

HNS-OAuth

HNS-SharedKey

NonHNS-SharedKey

AppendBlob-HNS-OAuth

saxenapranav commented Jan 23, 2024

hadoop-yetus commented Jan 23, 2024

hadoop-yetus commented Jan 23, 2024

steveloughran commented Jan 25, 2024

steveloughran left a comment

Choose a reason for hiding this comment

steveloughran commented Jan 25, 2024

saxenapranav commented Jan 29, 2024