HADOOP-17259. Allow SSLFactory fallback to input config if ssl-*.xml … #2301

xiaoyuyao · 2020-09-11T22:51:49Z

…fail to load from classpath.

Issue Link
https://issues.apache.org/jira/browse/HADOOP-17259

Description
Some applications like Tez does not have ssl-client.xml and ssl-server.xml in classpath. Instead, it directly pass the parsed SSL configuration as the input configuration object. This ticket is opened to allow this case. TEZ-4096 attempts to solve this issue but but take a different approach which may not work in existing Hadoop clients that use SSLFactory from hadoop-common.

This ticket is opened to allow SSLFactory to accept SSL config from input if SSL config can't be loaded from classpath.

How is the patch tested:
Added UT.

…fail to load from classpath.

hadoop-yetus · 2020-09-12T03:55:58Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	47m 35s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ trunk Compile Tests _
+1 💚	mvninstall	40m 18s	trunk passed
+1 💚	compile	36m 8s	trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	compile	31m 4s	trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+1 💚	checkstyle	1m 15s	trunk passed
+1 💚	mvnsite	2m 9s	trunk passed
+1 💚	shadedclient	21m 59s	branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 37s	trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	javadoc	1m 51s	trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+0 🆗	spotbugs	4m 3s	Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	3m 55s	trunk passed
		_ Patch Compile Tests _
+1 💚	mvninstall	1m 41s	the patch passed
+1 💚	compile	36m 46s	the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	javac	36m 46s	the patch passed
+1 💚	compile	30m 57s	the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+1 💚	javac	30m 57s	the patch passed
+1 💚	checkstyle	1m 8s	hadoop-common-project/hadoop-common: The patch generated 0 new + 25 unchanged - 1 fixed = 25 total (was 26)
+1 💚	mvnsite	2m 3s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	shadedclient	18m 15s	patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 39s	the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	javadoc	1m 57s	the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+1 💚	findbugs	3m 49s	the patch passed
		_ Other Tests _
+1 💚	unit	14m 28s	hadoop-common in the patch passed.
+1 💚	asflicense	1m 7s	The patch does not generate ASF License warnings.
		302m 1s

Subsystem	Report/Notes
Docker	ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2301/1/artifact/out/Dockerfile
GITHUB PR	#2301
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux 0f2cab5d2439 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `958cab8`
Default Java	Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2301/1/testReport/
Max. process+thread count	3260 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2301/1/console
versions	git=2.17.1 maven=3.6.0 findbugs=4.0.6
Powered by	Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2020-09-12T03:57:08Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	45m 25s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ trunk Compile Tests _
+1 💚	mvninstall	43m 1s	trunk passed
+1 💚	compile	36m 43s	trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	compile	30m 13s	trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+1 💚	checkstyle	1m 12s	trunk passed
+1 💚	mvnsite	2m 23s	trunk passed
+1 💚	shadedclient	21m 50s	branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 42s	trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	javadoc	2m 17s	trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+0 🆗	spotbugs	3m 42s	Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	3m 38s	trunk passed
		_ Patch Compile Tests _
+1 💚	mvninstall	1m 39s	the patch passed
+1 💚	compile	35m 54s	the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	javac	35m 54s	the patch passed
+1 💚	compile	30m 48s	the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+1 💚	javac	30m 48s	the patch passed
+1 💚	checkstyle	1m 14s	hadoop-common-project/hadoop-common: The patch generated 0 new + 25 unchanged - 1 fixed = 25 total (was 26)
+1 💚	mvnsite	2m 1s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	shadedclient	17m 33s	patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 38s	the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	javadoc	1m 56s	the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+1 💚	findbugs	4m 11s	the patch passed
		_ Other Tests _
-1 ❌	unit	14m 25s	hadoop-common in the patch passed.
+1 💚	asflicense	0m 57s	The patch does not generate ASF License warnings.
		300m 48s

Reason	Tests
Failed junit tests	hadoop.ipc.TestIPC

Subsystem	Report/Notes
Docker	ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2301/2/artifact/out/Dockerfile
GITHUB PR	#2301
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux d5e99a2866d3 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `958cab8`
Default Java	Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
unit	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2301/2/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2301/2/testReport/
Max. process+thread count	2921 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2301/2/console
versions	git=2.17.1 maven=3.6.0 findbugs=4.0.6
Powered by	Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

steveloughran · 2020-09-21T18:53:34Z

LGTM: +1 .. commit at your leisure

xiaoyuyao · 2020-09-21T19:40:51Z

Thanks @steveloughran for the review. I'll merge it shortly.

apache#2301)

apache#2301) (cherry picked from commit 6b5d9e2) Change-Id: I734b07d5674af7b45592edf683cff1ec9966a2e5

xiaoyuyao added 2 commits September 11, 2020 15:49

HADOOP-17259. Allow SSLFactory fallback to input config if ssl-*.xml …

6a83ed0

…fail to load from classpath.

fix the accidental change of test class name.

2f74d58

xiaoyuyao merged commit 6b5d9e2 into apache:trunk Sep 21, 2020

bilaharith pushed a commit to bilaharith/hadoop that referenced this pull request Sep 27, 2020

HADOOP-17259. Allow SSLFactory fallback to input config if ssl-*.xml … (

6100b23

apache#2301)

jojochuang pushed a commit to jojochuang/hadoop that referenced this pull request May 23, 2023

HADOOP-17259. Allow SSLFactory fallback to input config if ssl-*.xml … (

d95cf39

apache#2301) (cherry picked from commit 6b5d9e2) Change-Id: I734b07d5674af7b45592edf683cff1ec9966a2e5

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HADOOP-17259. Allow SSLFactory fallback to input config if ssl-*.xml … #2301

HADOOP-17259. Allow SSLFactory fallback to input config if ssl-*.xml … #2301

xiaoyuyao commented Sep 11, 2020 •

edited

hadoop-yetus commented Sep 12, 2020

hadoop-yetus commented Sep 12, 2020

steveloughran commented Sep 21, 2020

xiaoyuyao commented Sep 21, 2020

HADOOP-17259. Allow SSLFactory fallback to input config if ssl-*.xml … #2301

HADOOP-17259. Allow SSLFactory fallback to input config if ssl-*.xml … #2301

Conversation

xiaoyuyao commented Sep 11, 2020 • edited

hadoop-yetus commented Sep 12, 2020

hadoop-yetus commented Sep 12, 2020

steveloughran commented Sep 21, 2020

xiaoyuyao commented Sep 21, 2020

xiaoyuyao commented Sep 11, 2020 •

edited