HADOOP-18203. Backport HADOOP-15033 to branch-2.10. #4174

mccormickt12 · 2022-04-14T22:41:35Z

Description of PR

How was this patch tested?

For code changes:

Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

…animal-sniffer-maven-plugin to compile with Java 9. (cherry picked from commit d2d8f4a)

hadoop-yetus · 2022-04-15T00:34:06Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	8m 21s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ branch-2.10 Compile Tests _
+0 🆗	mvndep	3m 54s		Maven dependency ordering for branch
+1 💚	mvninstall	14m 6s		branch-2.10 passed
+1 💚	compile	13m 12s		branch-2.10 passed with JDK Azul Systems, Inc.-1.7.0_262-b10
+1 💚	compile	10m 48s		branch-2.10 passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07
+1 💚	checkstyle	2m 8s		branch-2.10 passed
+1 💚	mvnsite	1m 50s		branch-2.10 passed
+1 💚	javadoc	1m 54s		branch-2.10 passed with JDK Azul Systems, Inc.-1.7.0_262-b10
+1 💚	javadoc	1m 37s		branch-2.10 passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07
+0 🆗	spotbugs	0m 59s		branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
-1 ❌	spotbugs	2m 1s	/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html	hadoop-common-project/hadoop-common in branch-2.10 has 2 extant spotbugs warnings.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 23s		Maven dependency ordering for patch
+1 💚	mvninstall	0m 59s		the patch passed
+1 💚	compile	12m 21s		the patch passed with JDK Azul Systems, Inc.-1.7.0_262-b10
+1 💚	javac	12m 21s		the patch passed
+1 💚	compile	10m 43s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07
+1 💚	javac	10m 43s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	2m 33s		the patch passed
+1 💚	mvnsite	1m 53s		the patch passed
+1 💚	xml	0m 2s		The patch has no ill-formed XML file.
+1 💚	javadoc	1m 49s		the patch passed with JDK Azul Systems, Inc.-1.7.0_262-b10
+1 💚	javadoc	1m 38s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07
+0 🆗	spotbugs	0m 32s		hadoop-project has no data from spotbugs
			_ Other Tests _
+1 💚	unit	0m 30s		hadoop-project in the patch passed.
-1 ❌	unit	8m 41s	/patch-unit-hadoop-common-project_hadoop-common.txt	hadoop-common in the patch passed.
+1 💚	asflicense	0m 51s		The patch does not generate ASF License warnings.
		111m 42s

Reason	Tests
Failed junit tests	hadoop.io.compress.snappy.TestSnappyCompressorDecompressor
	hadoop.io.compress.TestCompressorDecompressor
	hadoop.fs.sftp.TestSFTPFileSystem

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4174/1/artifact/out/Dockerfile
GITHUB PR	#4174
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell xml
uname	Linux a2b07dff4b29 4.15.0-169-generic #177-Ubuntu SMP Thu Feb 3 10:50:38 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-2.10 / 2b468fa97456fc893a93c42fbe8ba1e0f2766fd3
Default Java	Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07
Multi-JDK versions	/usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_262-b10 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4174/1/testReport/
Max. process+thread count	1392 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-common-project/hadoop-common U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4174/1/console
versions	git=2.17.1 maven=3.6.0 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

Contributed by Dmitry Chuyko, (cherry picked from commit 2e0a451)

hadoop-yetus · 2022-04-19T00:13:48Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 48s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ branch-2.10 Compile Tests _
+0 🆗	mvndep	4m 1s		Maven dependency ordering for branch
+1 💚	mvninstall	14m 8s		branch-2.10 passed
+1 💚	compile	13m 19s		branch-2.10 passed with JDK Azul Systems, Inc.-1.7.0_262-b10
+1 💚	compile	11m 20s		branch-2.10 passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07
+1 💚	checkstyle	2m 31s		branch-2.10 passed
+1 💚	mvnsite	2m 34s		branch-2.10 passed
+1 💚	javadoc	2m 45s		branch-2.10 passed with JDK Azul Systems, Inc.-1.7.0_262-b10
+1 💚	javadoc	2m 26s		branch-2.10 passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07
+0 🆗	spotbugs	1m 24s		branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
-1 ❌	spotbugs	2m 24s	/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html	hadoop-common-project/hadoop-common in branch-2.10 has 2 extant spotbugs warnings.
-0 ⚠️	patch	2m 51s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 26s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 10s		the patch passed
+1 💚	compile	12m 34s		the patch passed with JDK Azul Systems, Inc.-1.7.0_262-b10
+1 💚	javac	12m 34s		the patch passed
+1 💚	compile	11m 23s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07
+1 💚	javac	11m 23s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	2m 29s		the patch passed
+1 💚	mvnsite	2m 39s		the patch passed
+1 💚	xml	0m 1s		The patch has no ill-formed XML file.
+1 💚	javadoc	2m 39s		the patch passed with JDK Azul Systems, Inc.-1.7.0_262-b10
+1 💚	javadoc	2m 24s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07
+0 🆗	spotbugs	0m 57s		hadoop-project has no data from spotbugs
			_ Other Tests _
+1 💚	unit	0m 56s		hadoop-project in the patch passed.
-1 ❌	unit	9m 14s	/patch-unit-hadoop-common-project_hadoop-common.txt	hadoop-common in the patch passed.
+1 💚	asflicense	1m 17s		The patch does not generate ASF License warnings.
		116m 27s

Reason	Tests
Failed junit tests	hadoop.io.compress.snappy.TestSnappyCompressorDecompressor
	hadoop.io.compress.TestCompressorDecompressor
	hadoop.fs.sftp.TestSFTPFileSystem

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4174/3/artifact/out/Dockerfile
GITHUB PR	#4174
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell xml
uname	Linux 720f48c419b4 4.15.0-169-generic #177-Ubuntu SMP Thu Feb 3 10:50:38 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-2.10 / `38d6a05`
Default Java	Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07
Multi-JDK versions	/usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_262-b10 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4174/3/testReport/
Max. process+thread count	1397 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-common-project/hadoop-common U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4174/3/console
versions	git=2.17.1 maven=3.6.0 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

iwasakims · 2022-04-20T06:26:24Z

hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java

+  private static final int JAVA_SPEC_VER = Math.max(8, Integer.parseInt(
+      System.getProperty("java.specification.version").split("\\.")[0]));


JAVA_SPEC_VER is set to 8 even on Java 7. Can we cover Java 7 here? If not, we should have a comment for justification at least.

iwasakims · 2022-04-20T06:27:35Z

hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestShell.java

+  public void testIsJavaVersionAtLeast() {
+    assertTrue(Shell.isJavaVersionAtLeast(8));
+  }


This test succeeds on Java 7 since the Shell.isJavaVersionAtLeast(8) returns true even on Java 7. We should have a comment for justification at least.

HADOOP-15133. [JDK9] Ignore com.sun.javadoc.* and com.sun.tools.* in …

728e8af

…animal-sniffer-maven-plugin to compile with Java 9. (cherry picked from commit d2d8f4a)

HADOOP-15033. Use java.util.zip.CRC32C for Java 9 and above

2f75bb7

Contributed by Dmitry Chuyko, (cherry picked from commit 2e0a451)

mccormickt12 force-pushed the bp-HADOOP-15133-and-HADOOP-15033 branch from 2b468fa to 2f75bb7 Compare April 15, 2022 16:45

Merge branch 'branch-2.10' into bp-HADOOP-15133-and-HADOOP-15033

38d6a05

iwasakims changed the title ~~Bp hadoop 15133 and hadoop 15033~~ HADOOP-18203. Backport HADOOP-15033 to branch-2.10. Apr 19, 2022

iwasakims reviewed Apr 20, 2022

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HADOOP-18203. Backport HADOOP-15033 to branch-2.10. #4174

HADOOP-18203. Backport HADOOP-15033 to branch-2.10. #4174

mccormickt12 commented Apr 14, 2022

hadoop-yetus commented Apr 15, 2022

hadoop-yetus commented Apr 19, 2022

iwasakims Apr 20, 2022

iwasakims Apr 20, 2022

		private static final int JAVA_SPEC_VER = Math.max(8, Integer.parseInt(
		System.getProperty("java.specification.version").split("\\.")[0]));

HADOOP-18203. Backport HADOOP-15033 to branch-2.10. #4174

Are you sure you want to change the base?

HADOOP-18203. Backport HADOOP-15033 to branch-2.10. #4174

Conversation

mccormickt12 commented Apr 14, 2022

Description of PR

How was this patch tested?

For code changes:

hadoop-yetus commented Apr 15, 2022

hadoop-yetus commented Apr 19, 2022

iwasakims Apr 20, 2022

Choose a reason for hiding this comment

iwasakims Apr 20, 2022

Choose a reason for hiding this comment