HADOOP-17653. Do not use guava's Files.createTempDir(). #2945

jojochuang · 2021-04-21T12:22:26Z

Guava's Files.createTempDir() is deprecated. The recommendation is to use JDK's java.nio.file.Files.createTempDirectory() instead.

Some of the code follows the internal implementation of Guava createTempDir() https://github.com/google/guava/blob/master/guava/src/com/google/common/io/Files.java#L423

Change-Id: I00b0836e1f56f58ed23ffbbf6e8e9ff5409a65bf

virajjasani

Left one minor comment, else looks good overall

...main/java/org/apache/hadoop/hdfs/server/federation/store/driver/impl/StateStoreFileImpl.java

Change-Id: Ifd49e036649fe9d1dd1a572b70b3ab9e01acf17a

steveloughran

LGTM; minor tweaks of imports suggested.

side issue: is there a way we could block curator.shaded imports? They're clearly accidental IDE ones...having them rejected in yetus &c would be great

...g/apache/hadoop/hdfs/server/common/blockaliasmap/impl/TestInMemoryLevelDBAliasMapClient.java

...a/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/TestLevelDbMockAliasMapClient.java

virajjasani · 2021-04-22T11:17:48Z

side issue: is there a way we could block curator.shaded imports? They're clearly accidental IDE ones...having them rejected in yetus &c would be great

@steveloughran This is good one. I can give it a try in a follow-up Jira?

aajisaka · 2021-04-22T16:47:06Z

side issue: is there a way we could block curator.shaded imports? They're clearly accidental IDE ones...having them rejected in yetus &c would be great

We can warn the imports by updating the illegal import section in checkstyle.yml.

hadoop/hadoop-build-tools/src/main/resources/checkstyle/checkstyle.xml

Lines 122 to 126 in eac7aef

    
           <module name="IllegalImport"> 
        
             <property name="regexp" value="true"/> 
        
             <property name="illegalPkgs" value="sun, com\.google\.common"/> 
        
             <property name="illegalClasses" value="^org\.apache\.hadoop\.thirdparty\.com\.google\.common\.io\.BaseEncoding, ^org\.apache\.hadoop\.thirdparty\.com\.google\.common\.base\.(Optional|Function|Predicate|Supplier), ^org\.apache\.hadoop\.thirdparty\.com\.google\.common\.collect\.(ImmutableListMultimap)"/> 
        
           </module>

virajjasani · 2021-04-22T16:55:39Z

side issue: is there a way we could block curator.shaded imports? They're clearly accidental IDE ones...having them rejected in yetus &c would be great

We can warn the imports by updating the illegal import section in checkstyle.yml.

hadoop/hadoop-build-tools/src/main/resources/checkstyle/checkstyle.xml

Lines 122 to 126 in eac7aef

<module name="IllegalImport">

<property name="regexp" value="true"/>

<property name="illegalPkgs" value="sun, com\.google\.common"/>

<property name="illegalClasses" value="^org\.apache\.hadoop\.thirdparty\.com\.google\.common\.io\.BaseEncoding, ^org\.apache\.hadoop\.thirdparty\.com\.google\.common\.base\.(Optional|Function|Predicate|Supplier), ^org\.apache\.hadoop\.thirdparty\.com\.google\.common\.collect\.(ImmutableListMultimap)"/>

</module>

@aajisaka Although this is good option, but code can still be checked-in with this import. In addition to this change, we can also use restrict-imports-enforcer-rule dependency to restrict import such that if used, we will see mvn build failure.

If you are fine, I can create PR with this enforcer in parallel?

hadoop-yetus · 2021-04-22T18:39:08Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 1s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 6 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	15m 48s		Maven dependency ordering for branch
+1 💚	mvninstall	24m 48s		trunk passed
+1 💚	compile	22m 23s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	compile	19m 10s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	checkstyle	4m 4s		trunk passed
+1 💚	mvnsite	5m 19s		trunk passed
+1 💚	javadoc	4m 15s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	4m 49s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	10m 13s		trunk passed
+1 💚	shadedclient	17m 5s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 22s		Maven dependency ordering for patch
+1 💚	mvninstall	4m 2s		the patch passed
+1 💚	compile	21m 43s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javac	21m 43s		the patch passed
+1 💚	compile	19m 1s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	javac	19m 1s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	3m 56s		the patch passed
+1 💚	mvnsite	5m 17s		the patch passed
+1 💚	javadoc	4m 16s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	4m 51s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	11m 28s		the patch passed
+1 💚	shadedclient	16m 51s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	344m 8s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
-1 ❌	unit	104m 41s	/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt	hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚	unit	24m 16s		hadoop-yarn-server-nodemanager in the patch passed.
-1 ❌	unit	26m 50s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt	hadoop-hdfs-rbf in the patch passed.
+1 💚	unit	21m 36s		hadoop-yarn-services-core in the patch passed.
+1 💚	unit	1m 5s		hadoop-yarn-csi in the patch passed.
+1 💚	asflicense	1m 6s		The patch does not generate ASF License warnings.
		749m 11s

Reason	Tests
Failed junit tests	hadoop.hdfs.qjournal.server.TestJournalNodeRespectsBindHostKeys
	hadoop.hdfs.server.datanode.TestBlockScanner
	hadoop.hdfs.TestRollingUpgrade
	hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
	hadoop.hdfs.server.namenode.TestFileTruncate
	hadoop.hdfs.TestViewDistributedFileSystem
	hadoop.hdfs.server.namenode.ha.TestBootstrapStandby
	hadoop.hdfs.TestPersistBlocks
	hadoop.hdfs.server.namenode.ha.TestEditLogTailer
	hadoop.hdfs.TestDFSShell
	hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots
	hadoop.hdfs.server.datanode.TestIncrementalBrVariations
	hadoop.hdfs.server.datanode.fsdataset.impl.TestFsVolumeList
	hadoop.hdfs.server.namenode.TestDecommissioningStatusWithBackoffMonitor
	hadoop.hdfs.server.namenode.TestDecommissioningStatus
	hadoop.yarn.server.resourcemanager.monitor.invariants.TestMetricsInvariantChecker
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerAsyncScheduling
	hadoop.yarn.server.resourcemanager.TestRMRestart
	hadoop.hdfs.server.federation.router.TestRouterFederationRename
	hadoop.hdfs.server.federation.router.TestRouterRpc
	hadoop.hdfs.server.federation.router.TestRouterFaultTolerant

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2945/2/artifact/out/Dockerfile
GITHUB PR	#2945
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 72c7ae521028 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `fa71444`
Default Java	Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2945/2/testReport/
Max. process+thread count	2183 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-hdfs-project/hadoop-hdfs-rbf hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2945/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

aajisaka · 2021-04-23T01:37:10Z

If you are fine, I can create PR with this enforcer in parallel?

Yes. Thank you!

virajjasani · 2021-04-23T06:31:39Z

If you are fine, I can create PR with this enforcer in parallel?

Yes. Thank you!

Thanks @aajisaka. I have the patch ready and done local testing to verify this as well. Will create PR soon after this current PR gets merged (until then new PR's QA bot will keep failing to build in the presence of imports).

Change-Id: I478c4b7dbf9c31a5d4fa70e8a07a73a504c3cf17

steveloughran

+1 from me

hadoop-yetus · 2021-04-23T22:24:20Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 6s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 6 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	15m 25s		Maven dependency ordering for branch
+1 💚	mvninstall	27m 26s		trunk passed
+1 💚	compile	29m 29s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	compile	24m 28s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	checkstyle	5m 8s		trunk passed
+1 💚	mvnsite	6m 48s		trunk passed
+1 💚	javadoc	4m 58s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	5m 37s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	12m 24s		trunk passed
+1 💚	shadedclient	18m 12s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 21s		Maven dependency ordering for patch
+1 💚	mvninstall	4m 4s		the patch passed
+1 💚	compile	21m 42s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javac	21m 42s		the patch passed
+1 💚	compile	18m 59s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	javac	18m 59s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	3m 55s		the patch passed
+1 💚	mvnsite	5m 23s		the patch passed
+1 💚	javadoc	4m 13s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	4m 49s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	11m 28s		the patch passed
+1 💚	shadedclient	17m 9s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	375m 32s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
-1 ❌	unit	104m 55s	/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt	hadoop-yarn-server-resourcemanager in the patch passed.
-1 ❌	unit	22m 32s	/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt	hadoop-yarn-server-nodemanager in the patch passed.
-1 ❌	unit	23m 30s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt	hadoop-hdfs-rbf in the patch passed.
+1 💚	unit	20m 36s		hadoop-yarn-services-core in the patch passed.
+1 💚	unit	0m 53s		hadoop-yarn-csi in the patch passed.
+1 💚	asflicense	0m 56s		The patch does not generate ASF License warnings.
		796m 54s

Reason	Tests
Failed junit tests	hadoop.hdfs.qjournal.server.TestJournalNodeRespectsBindHostKeys
	hadoop.hdfs.web.TestWebHdfsFileSystemContract
	hadoop.hdfs.server.datanode.TestBlockScanner
	hadoop.hdfs.TestRollingUpgrade
	hadoop.hdfs.server.namenode.ha.TestBootstrapStandby
	hadoop.hdfs.TestPersistBlocks
	hadoop.hdfs.server.namenode.ha.TestEditLogTailer
	hadoop.hdfs.TestDFSShell
	hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots
	hadoop.hdfs.TestSnapshotCommands
	hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl
	hadoop.hdfs.server.datanode.fsdataset.impl.TestFsVolumeList
	hadoop.hdfs.server.namenode.TestDecommissioningStatusWithBackoffMonitor
	hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes
	hadoop.hdfs.server.namenode.TestDecommissioningStatus
	hadoop.yarn.server.resourcemanager.monitor.invariants.TestMetricsInvariantChecker
	hadoop.yarn.server.nodemanager.amrmproxy.TestFederationInterceptor
	hadoop.hdfs.server.federation.router.TestRouterFederationRename

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2945/3/artifact/out/Dockerfile
GITHUB PR	#2945
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 2a7c3a3c21b3 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `72c14dd`
Default Java	Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2945/3/testReport/
Max. process+thread count	2176 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-hdfs-project/hadoop-hdfs-rbf hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2945/3/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

jojochuang · 2021-04-26T07:02:16Z

trigger the precommit build again just to be on the safe side.

hadoop-yetus · 2021-04-26T08:25:51Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 53s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 6 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	16m 6s		Maven dependency ordering for branch
-1 ❌	mvninstall	0m 27s	/branch-mvninstall-root.txt	root in trunk failed.
-1 ❌	compile	0m 16s	/branch-compile-root-jdkUbuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.txt	root in trunk failed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.
-1 ❌	compile	0m 14s	/branch-compile-root-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08.txt	root in trunk failed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08.
+1 💚	checkstyle	3m 50s		trunk passed
-1 ❌	mvnsite	0m 18s	/branch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in trunk failed.
-1 ❌	mvnsite	0m 24s	/branch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt	hadoop-yarn-server-resourcemanager in trunk failed.
-1 ❌	mvnsite	0m 15s	/branch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt	hadoop-yarn-server-nodemanager in trunk failed.
-1 ❌	mvnsite	0m 21s	/branch-mvnsite-hadoop-hdfs-project_hadoop-hdfs-rbf.txt	hadoop-hdfs-rbf in trunk failed.
-1 ❌	mvnsite	0m 16s	/branch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-services_hadoop-yarn-services-core.txt	hadoop-yarn-services-core in trunk failed.
-1 ❌	mvnsite	0m 21s	/branch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-csi.txt	hadoop-yarn-csi in trunk failed.
+1 💚	javadoc	2m 57s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	3m 36s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
-1 ❌	spotbugs	0m 11s	/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in trunk failed.
-1 ❌	spotbugs	0m 17s	/branch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt	hadoop-yarn-server-resourcemanager in trunk failed.
-1 ❌	spotbugs	0m 16s	/branch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt	hadoop-yarn-server-nodemanager in trunk failed.
-1 ❌	spotbugs	0m 14s	/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-rbf.txt	hadoop-hdfs-rbf in trunk failed.
-1 ❌	spotbugs	0m 14s	/branch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-services_hadoop-yarn-services-core.txt	hadoop-yarn-services-core in trunk failed.
-1 ❌	spotbugs	0m 14s	/branch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-csi.txt	hadoop-yarn-csi in trunk failed.
+1 💚	shadedclient	29m 26s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 20s		Maven dependency ordering for patch
-1 ❌	mvninstall	0m 11s	/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch failed.
-1 ❌	mvninstall	0m 15s	/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt	hadoop-yarn-server-resourcemanager in the patch failed.
-1 ❌	mvninstall	0m 13s	/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt	hadoop-yarn-server-nodemanager in the patch failed.
-1 ❌	mvninstall	0m 14s	/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs-rbf.txt	hadoop-hdfs-rbf in the patch failed.
-1 ❌	mvninstall	0m 12s	/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-services_hadoop-yarn-services-core.txt	hadoop-yarn-services-core in the patch failed.
-1 ❌	mvninstall	0m 13s	/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-csi.txt	hadoop-yarn-csi in the patch failed.
-1 ❌	compile	0m 14s	/patch-compile-root-jdkUbuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.txt	root in the patch failed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.
-1 ❌	javac	0m 14s	/patch-compile-root-jdkUbuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.txt	root in the patch failed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.
-1 ❌	compile	0m 14s	/patch-compile-root-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08.txt	root in the patch failed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08.
-1 ❌	javac	0m 14s	/patch-compile-root-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08.txt	root in the patch failed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08.
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	3m 44s		the patch passed
-1 ❌	mvnsite	0m 12s	/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch failed.
-1 ❌	mvnsite	0m 18s	/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt	hadoop-yarn-server-resourcemanager in the patch failed.
-1 ❌	mvnsite	0m 15s	/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt	hadoop-yarn-server-nodemanager in the patch failed.
-1 ❌	mvnsite	0m 16s	/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs-rbf.txt	hadoop-hdfs-rbf in the patch failed.
-1 ❌	mvnsite	0m 14s	/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-services_hadoop-yarn-services-core.txt	hadoop-yarn-services-core in the patch failed.
-1 ❌	mvnsite	0m 14s	/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-csi.txt	hadoop-yarn-csi in the patch failed.
+1 💚	javadoc	2m 54s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	3m 38s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
-1 ❌	spotbugs	0m 11s	/patch-spotbugs-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch failed.
-1 ❌	spotbugs	0m 17s	/patch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt	hadoop-yarn-server-resourcemanager in the patch failed.
-1 ❌	spotbugs	0m 15s	/patch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt	hadoop-yarn-server-nodemanager in the patch failed.
-1 ❌	spotbugs	0m 15s	/patch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-rbf.txt	hadoop-hdfs-rbf in the patch failed.
-1 ❌	spotbugs	0m 13s	/patch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-services_hadoop-yarn-services-core.txt	hadoop-yarn-services-core in the patch failed.
-1 ❌	spotbugs	0m 14s	/patch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-csi.txt	hadoop-yarn-csi in the patch failed.
+1 💚	shadedclient	33m 34s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	0m 15s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch failed.
-1 ❌	unit	0m 21s	/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt	hadoop-yarn-server-resourcemanager in the patch failed.
-1 ❌	unit	0m 13s	/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt	hadoop-yarn-server-nodemanager in the patch failed.
-1 ❌	unit	0m 15s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt	hadoop-hdfs-rbf in the patch failed.
-1 ❌	unit	0m 17s	/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-services_hadoop-yarn-services-core.txt	hadoop-yarn-services-core in the patch failed.
-1 ❌	unit	0m 15s	/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-csi.txt	hadoop-yarn-csi in the patch failed.
+1 💚	asflicense	0m 29s		The patch does not generate ASF License warnings.
		83m 6s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2945/4/artifact/out/Dockerfile
GITHUB PR	#2945
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 4ae5074940f9 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `72c14dd`
Default Java	Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2945/4/testReport/
Max. process+thread count	619 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-hdfs-project/hadoop-hdfs-rbf hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2945/4/console
versions	git=2.25.1 maven=3.6.3
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-04-26T22:03:27Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 51s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 6 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	16m 7s		Maven dependency ordering for branch
+1 💚	mvninstall	23m 7s		trunk passed
+1 💚	compile	22m 22s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	compile	19m 0s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	checkstyle	4m 3s		trunk passed
+1 💚	mvnsite	5m 22s		trunk passed
+1 💚	javadoc	4m 16s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	5m 2s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	10m 16s		trunk passed
+1 💚	shadedclient	16m 34s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 20s		Maven dependency ordering for patch
+1 💚	mvninstall	4m 2s		the patch passed
+1 💚	compile	21m 37s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javac	21m 37s		the patch passed
+1 💚	compile	19m 0s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	javac	19m 0s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	3m 54s		the patch passed
+1 💚	mvnsite	5m 20s		the patch passed
+1 💚	javadoc	4m 16s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	4m 53s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	11m 23s		the patch passed
+1 💚	shadedclient	17m 8s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	373m 55s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	unit	102m 7s		hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚	unit	22m 38s		hadoop-yarn-server-nodemanager in the patch passed.
-1 ❌	unit	23m 36s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt	hadoop-hdfs-rbf in the patch passed.
+1 💚	unit	20m 27s		hadoop-yarn-services-core in the patch passed.
+1 💚	unit	0m 53s		hadoop-yarn-csi in the patch passed.
+1 💚	asflicense	1m 2s		The patch does not generate ASF License warnings.
		768m 4s

Reason	Tests
Failed junit tests	hadoop.hdfs.qjournal.server.TestJournalNodeRespectsBindHostKeys
	hadoop.hdfs.server.namenode.TestFsck
	hadoop.hdfs.server.datanode.TestBlockScanner
	hadoop.hdfs.server.namenode.TestAddOverReplicatedStripedBlocks
	hadoop.fs.viewfs.TestViewFileSystemOverloadSchemeWithHdfsScheme
	hadoop.hdfs.TestViewDistributedFileSystemWithMountLinks
	hadoop.hdfs.server.namenode.ha.TestBootstrapStandby
	hadoop.hdfs.TestPersistBlocks
	hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer
	hadoop.hdfs.server.namenode.ha.TestEditLogTailer
	hadoop.hdfs.TestDFSShell
	hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots
	hadoop.hdfs.server.datanode.fsdataset.impl.TestFsVolumeList
	hadoop.fs.viewfs.TestViewFSOverloadSchemeWithMountTableConfigInHDFS
	hadoop.hdfs.server.namenode.TestDecommissioningStatusWithBackoffMonitor
	hadoop.hdfs.server.namenode.TestDecommissioningStatus
	hadoop.hdfs.server.federation.router.TestRouterFederationRename
	hadoop.hdfs.server.federation.router.TestRouterRpc

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2945/5/artifact/out/Dockerfile
GITHUB PR	#2945
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 2f4f83bb3c0a 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `72c14dd`
Default Java	Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2945/5/testReport/
Max. process+thread count	2166 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-hdfs-project/hadoop-hdfs-rbf hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2945/5/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

steveloughran · 2021-04-28T09:30:30Z

Do we know if any of the test failures are related or is this just highlighting all tests with issues because the patch spans the code?

jojochuang · 2021-04-30T02:00:11Z

TestRouterFederationRename failure looks caused by this PR. I'll check again.

aajisaka · 2021-05-02T01:37:37Z

TestRouterFederationRename failure looks caused by this PR.

Is it related? I applied the patch and passed the test locally.

aajisaka · 2021-05-02T01:39:28Z

The test has been fixed by #2954

aajisaka · 2021-05-02T02:10:58Z

Thank you @jojochuang, @virajjasani, and @steveloughran

Reviewed-by: Steve Loughran <stevel@apache.org> Signed-off-by: Akira Ajisaka <aajisaka@apache.org> (cherry picked from commit f1e1809)

virajjasani · 2021-05-02T06:41:27Z

Follow-up PR to restrict imports from org.apache.curator.shaded: #2969

Reviewed-by: Steve Loughran <stevel@apache.org> Signed-off-by: Akira Ajisaka <aajisaka@apache.org>

…pache#2945) Reviewed-by: Steve Loughran <stevel@apache.org> Signed-off-by: Akira Ajisaka <aajisaka@apache.org> # Conflicts: # hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/TestInMemoryLevelDBAliasMapClient.java # hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/TestLevelDbMockAliasMapClient.java # hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core/src/main/java/org/apache/hadoop/yarn/service/client/ServiceClient.java # hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/test/java/org/apache/hadoop/yarn/csi/client/TestCsiClient.java Change-Id: I409cb52a6c5dac7343be66ce3af0c25ba659db72

HADOOP-17653. Do not use guava's Files.createTempDir().

8150e98

Change-Id: I00b0836e1f56f58ed23ffbbf6e8e9ff5409a65bf

virajjasani reviewed Apr 21, 2021

View reviewed changes

...main/java/org/apache/hadoop/hdfs/server/federation/store/driver/impl/StateStoreFileImpl.java Show resolved Hide resolved

Log a debug message if unable to create a temp directory.

fa71444

Change-Id: Ifd49e036649fe9d1dd1a572b70b3ab9e01acf17a

steveloughran requested changes Apr 22, 2021

View reviewed changes

...g/apache/hadoop/hdfs/server/common/blockaliasmap/impl/TestInMemoryLevelDBAliasMapClient.java Outdated Show resolved Hide resolved

...a/org/apache/hadoop/hdfs/server/common/blockaliasmap/impl/TestLevelDbMockAliasMapClient.java Outdated Show resolved Hide resolved

Move imports to the right order.

72c14dd

Change-Id: I478c4b7dbf9c31a5d4fa70e8a07a73a504c3cf17

steveloughran approved these changes Apr 23, 2021

View reviewed changes

aajisaka approved these changes May 2, 2021

View reviewed changes

aajisaka merged commit f1e1809 into apache:trunk May 2, 2021

aajisaka pushed a commit that referenced this pull request May 2, 2021

HADOOP-17653. Do not use guava's Files.createTempDir(). (#2945)

670205c

Reviewed-by: Steve Loughran <stevel@apache.org> Signed-off-by: Akira Ajisaka <aajisaka@apache.org> (cherry picked from commit f1e1809)

kiran-maturi pushed a commit to kiran-maturi/hadoop that referenced this pull request Nov 24, 2021

HADOOP-17653. Do not use guava's Files.createTempDir(). (apache#2945)

a365828

Reviewed-by: Steve Loughran <stevel@apache.org> Signed-off-by: Akira Ajisaka <aajisaka@apache.org>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HADOOP-17653. Do not use guava's Files.createTempDir(). #2945

HADOOP-17653. Do not use guava's Files.createTempDir(). #2945

jojochuang commented Apr 21, 2021

virajjasani left a comment

steveloughran left a comment

virajjasani commented Apr 22, 2021 •

edited

Loading

aajisaka commented Apr 22, 2021

virajjasani commented Apr 22, 2021 •

edited

Loading

hadoop-yetus commented Apr 22, 2021

aajisaka commented Apr 23, 2021

virajjasani commented Apr 23, 2021

steveloughran left a comment

hadoop-yetus commented Apr 23, 2021

jojochuang commented Apr 26, 2021

hadoop-yetus commented Apr 26, 2021

hadoop-yetus commented Apr 26, 2021

steveloughran commented Apr 28, 2021

jojochuang commented Apr 30, 2021

aajisaka commented May 2, 2021

aajisaka commented May 2, 2021

aajisaka commented May 2, 2021

virajjasani commented May 2, 2021

HADOOP-17653. Do not use guava's Files.createTempDir(). #2945

HADOOP-17653. Do not use guava's Files.createTempDir(). #2945

Conversation

jojochuang commented Apr 21, 2021

virajjasani left a comment

Choose a reason for hiding this comment

steveloughran left a comment

Choose a reason for hiding this comment

virajjasani commented Apr 22, 2021 • edited Loading

aajisaka commented Apr 22, 2021

virajjasani commented Apr 22, 2021 • edited Loading

hadoop-yetus commented Apr 22, 2021

aajisaka commented Apr 23, 2021

virajjasani commented Apr 23, 2021

steveloughran left a comment

Choose a reason for hiding this comment

hadoop-yetus commented Apr 23, 2021

jojochuang commented Apr 26, 2021

hadoop-yetus commented Apr 26, 2021

hadoop-yetus commented Apr 26, 2021

steveloughran commented Apr 28, 2021

jojochuang commented Apr 30, 2021

aajisaka commented May 2, 2021

aajisaka commented May 2, 2021

aajisaka commented May 2, 2021

virajjasani commented May 2, 2021

virajjasani commented Apr 22, 2021 •

edited

Loading

virajjasani commented Apr 22, 2021 •

edited

Loading