MAPREDUCE-7469. NNBench createControlFiles should use thread pool to improve performance. #6463

LiuGuH · 2024-01-18T08:22:24Z

Description of PR

JIRA: MAPREDUCE-7469 NNBench createControlFiles should use thread pool to improve performance.

How was this patch tested?

Add test case for this.

hadoop-yetus · 2024-01-18T12:33:57Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 34s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
-1 ❌	mvninstall	41m 33s	/branch-mvninstall-root.txt	root in trunk failed.
+1 💚	compile	0m 34s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	compile	0m 32s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	checkstyle	0m 40s		trunk passed
+1 💚	mvnsite	0m 37s		trunk passed
+1 💚	javadoc	0m 33s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 26s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	0m 51s		trunk passed
+1 💚	shadedclient	32m 27s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 27s		the patch passed
+1 💚	compile	0m 26s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javac	0m 26s		the patch passed
+1 💚	compile	0m 25s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	javac	0m 25s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 28s	/results-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient: The patch generated 5 new + 144 unchanged - 0 fixed = 149 total (was 144)
+1 💚	mvnsite	0m 27s		the patch passed
+1 💚	javadoc	0m 20s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 19s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	0m 51s		the patch passed
+1 💚	shadedclient	32m 20s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	130m 41s		hadoop-mapreduce-client-jobclient in the patch passed.
+1 💚	asflicense	0m 40s		The patch does not generate ASF License warnings.
		250m 17s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6463/1/artifact/out/Dockerfile
GITHUB PR	#6463
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux c70ec5b67c93 5.15.0-91-generic #101-Ubuntu SMP Tue Nov 14 13:30:08 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `b96dc48`
Default Java	Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6463/1/testReport/
Max. process+thread count	1225 (vs. ulimit of 5500)
modules	C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6463/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

…mprove performance.

slfan1989 · 2024-01-19T05:51:32Z

@LiuGuH after a quick review, it seems reasonable to me. I will take a closer look later.

slfan1989 · 2024-01-19T05:54:54Z

...e-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/hdfs/NNBench.java

+      try {
+        list.get(i).get();
+      } catch (InterruptedException | ExecutionException e) {
+        LOG.error("Creating control files Error.");


LOG.error("Creating control files Error.", e);

Changed. Thanks

hadoop-yetus · 2024-01-19T08:04:23Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 35s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	43m 35s		trunk passed
+1 💚	compile	0m 33s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	compile	0m 31s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	checkstyle	0m 40s		trunk passed
+1 💚	mvnsite	0m 37s		trunk passed
+1 💚	javadoc	0m 35s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 27s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	0m 51s		trunk passed
+1 💚	shadedclient	32m 4s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 27s		the patch passed
+1 💚	compile	0m 25s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javac	0m 25s		the patch passed
+1 💚	compile	0m 24s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	javac	0m 24s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 28s		the patch passed
+1 💚	mvnsite	0m 27s		the patch passed
+1 💚	javadoc	0m 19s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 19s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	0m 50s		the patch passed
+1 💚	shadedclient	32m 1s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	132m 16s		hadoop-mapreduce-client-jobclient in the patch passed.
+1 💚	asflicense	0m 41s		The patch does not generate ASF License warnings.
		253m 8s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6463/2/artifact/out/Dockerfile
GITHUB PR	#6463
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 664c1e5af7c6 5.15.0-91-generic #101-Ubuntu SMP Tue Nov 14 13:30:08 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `e9aaaea`
Default Java	Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6463/2/testReport/
Max. process+thread count	1210 (vs. ulimit of 5500)
modules	C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6463/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2024-01-19T10:45:56Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 22s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	31m 11s		trunk passed
+1 💚	compile	0m 20s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	compile	0m 18s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	checkstyle	0m 22s		trunk passed
+1 💚	mvnsite	0m 24s		trunk passed
+1 💚	javadoc	0m 24s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 18s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	0m 36s		trunk passed
+1 💚	shadedclient	19m 18s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 15s		the patch passed
+1 💚	compile	0m 15s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javac	0m 15s		the patch passed
+1 💚	compile	0m 14s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	javac	0m 14s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 15s		the patch passed
+1 💚	mvnsite	0m 19s		the patch passed
+1 💚	javadoc	0m 13s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 14s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	0m 32s		the patch passed
+1 💚	shadedclient	19m 19s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	114m 9s		hadoop-mapreduce-client-jobclient in the patch passed.
+1 💚	asflicense	0m 24s		The patch does not generate ASF License warnings.
		192m 10s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6463/4/artifact/out/Dockerfile
GITHUB PR	#6463
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux e2b47b152974 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `82d36c8`
Default Java	Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6463/4/testReport/
Max. process+thread count	1236 (vs. ulimit of 5500)
modules	C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6463/4/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2024-01-19T11:43:15Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 30s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	42m 43s		trunk passed
+1 💚	compile	0m 34s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	compile	0m 32s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	checkstyle	0m 42s		trunk passed
+1 💚	mvnsite	0m 37s		trunk passed
+1 💚	javadoc	0m 33s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 26s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	0m 51s		trunk passed
+1 💚	shadedclient	32m 38s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 25s		the patch passed
+1 💚	compile	0m 27s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javac	0m 27s		the patch passed
+1 💚	compile	0m 25s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	javac	0m 25s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 28s		the patch passed
+1 💚	mvnsite	0m 28s		the patch passed
+1 💚	javadoc	0m 20s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 18s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	0m 50s		the patch passed
+1 💚	shadedclient	31m 51s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	130m 21s		hadoop-mapreduce-client-jobclient in the patch passed.
+1 💚	asflicense	0m 43s		The patch does not generate ASF License warnings.
		250m 58s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6463/3/artifact/out/Dockerfile
GITHUB PR	#6463
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 2cff18e37959 5.15.0-91-generic #101-Ubuntu SMP Tue Nov 14 13:30:08 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `82d36c8`
Default Java	Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6463/3/testReport/
Max. process+thread count	1218 (vs. ulimit of 5500)
modules	C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6463/3/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

slfan1989 · 2024-01-22T12:02:22Z

LGTM

saxepran · 2024-01-27T11:06:16Z

...e-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/hdfs/NNBench.java

+      }
+    }
+
+    executorService.shutdown();


If we are shutting down the service, should we spawn executorService in this method only instead of creating at object creation.

Done. Thanks.

hadoop-yetus · 2024-01-29T07:35:02Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 37s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	42m 31s		trunk passed
+1 💚	compile	0m 33s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	compile	0m 33s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	checkstyle	0m 42s		trunk passed
+1 💚	mvnsite	0m 37s		trunk passed
+1 💚	javadoc	0m 35s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 28s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	0m 52s		trunk passed
+1 💚	shadedclient	32m 17s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 26s		the patch passed
+1 💚	compile	0m 26s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javac	0m 26s		the patch passed
+1 💚	compile	0m 25s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	javac	0m 25s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 28s		the patch passed
+1 💚	mvnsite	0m 28s		the patch passed
+1 💚	javadoc	0m 20s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 19s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	0m 50s		the patch passed
+1 💚	shadedclient	33m 5s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	131m 46s		hadoop-mapreduce-client-jobclient in the patch passed.
+1 💚	asflicense	0m 42s		The patch does not generate ASF License warnings.
		253m 2s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6463/5/artifact/out/Dockerfile
GITHUB PR	#6463
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 07ea3b5a8216 5.15.0-91-generic #101-Ubuntu SMP Tue Nov 14 13:30:08 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `689fa7d`
Default Java	Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6463/5/testReport/
Max. process+thread count	1207 (vs. ulimit of 5500)
modules	C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6463/5/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

saxepran

Thanks for taking the comment. LGTM!

LiuGuH · 2024-02-02T07:41:55Z

@slfan1989 ， hello, sir, can we merge this , thanks.

slfan1989 · 2024-02-03T20:57:31Z

@LiuGuH Sorry for the late reply, we need to trigger compilation again. If there is no issue, the PR will be merged.

LiuGuH · 2024-03-05T12:54:39Z

@LiuGuH Sorry for the late reply, we need to trigger compilation again. If there is no issue, the PR will be merged.

@slfan1989 Sorry for the late reply because I took a long vacation. Trigger compilation now. Thanks

hadoop-yetus · 2024-03-05T17:28:33Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 32s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	59m 14s		trunk passed
+1 💚	compile	0m 37s		trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	compile	0m 35s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	checkstyle	0m 44s		trunk passed
+1 💚	mvnsite	0m 39s		trunk passed
+1 💚	javadoc	0m 35s		trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javadoc	0m 28s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	0m 52s		trunk passed
+1 💚	shadedclient	34m 45s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 25s		the patch passed
+1 💚	compile	0m 26s		the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javac	0m 26s		the patch passed
+1 💚	compile	0m 25s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	javac	0m 25s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 28s		the patch passed
+1 💚	mvnsite	0m 26s		the patch passed
+1 💚	javadoc	0m 19s		the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚	javadoc	0m 19s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	0m 49s		the patch passed
+1 💚	shadedclient	33m 27s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	132m 57s		hadoop-mapreduce-client-jobclient in the patch passed.
+1 💚	asflicense	0m 43s		The patch does not generate ASF License warnings.
		274m 25s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6463/6/artifact/out/Dockerfile
GITHUB PR	#6463
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 9dc282a1de7f 5.15.0-91-generic #101-Ubuntu SMP Tue Nov 14 13:30:08 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `d910acb`
Default Java	Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6463/6/testReport/
Max. process+thread count	1237 (vs. ulimit of 5500)
modules	C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6463/6/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

github-actions bot added MapReduce trunk labels Jan 18, 2024

MAPREDUCE-7469 NNBench createControlFiles should use thread pool to i…

e9aaaea

…mprove performance.

LiuGuH force-pushed the NNBench branch from b96dc48 to e9aaaea Compare January 19, 2024 03:50

slfan1989 self-requested a review January 19, 2024 05:51

slfan1989 reviewed Jan 19, 2024

View reviewed changes

Optimization error log

82d36c8

LiuGuH force-pushed the NNBench branch from abe7bc6 to 82d36c8 Compare January 19, 2024 07:32

slfan1989 approved these changes Jan 23, 2024

View reviewed changes

saxepran reviewed Jan 27, 2024

View reviewed changes

spawn executorService in createControlFiles()

689fa7d

LiuGuH requested a review from saxepran January 29, 2024 03:21

saxepran approved these changes Jan 29, 2024

View reviewed changes

triggle compile only

d910acb

LiuGuH requested a review from slfan1989 March 7, 2024 03:19

slfan1989 approved these changes Mar 7, 2024

View reviewed changes

slfan1989 changed the title ~~MAPREDUCE-7469 NNBench createControlFiles should use thread pool to i…~~ MAPREDUCE-7469. NNBench createControlFiles should use thread pool to improve performance. Mar 15, 2024

slfan1989 merged commit a60b5e2 into apache:trunk Mar 22, 2024
4 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MAPREDUCE-7469. NNBench createControlFiles should use thread pool to improve performance. #6463

MAPREDUCE-7469. NNBench createControlFiles should use thread pool to improve performance. #6463

LiuGuH commented Jan 18, 2024

hadoop-yetus commented Jan 18, 2024

slfan1989 commented Jan 19, 2024

slfan1989 Jan 19, 2024

LiuGuH Jan 19, 2024

hadoop-yetus commented Jan 19, 2024

hadoop-yetus commented Jan 19, 2024

hadoop-yetus commented Jan 19, 2024

slfan1989 commented Jan 22, 2024

saxepran Jan 27, 2024

LiuGuH Jan 29, 2024

hadoop-yetus commented Jan 29, 2024

saxepran left a comment

LiuGuH commented Feb 2, 2024

slfan1989 commented Feb 3, 2024

LiuGuH commented Mar 5, 2024

hadoop-yetus commented Mar 5, 2024

MAPREDUCE-7469. NNBench createControlFiles should use thread pool to improve performance. #6463

MAPREDUCE-7469. NNBench createControlFiles should use thread pool to improve performance. #6463

Conversation

LiuGuH commented Jan 18, 2024

Description of PR

How was this patch tested?

hadoop-yetus commented Jan 18, 2024

slfan1989 commented Jan 19, 2024

slfan1989 Jan 19, 2024

Choose a reason for hiding this comment

LiuGuH Jan 19, 2024

Choose a reason for hiding this comment

hadoop-yetus commented Jan 19, 2024

hadoop-yetus commented Jan 19, 2024

hadoop-yetus commented Jan 19, 2024

slfan1989 commented Jan 22, 2024

saxepran Jan 27, 2024

Choose a reason for hiding this comment

LiuGuH Jan 29, 2024

Choose a reason for hiding this comment

hadoop-yetus commented Jan 29, 2024

saxepran left a comment

Choose a reason for hiding this comment

LiuGuH commented Feb 2, 2024

slfan1989 commented Feb 3, 2024

LiuGuH commented Mar 5, 2024

hadoop-yetus commented Mar 5, 2024