YARN-11137. Improve log message in FederationClientInterceptor #4336

slfan1989 · 2022-05-20T11:59:10Z

Description of PR

JIRA:YARN-11137. Improve log message in FederationClientInterceptor.

For code changes:

The log methods are inconsistent, some use the splicing method, and some use the placeholder method.

1.org.apache.hadoop.yarn.server.router.clientrmsubmit.FederationClientInterceptor#getNewApplication

for (int i = 0; i < numSubmitRetries; ++i) {
   SubClusterId subClusterId = getRandomActiveSubCluster(subClustersActive);
   LOG.debug(
       "getNewApplication try #{} on SubCluster {}", i, subClusterId);
   ApplicationClientProtocol clientRMProxy =
        getClientRMProxyForSubCluster(subClusterId);
  ...
}

2.org.apache.hadoop.yarn.server.router.clientrmsubmit.FederationClientInterceptor#submitApplication

for (int i = 0; i < numSubmitRetries; ++i) {      
         SubClusterId subClusterId = policyFacade.getHomeSubcluster(
          request.getApplicationSubmissionContext(), blacklist);
      LOG.info("submitApplication appId" + applicationId + " try #" + i
          + " on SubCluster " + subClusterId);
   ...
}

I think the first way is better.

hadoop-yetus · 2022-05-20T13:35:21Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 39s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	37m 28s		trunk passed
+1 💚	compile	0m 49s		trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	compile	0m 46s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	checkstyle	0m 48s		trunk passed
+1 💚	mvnsite	0m 51s		trunk passed
+1 💚	javadoc	0m 55s		trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javadoc	0m 44s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	1m 18s		trunk passed
+1 💚	shadedclient	20m 40s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 31s		the patch passed
+1 💚	compile	0m 32s		the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javac	0m 32s		the patch passed
+1 💚	compile	0m 29s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	javac	0m 29s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 22s		the patch passed
+1 💚	mvnsite	0m 31s		the patch passed
+1 💚	javadoc	0m 29s		the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javadoc	0m 27s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	0m 58s		the patch passed
+1 💚	shadedclient	20m 6s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	2m 50s	/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt	hadoop-yarn-server-router in the patch passed.
+1 💚	asflicense	0m 49s		The patch does not generate ASF License warnings.
		94m 50s

Reason	Tests
Failed junit tests	hadoop.yarn.server.router.clientrm.TestFederationClientInterceptor

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4336/1/artifact/out/Dockerfile
GITHUB PR	#4336
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux ca81d51f6979 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24 17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `247cff6`
Default Java	Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4336/1/testReport/
Max. process+thread count	852 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4336/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-05-20T17:26:40Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 37s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	37m 31s		trunk passed
+1 💚	compile	0m 49s		trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	compile	0m 47s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	checkstyle	0m 48s		trunk passed
+1 💚	mvnsite	0m 51s		trunk passed
+1 💚	javadoc	0m 55s		trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javadoc	0m 45s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	1m 19s		trunk passed
+1 💚	shadedclient	20m 33s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 31s		the patch passed
+1 💚	compile	0m 32s		the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javac	0m 32s		the patch passed
+1 💚	compile	0m 28s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	javac	0m 28s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 23s		the patch passed
+1 💚	mvnsite	0m 31s		the patch passed
+1 💚	javadoc	0m 29s		the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javadoc	0m 27s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	0m 58s		the patch passed
+1 💚	shadedclient	19m 55s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	2m 49s		hadoop-yarn-server-router in the patch passed.
+1 💚	asflicense	0m 49s		The patch does not generate ASF License warnings.
		94m 34s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4336/2/artifact/out/Dockerfile
GITHUB PR	#4336
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 923a39c20dfb 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24 17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `c103214`
Default Java	Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4336/2/testReport/
Max. process+thread count	759 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4336/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

slfan1989 · 2022-05-20T22:24:14Z

Hi, @goiri please help to review the code ,thank you very match!

hadoop-yetus · 2022-05-22T11:56:03Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 38s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	36m 54s		trunk passed
+1 💚	compile	0m 49s		trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	compile	0m 47s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	checkstyle	0m 48s		trunk passed
+1 💚	mvnsite	0m 51s		trunk passed
+1 💚	javadoc	0m 55s		trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javadoc	0m 44s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	1m 18s		trunk passed
+1 💚	shadedclient	20m 22s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 31s		the patch passed
+1 💚	compile	0m 31s		the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javac	0m 31s		the patch passed
+1 💚	compile	0m 29s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	javac	0m 29s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 23s		the patch passed
+1 💚	mvnsite	0m 31s		the patch passed
+1 💚	javadoc	0m 29s		the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javadoc	0m 27s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	0m 57s		the patch passed
+1 💚	shadedclient	20m 5s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	2m 55s		hadoop-yarn-server-router in the patch passed.
+1 💚	asflicense	0m 50s		The patch does not generate ASF License warnings.
		94m 4s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4336/3/artifact/out/Dockerfile
GITHUB PR	#4336
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux f4a1616b39de 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24 17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `61a2f13`
Default Java	Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4336/3/testReport/
Max. process+thread count	762 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4336/3/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-05-22T12:52:30Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 1s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	36m 57s		trunk passed
+1 💚	compile	0m 50s		trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	compile	0m 47s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	checkstyle	0m 48s		trunk passed
+1 💚	mvnsite	0m 51s		trunk passed
+1 💚	javadoc	0m 57s		trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javadoc	0m 45s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	1m 19s		trunk passed
+1 💚	shadedclient	20m 43s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 31s		the patch passed
+1 💚	compile	0m 32s		the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javac	0m 32s		the patch passed
+1 💚	compile	0m 30s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	javac	0m 30s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 22s		the patch passed
+1 💚	mvnsite	0m 31s		the patch passed
+1 💚	javadoc	0m 28s		the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javadoc	0m 28s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	0m 57s		the patch passed
+1 💚	shadedclient	20m 4s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	2m 52s		hadoop-yarn-server-router in the patch passed.
+1 💚	asflicense	0m 49s		The patch does not generate ASF License warnings.
		94m 33s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4336/4/artifact/out/Dockerfile
GITHUB PR	#4336
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 69c21655b75f 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `4aa6294`
Default Java	Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4336/4/testReport/
Max. process+thread count	758 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4336/4/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

slfan1989 · 2022-05-24T22:13:23Z

Hi, @goiri @tomscut please help to review the code , thank you very match!

…e#4336)

slfan1989 added 3 commits May 20, 2022 02:11

YARN-11137. Improve log message in FederationClientInterceptor.

09a573d

YARN-11137. Fix CheckStyle.

710b0d2

YARN-11137. Fix CheckStyle.

247cff6

YARN-11137. Fix CheckStyle.

c103214

slfan1989 and others added 2 commits May 22, 2022 03:20

Merge branch 'apache:trunk' into YARN-11137

61a2f13

YARN-11137. Improve log message in FederationClientInterceptor.

4aa6294

slfan1989 mentioned this pull request May 23, 2022

YARN-10487. Support getQueueUserAcls, listReservations, getApplicationAttempts, getContainerReport, getContainers, getResourceTypeInfo API's for Federation #4341

Merged

goiri approved these changes May 25, 2022

View reviewed changes

goiri merged commit 6787680 into apache:trunk May 25, 2022

HarshitGupta11 pushed a commit to HarshitGupta11/hadoop that referenced this pull request Nov 28, 2022

YARN-11137. Improve log message in FederationClientInterceptor (apach…

6899a4d

…e#4336)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

YARN-11137. Improve log message in FederationClientInterceptor #4336

YARN-11137. Improve log message in FederationClientInterceptor #4336

slfan1989 commented May 20, 2022 •

edited

Loading

hadoop-yetus commented May 20, 2022

hadoop-yetus commented May 20, 2022

slfan1989 commented May 20, 2022

hadoop-yetus commented May 22, 2022

hadoop-yetus commented May 22, 2022

slfan1989 commented May 24, 2022

YARN-11137. Improve log message in FederationClientInterceptor #4336

YARN-11137. Improve log message in FederationClientInterceptor #4336

Conversation

slfan1989 commented May 20, 2022 • edited Loading

Description of PR

For code changes:

hadoop-yetus commented May 20, 2022

hadoop-yetus commented May 20, 2022

slfan1989 commented May 20, 2022

hadoop-yetus commented May 22, 2022

hadoop-yetus commented May 22, 2022

slfan1989 commented May 24, 2022

slfan1989 commented May 20, 2022 •

edited

Loading