HDFS-17026. RBF: NamenodeHeartbeatService should update JMX report with configurable frequency #5691

hchaverri · 2023-05-26T04:31:41Z

…er frequency

Description of PR

Changes to the NamenodeHeartbeatService to update JMX reports on a configurable frequency or to disable JMX updates altogether, as opposed to updating it every time the service wakes up. This will help reduce the load on Namenodes in clusters with a high number of routers.
With these changes, we expect the NamenodeStatusReport to contain slightly outdated Namenode information, which should not impact critical paths for routers.

How was this patch tested?

Added unit test on TestRouterNamenodeMonitoring#testJmxRequestFrequency that tests the following configurations:

Default configuration which should update JMX every time the report is generated.
Configuration with JMX updates disabled, which should never update JMX
Configuration with a high JMX update interval, which should not update JMX every time the report is generated.

For code changes:

Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

…er frequency

hadoop-yetus · 2023-05-26T06:23:47Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 48s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	33m 15s		trunk passed
+1 💚	compile	0m 41s		trunk passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
+1 💚	compile	0m 36s		trunk passed with JDK Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	checkstyle	0m 35s		trunk passed
+1 💚	mvnsite	0m 42s		trunk passed
+1 💚	javadoc	0m 46s		trunk passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
+1 💚	javadoc	0m 32s		trunk passed with JDK Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	spotbugs	1m 30s		trunk passed
+1 💚	shadedclient	20m 46s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 30s		the patch passed
+1 💚	compile	0m 31s		the patch passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
+1 💚	javac	0m 31s		the patch passed
+1 💚	compile	0m 28s		the patch passed with JDK Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	javac	0m 28s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 19s	/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs-rbf.txt	hadoop-hdfs-project/hadoop-hdfs-rbf: The patch generated 5 new + 0 unchanged - 0 fixed = 5 total (was 0)
+1 💚	mvnsite	0m 31s		the patch passed
+1 💚	javadoc	0m 28s		the patch passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
+1 💚	javadoc	0m 24s		the patch passed with JDK Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	spotbugs	1m 17s		the patch passed
+1 💚	shadedclient	20m 29s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	22m 4s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt	hadoop-hdfs-rbf in the patch passed.
+1 💚	asflicense	0m 40s		The patch does not generate ASF License warnings.
		110m 47s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.federation.router.TestRBFConfigFields

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5691/1/artifact/out/Dockerfile
GITHUB PR	#5691
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux bf4d34a34362 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3 19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `1ccc1aa`
Default Java	Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~20.04-b09
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~20.04-b09
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5691/1/testReport/
Max. process+thread count	2737 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5691/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

Hexiaoqiao

Leave some nit comments inline. FYI. Please check the failed unit test and checkstyles.

...op-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RBFConfigKeys.java

.../src/main/java/org/apache/hadoop/hdfs/server/federation/router/NamenodeHeartbeatService.java

hadoop-yetus · 2023-05-26T17:02:51Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 49s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+0 🆗	xmllint	0m 1s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	35m 15s		trunk passed
+1 💚	compile	0m 39s		trunk passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
+1 💚	compile	0m 36s		trunk passed with JDK Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	checkstyle	0m 35s		trunk passed
+1 💚	mvnsite	0m 42s		trunk passed
+1 💚	javadoc	0m 46s		trunk passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
+1 💚	javadoc	0m 33s		trunk passed with JDK Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	spotbugs	1m 30s		trunk passed
+1 💚	shadedclient	20m 39s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 31s		the patch passed
+1 💚	compile	0m 31s		the patch passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
+1 💚	javac	0m 31s		the patch passed
+1 💚	compile	0m 28s		the patch passed with JDK Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	javac	0m 28s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 19s		the patch passed
+1 💚	mvnsite	0m 31s		the patch passed
+1 💚	javadoc	0m 27s		the patch passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
+1 💚	javadoc	0m 24s		the patch passed with JDK Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	spotbugs	1m 15s		the patch passed
+1 💚	shadedclient	20m 31s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	21m 57s		hadoop-hdfs-rbf in the patch passed.
+1 💚	asflicense	0m 39s		The patch does not generate ASF License warnings.
		112m 18s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5691/2/artifact/out/Dockerfile
GITHUB PR	#5691
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname	Linux 787e0b2de29f 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3 19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `ec2fb86`
Default Java	Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~20.04-b09
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~20.04-b09
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5691/2/testReport/
Max. process+thread count	2703 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5691/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

…valMs to 0

.../src/main/java/org/apache/hadoop/hdfs/server/federation/router/NamenodeHeartbeatService.java

hadoop-yetus · 2023-05-26T19:38:08Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 46s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	35m 14s		trunk passed
+1 💚	compile	0m 40s		trunk passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
+1 💚	compile	0m 36s		trunk passed with JDK Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	checkstyle	0m 34s		trunk passed
+1 💚	mvnsite	0m 41s		trunk passed
+1 💚	javadoc	0m 47s		trunk passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
+1 💚	javadoc	0m 32s		trunk passed with JDK Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	spotbugs	1m 31s		trunk passed
+1 💚	shadedclient	20m 48s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 29s		the patch passed
+1 💚	compile	0m 32s		the patch passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
+1 💚	javac	0m 32s		the patch passed
+1 💚	compile	0m 29s		the patch passed with JDK Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	javac	0m 29s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 19s		the patch passed
+1 💚	mvnsite	0m 32s		the patch passed
+1 💚	javadoc	0m 29s		the patch passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
+1 💚	javadoc	0m 24s		the patch passed with JDK Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	spotbugs	1m 19s		the patch passed
+1 💚	shadedclient	20m 40s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	22m 5s		hadoop-hdfs-rbf in the patch passed.
+1 💚	asflicense	0m 40s		The patch does not generate ASF License warnings.
		112m 45s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5691/3/artifact/out/Dockerfile
GITHUB PR	#5691
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname	Linux 5f07baf11f55 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3 19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `1f2158f`
Default Java	Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~20.04-b09
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~20.04-b09
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5691/3/testReport/
Max. process+thread count	2598 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5691/3/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2023-05-26T20:38:33Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 54s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	39m 55s		trunk passed
+1 💚	compile	0m 40s		trunk passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
+1 💚	compile	0m 37s		trunk passed with JDK Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	checkstyle	0m 35s		trunk passed
+1 💚	mvnsite	0m 41s		trunk passed
+1 💚	javadoc	0m 46s		trunk passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
+1 💚	javadoc	0m 29s		trunk passed with JDK Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	spotbugs	1m 32s		trunk passed
+1 💚	shadedclient	21m 47s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 31s		the patch passed
+1 💚	compile	0m 30s		the patch passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
+1 💚	javac	0m 30s		the patch passed
+1 💚	compile	0m 29s		the patch passed with JDK Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	javac	0m 29s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 19s		the patch passed
+1 💚	mvnsite	0m 32s		the patch passed
+1 💚	javadoc	0m 28s		the patch passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
+1 💚	javadoc	0m 25s		the patch passed with JDK Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	spotbugs	1m 18s		the patch passed
+1 💚	shadedclient	20m 31s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	22m 30s		hadoop-hdfs-rbf in the patch passed.
+1 💚	asflicense	0m 39s		The patch does not generate ASF License warnings.
		118m 52s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5691/4/artifact/out/Dockerfile
GITHUB PR	#5691
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname	Linux f9ff3dfd52d0 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3 19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `3e72f2b`
Default Java	Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~20.04-b09
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~20.04-b09
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5691/4/testReport/
Max. process+thread count	2172 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5691/4/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

simbadzina

LGTM.

How does the AvailableSpaceResolver and related code behave when we fully disable updating JMX parameters?

.../src/main/java/org/apache/hadoop/hdfs/server/federation/router/NamenodeHeartbeatService.java

hadoop-yetus · 2023-05-27T18:17:14Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 49s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	33m 15s		trunk passed
+1 💚	compile	0m 41s		trunk passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
+1 💚	compile	0m 38s		trunk passed with JDK Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	checkstyle	0m 35s		trunk passed
+1 💚	mvnsite	0m 43s		trunk passed
+1 💚	javadoc	0m 46s		trunk passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
+1 💚	javadoc	0m 33s		trunk passed with JDK Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	spotbugs	1m 30s		trunk passed
+1 💚	shadedclient	20m 41s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 30s		the patch passed
+1 💚	compile	0m 32s		the patch passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
+1 💚	javac	0m 32s		the patch passed
+1 💚	compile	0m 29s		the patch passed with JDK Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	javac	0m 29s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 17s		the patch passed
+1 💚	mvnsite	0m 31s		the patch passed
+1 💚	javadoc	0m 28s		the patch passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
+1 💚	javadoc	0m 24s		the patch passed with JDK Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	spotbugs	1m 15s		the patch passed
+1 💚	shadedclient	20m 32s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	22m 8s		hadoop-hdfs-rbf in the patch passed.
+1 💚	asflicense	0m 40s		The patch does not generate ASF License warnings.
		110m 33s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5691/5/artifact/out/Dockerfile
GITHUB PR	#5691
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname	Linux 0dd49994fd7a 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3 19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `71626bd`
Default Java	Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~20.04-b09
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~20.04-b09
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5691/5/testReport/
Max. process+thread count	2696 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5691/5/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

.../src/main/java/org/apache/hadoop/hdfs/server/federation/router/NamenodeHeartbeatService.java

hadoop-yetus · 2023-05-29T19:59:22Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 48s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+0 🆗	xmllint	0m 1s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	35m 13s		trunk passed
+1 💚	compile	0m 40s		trunk passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
+1 💚	compile	0m 37s		trunk passed with JDK Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	checkstyle	0m 34s		trunk passed
+1 💚	mvnsite	0m 42s		trunk passed
+1 💚	javadoc	0m 46s		trunk passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
+1 💚	javadoc	0m 33s		trunk passed with JDK Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	spotbugs	1m 30s		trunk passed
+1 💚	shadedclient	20m 29s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 31s		the patch passed
+1 💚	compile	0m 31s		the patch passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
+1 💚	javac	0m 31s		the patch passed
+1 💚	compile	0m 27s		the patch passed with JDK Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	javac	0m 27s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 19s		the patch passed
+1 💚	mvnsite	0m 32s		the patch passed
+1 💚	javadoc	0m 28s		the patch passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
+1 💚	javadoc	0m 24s		the patch passed with JDK Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	spotbugs	1m 19s		the patch passed
+1 💚	shadedclient	20m 48s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	22m 12s		hadoop-hdfs-rbf in the patch passed.
+1 💚	asflicense	0m 39s		The patch does not generate ASF License warnings.
		112m 41s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5691/6/artifact/out/Dockerfile
GITHUB PR	#5691
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname	Linux 4dc731c52ecc 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3 19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `39d6aaf`
Default Java	Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~20.04-b09
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_362-8u372-ga~~us1-0ubuntu1~~20.04-b09
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5691/6/testReport/
Max. process+thread count	2703 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5691/6/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

yzhang559

LGTM

hchaverri · 2023-05-30T21:40:49Z

@Hexiaoqiao @virajjasani @mkuchenbecker we have a few approvals already. Are you ok with closing this PR?

Hexiaoqiao

LGTM. +1.

Hexiaoqiao · 2023-05-31T06:28:14Z

Committed to trunk. Thanks @hchaverri for your contributions! And thanks every reviewers (Too many to mentioned folks)!

…th configurable frequency. (apache#5691). Contributed by hchaverri. (apache#205)

HDFS-17026. Configuring NamenodeHeartbeatService to update JMX on low…

1ccc1aa

…er frequency

Hexiaoqiao reviewed May 26, 2023

View reviewed changes

...op-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RBFConfigKeys.java Outdated Show resolved Hide resolved

.../src/main/java/org/apache/hadoop/hdfs/server/federation/router/NamenodeHeartbeatService.java Outdated Show resolved Hide resolved

Fixing checkstyle and adding properties to hdfs-rbf-default.xml

ec2fb86

virajjasani reviewed May 26, 2023

View reviewed changes

.../src/main/java/org/apache/hadoop/hdfs/server/federation/router/NamenodeHeartbeatService.java Outdated Show resolved Hide resolved

Removing updateJmxEnabled property in favor of setting updateJmxInter…

1f2158f

…valMs to 0

mkuchenbecker reviewed May 26, 2023

View reviewed changes

.../src/main/java/org/apache/hadoop/hdfs/server/federation/router/NamenodeHeartbeatService.java Show resolved Hide resolved

simbadzina reviewed May 26, 2023

View reviewed changes

.../src/main/java/org/apache/hadoop/hdfs/server/federation/router/NamenodeHeartbeatService.java Outdated Show resolved Hide resolved

hchaverri changed the title ~~HDFS-17026. Configuring NamenodeHeartbeatService to update JMX on low…~~ HDFS-17026. RBF: Configuring NamenodeHeartbeatService to update JMX on low… May 26, 2023

Fix in shouldUpdateJmx() logic

3e72f2b

simbadzina reviewed May 27, 2023

View reviewed changes

.../src/main/java/org/apache/hadoop/hdfs/server/federation/router/NamenodeHeartbeatService.java Outdated Show resolved Hide resolved

goiri reviewed May 27, 2023

View reviewed changes

.../src/main/java/org/apache/hadoop/hdfs/server/federation/router/NamenodeHeartbeatService.java Outdated Show resolved Hide resolved

.../src/main/java/org/apache/hadoop/hdfs/server/federation/router/NamenodeHeartbeatService.java Outdated Show resolved Hide resolved

Using getTimeDuration

71626bd

simbadzina approved these changes May 27, 2023

View reviewed changes

goiri reviewed May 29, 2023

View reviewed changes

.../src/main/java/org/apache/hadoop/hdfs/server/federation/router/NamenodeHeartbeatService.java Outdated Show resolved Hide resolved

Reformatting shouldUpdateJmx method

39d6aaf

yzhang559 approved these changes May 30, 2023

View reviewed changes

simbadzina approved these changes May 30, 2023

View reviewed changes

goiri approved these changes May 30, 2023

View reviewed changes

Hexiaoqiao changed the title ~~HDFS-17026. RBF: Configuring NamenodeHeartbeatService to update JMX on low…~~ HDFS-17026. RBF: NamenodeHeartbeatService should update JMX report with configurable frequency May 31, 2023

Hexiaoqiao approved these changes May 31, 2023

View reviewed changes

Hexiaoqiao merged commit 124313d into apache:trunk May 31, 2023

hchaverri mentioned this pull request Jun 3, 2023

HDFS-17026. RBF: NamenodeHeartbeatService should update JMX report with configurable frequency #5714

Closed

4 tasks

NyteKnight pushed a commit to NyteKnight/hadoop that referenced this pull request Jun 25, 2024

HDFS-17026. RBF: NamenodeHeartbeatService should update JMX report wi…

0336d1f

…th configurable frequency. (apache#5691). Contributed by hchaverri. (apache#205)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HDFS-17026. RBF: NamenodeHeartbeatService should update JMX report with configurable frequency #5691

HDFS-17026. RBF: NamenodeHeartbeatService should update JMX report with configurable frequency #5691

hchaverri commented May 26, 2023

hadoop-yetus commented May 26, 2023

Hexiaoqiao left a comment

hadoop-yetus commented May 26, 2023

hadoop-yetus commented May 26, 2023

hadoop-yetus commented May 26, 2023

simbadzina left a comment

hadoop-yetus commented May 27, 2023

hadoop-yetus commented May 29, 2023

yzhang559 left a comment

hchaverri commented May 30, 2023

Hexiaoqiao left a comment

Hexiaoqiao commented May 31, 2023

HDFS-17026. RBF: NamenodeHeartbeatService should update JMX report with configurable frequency #5691

HDFS-17026. RBF: NamenodeHeartbeatService should update JMX report with configurable frequency #5691

Conversation

hchaverri commented May 26, 2023

Description of PR

How was this patch tested?

For code changes:

hadoop-yetus commented May 26, 2023

Hexiaoqiao left a comment

Choose a reason for hiding this comment

hadoop-yetus commented May 26, 2023

hadoop-yetus commented May 26, 2023

hadoop-yetus commented May 26, 2023

simbadzina left a comment

Choose a reason for hiding this comment

hadoop-yetus commented May 27, 2023

hadoop-yetus commented May 29, 2023

yzhang559 left a comment

Choose a reason for hiding this comment

hchaverri commented May 30, 2023

Hexiaoqiao left a comment

Choose a reason for hiding this comment

Hexiaoqiao commented May 31, 2023