HDFS-16461. Expose JournalNode storage info in the jmx metrics #4002

virajjasani · 2022-02-20T05:58:33Z

Description of PR

We should expose the list of storage info of JournalNode's journals (including layout version, namespace id, cluster id and creation time of the FS state) in the jmx metrics.

How was this patch tested?

Local dev cluster and UT.

Updated screenshot:

For code changes:

Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?

hadoop-yetus · 2022-02-20T13:37:26Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 52s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	35m 2s		trunk passed
+1 💚	compile	1m 30s		trunk passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	compile	1m 21s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	checkstyle	0m 58s		trunk passed
+1 💚	mvnsite	1m 26s		trunk passed
+1 💚	javadoc	1m 2s		trunk passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	1m 29s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	3m 23s		trunk passed
+1 💚	shadedclient	25m 43s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 19s		the patch passed
+1 💚	compile	1m 28s		the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	javac	1m 28s		the patch passed
+1 💚	compile	1m 16s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	javac	1m 16s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 51s	/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 6 unchanged - 1 fixed = 7 total (was 7)
+1 💚	mvnsite	1m 20s		the patch passed
+1 💚	javadoc	0m 53s		the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	1m 23s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	3m 26s		the patch passed
+1 💚	shadedclient	25m 41s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	348m 38s		hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 38s		The patch does not generate ASF License warnings.
		456m 56s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4002/1/artifact/out/Dockerfile
GITHUB PR	#4002
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 276fa69bac8a 4.15.0-153-generic #160-Ubuntu SMP Thu Jul 29 06:54:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `17c215b`
Default Java	Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4002/1/testReport/
Max. process+thread count	2645 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4002/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-02-20T21:53:25Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 51s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	35m 6s		trunk passed
+1 💚	compile	1m 30s		trunk passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	compile	1m 21s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	checkstyle	0m 57s		trunk passed
+1 💚	mvnsite	1m 29s		trunk passed
+1 💚	javadoc	1m 2s		trunk passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	1m 27s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	3m 25s		trunk passed
+1 💚	shadedclient	25m 45s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 19s		the patch passed
+1 💚	compile	1m 23s		the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	javac	1m 23s		the patch passed
+1 💚	compile	1m 15s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	javac	1m 15s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 51s		hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 6 unchanged - 1 fixed = 6 total (was 7)
+1 💚	mvnsite	1m 22s		the patch passed
+1 💚	javadoc	0m 54s		the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	1m 21s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	3m 28s		the patch passed
+1 💚	shadedclient	26m 21s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	353m 55s		hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 38s		The patch does not generate ASF License warnings.
		462m 35s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4002/2/artifact/out/Dockerfile
GITHUB PR	#4002
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 9f187681c88c 4.15.0-153-generic #160-Ubuntu SMP Thu Jul 29 06:54:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `19ee699`
Default Java	Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4002/2/testReport/
Max. process+thread count	1964 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4002/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

tasanuma · 2022-02-21T00:06:05Z

How about using HashMap to shape it like getJournalsStatus()?

virajjasani · 2022-02-21T07:06:54Z

Thanks @tasanuma, updated PR with your suggestion, updated screenshot also in the description.

tasanuma · 2022-02-21T10:39:42Z

+1, pending Jenkins.

hadoop-yetus · 2022-02-21T14:50:58Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 52s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 1s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	35m 50s		trunk passed
+1 💚	compile	1m 32s		trunk passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	compile	1m 23s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	checkstyle	0m 57s		trunk passed
+1 💚	mvnsite	1m 28s		trunk passed
+1 💚	javadoc	1m 1s		trunk passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	1m 31s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	3m 24s		trunk passed
+1 💚	shadedclient	25m 42s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 22s		the patch passed
+1 💚	compile	1m 28s		the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	javac	1m 28s		the patch passed
+1 💚	compile	1m 16s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	javac	1m 16s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 51s		hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 21 unchanged - 1 fixed = 21 total (was 22)
+1 💚	mvnsite	1m 22s		the patch passed
+1 💚	javadoc	0m 53s		the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	1m 24s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	3m 25s		the patch passed
+1 💚	shadedclient	25m 21s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	355m 37s		hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 37s		The patch does not generate ASF License warnings.
		464m 21s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4002/3/artifact/out/Dockerfile
GITHUB PR	#4002
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux b649b40e070d 4.15.0-153-generic #160-Ubuntu SMP Thu Jul 29 06:54:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `d6d4a9e`
Default Java	Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4002/3/testReport/
Max. process+thread count	1902 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4002/3/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

tasanuma · 2022-02-22T04:04:48Z

Thanks for your contribution, @virajjasani.

…e#4002)

HDFS-16461. Expose JournalNode storage info in the jmx metrics

17c215b

fix checkstyle

19ee699

addressing review by @tasanuma

d6d4a9e

tasanuma approved these changes Feb 22, 2022

View reviewed changes

tasanuma merged commit e363f51 into apache:trunk Feb 22, 2022

HarshitGupta11 pushed a commit to HarshitGupta11/hadoop that referenced this pull request Nov 28, 2022

HDFS-16461. Expose JournalNode storage info in the jmx metrics (apach…

9d5fcc7

…e#4002)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HDFS-16461. Expose JournalNode storage info in the jmx metrics #4002

HDFS-16461. Expose JournalNode storage info in the jmx metrics #4002

virajjasani commented Feb 20, 2022 •

edited

Loading

hadoop-yetus commented Feb 20, 2022

hadoop-yetus commented Feb 20, 2022

tasanuma commented Feb 21, 2022

virajjasani commented Feb 21, 2022

tasanuma commented Feb 21, 2022

hadoop-yetus commented Feb 21, 2022

tasanuma commented Feb 22, 2022

HDFS-16461. Expose JournalNode storage info in the jmx metrics #4002

HDFS-16461. Expose JournalNode storage info in the jmx metrics #4002

Conversation

virajjasani commented Feb 20, 2022 • edited Loading

Description of PR

How was this patch tested?

For code changes:

hadoop-yetus commented Feb 20, 2022

hadoop-yetus commented Feb 20, 2022

tasanuma commented Feb 21, 2022

virajjasani commented Feb 21, 2022

tasanuma commented Feb 21, 2022

hadoop-yetus commented Feb 21, 2022

tasanuma commented Feb 22, 2022

virajjasani commented Feb 20, 2022 •

edited

Loading