HDFS-15496. Add UI for deleted snapshots #2212

vivekratnavel · 2020-08-11T02:42:36Z

This patch adds a few additional columns to snapshots table in Namenode UI including the snapshot deletion status as shown in the screen-shot below.

https://issues.apache.org/jira/browse/HDFS-15496

bshashikant · 2020-08-11T07:18:39Z

Thanks @vivekratnavel for putting up the patch. The patch in general looks good. Some comments inline:

Since the patch modifies SnapshotInfo class, let's remove SnapshotStatus.Bean()
Having different column for snapshotName and then snapshot path may not be useful. Instead can we just have one column for the snapshot path (snapshotName is implicit).
Snapshot permission, owner and group added newly to the UI page .. Any specific reason?

hadoop-yetus · 2020-08-11T08:53:50Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	31m 54s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
		_ trunk Compile Tests _
+1 💚	mvninstall	29m 4s	trunk passed
+1 💚	compile	1m 17s	trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	compile	1m 10s	trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+1 💚	checkstyle	0m 49s	trunk passed
+1 💚	mvnsite	1m 17s	trunk passed
+1 💚	shadedclient	16m 20s	branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 51s	trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	javadoc	1m 21s	trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+0 🆗	spotbugs	2m 57s	Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	2m 56s	trunk passed
		_ Patch Compile Tests _
+1 💚	mvninstall	1m 7s	the patch passed
+1 💚	compile	1m 9s	the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	javac	1m 9s	the patch passed
+1 💚	compile	1m 2s	the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+1 💚	javac	1m 2s	the patch passed
-0 ⚠️	checkstyle	0m 39s	hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 11 unchanged - 0 fixed = 12 total (was 11)
+1 💚	mvnsite	1m 7s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	shadedclient	13m 53s	patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 47s	the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	javadoc	1m 20s	the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+1 💚	findbugs	2m 56s	the patch passed
		_ Other Tests _
-1 ❌	unit	121m 54s	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 42s	The patch does not generate ASF License warnings.
		235m 34s

Reason	Tests
Failed junit tests	hadoop.fs.contract.hdfs.TestHDFSContractMultipartUploader
	hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped
	hadoop.hdfs.TestGetFileChecksum

Subsystem	Report/Notes
Docker	ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2212/1/artifact/out/Dockerfile
GITHUB PR	#2212
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux 7d2205311312 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `32895f4`
Default Java	Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
checkstyle	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2212/1/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
unit	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2212/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2212/1/testReport/
Max. process+thread count	3755 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2212/1/console
versions	git=2.17.1 maven=3.6.0 findbugs=4.0.6
Powered by	Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

bharatviswa504

+1 LGTM.

vivekratnavel · 2020-08-11T20:58:08Z

@bshashikant @bharatviswa504 Thanks for the reviews!

Since the patch modifies SnapshotInfo class, let's remove SnapshotStatus.Bean()

Done

Having different column for snapshotName and then snapshot path may not be useful. Instead can we just have one column for the snapshot path (snapshotName is implicit).

Done

Snapshot permission, owner and group added newly to the UI page .. Any specific reason?

I added these new columns to be consistent with the display of snapshottable directories table and to provide more useful information about snapshots to the user.

Please take another look at the updated patch. Thanks!

vivekratnavel · 2020-08-11T20:59:31Z

Screen-shot of UI with the latest patch is shown below:

hadoop-yetus · 2020-08-12T01:00:31Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 6s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
		_ trunk Compile Tests _
+0 🆗	mvndep	3m 14s	Maven dependency ordering for branch
+1 💚	mvninstall	28m 10s	trunk passed
+1 💚	compile	4m 18s	trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	compile	3m 49s	trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09
+1 💚	checkstyle	0m 58s	trunk passed
+1 💚	mvnsite	2m 5s	trunk passed
+1 💚	shadedclient	18m 49s	branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	1m 25s	trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	javadoc	1m 52s	trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09
+0 🆗	spotbugs	2m 26s	Used deprecated FindBugs config; considering switching to SpotBugs.
-1 ❌	findbugs	3m 7s	hadoop-hdfs-project/hadoop-hdfs in trunk has 2 extant findbugs warnings.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 22s	Maven dependency ordering for patch
+1 💚	mvninstall	1m 54s	the patch passed
+1 💚	compile	4m 10s	the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	javac	4m 10s	the patch passed
+1 💚	compile	3m 42s	the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09
+1 💚	javac	3m 42s	the patch passed
+1 💚	checkstyle	0m 50s	the patch passed
+1 💚	mvnsite	1m 54s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	shadedclient	15m 24s	patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	1m 18s	the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	javadoc	1m 49s	the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09
+1 💚	findbugs	5m 47s	the patch passed
		_ Other Tests _
+1 💚	unit	1m 59s	hadoop-hdfs-client in the patch passed.
-1 ❌	unit	108m 40s	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 37s	The patch does not generate ASF License warnings.
		220m 41s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.sps.TestExternalStoragePolicySatisfier
	hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped
	hadoop.hdfs.TestMultipleNNPortQOP
	hadoop.fs.contract.hdfs.TestHDFSContractMultipartUploader

Subsystem	Report/Notes
Docker	ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2212/2/artifact/out/Dockerfile
GITHUB PR	#2212
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux 27fff0c16184 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `3fd3aeb`
Default Java	Private Build-1.8.0_252-8u252-b09-1~18.04-b09
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_252-8u252-b09-1~18.04-b09
findbugs	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2212/2/artifact/out/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
unit	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2212/2/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2212/2/testReport/
Max. process+thread count	2861 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2212/2/console
versions	git=2.17.1 maven=3.6.0 findbugs=4.0.6
Powered by	Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

bshashikant · 2020-08-12T10:43:03Z

Let's remove the permission/owner/group column as these are not really required to show.

bshashikant

Will the snapshots will be first grouped by the snapshottable directory and then sorted on the snapshot Ids always ?

vivekratnavel · 2020-08-12T20:31:44Z

In the latest patch, I removed the permission/owner/group columns. I also verified that the snapshots will be grouped by snapshottable directory and then sorted on the snapshot Ids always.

hadoop-yetus · 2020-08-13T00:46:08Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 15s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
		_ trunk Compile Tests _
+0 🆗	mvndep	3m 19s	Maven dependency ordering for branch
+1 💚	mvninstall	30m 20s	trunk passed
+1 💚	compile	5m 46s	trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	compile	4m 55s	trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09
+1 💚	checkstyle	1m 6s	trunk passed
+1 💚	mvnsite	2m 27s	trunk passed
+1 💚	shadedclient	19m 14s	branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	1m 25s	trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	javadoc	1m 52s	trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09
+0 🆗	spotbugs	2m 28s	Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	5m 35s	trunk passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 22s	Maven dependency ordering for patch
+1 💚	mvninstall	1m 53s	the patch passed
+1 💚	compile	4m 10s	the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	javac	4m 10s	the patch passed
+1 💚	compile	3m 45s	the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09
+1 💚	javac	3m 45s	the patch passed
+1 💚	checkstyle	0m 51s	the patch passed
+1 💚	mvnsite	1m 55s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	shadedclient	15m 28s	patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	1m 18s	the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	javadoc	1m 45s	the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09
+1 💚	findbugs	5m 41s	the patch passed
		_ Other Tests _
+1 💚	unit	1m 57s	hadoop-hdfs-client in the patch passed.
-1 ❌	unit	108m 49s	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 36s	The patch does not generate ASF License warnings.
		226m 13s

Reason	Tests
Failed junit tests	hadoop.hdfs.TestStripedFileAppend
	hadoop.fs.contract.hdfs.TestHDFSContractMultipartUploader
	hadoop.hdfs.TestRollingUpgrade

Subsystem	Report/Notes
Docker	ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2212/3/artifact/out/Dockerfile
GITHUB PR	#2212
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux 815c830f6860 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `e592ec5`
Default Java	Private Build-1.8.0_252-8u252-b09-1~18.04-b09
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_252-8u252-b09-1~18.04-b09
unit	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2212/3/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2212/3/testReport/
Max. process+thread count	2796 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2212/3/console
versions	git=2.17.1 maven=3.6.0 findbugs=4.0.6
Powered by	Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

(cherry picked from commit cb50e3f) Change-Id: I7bd4ee15468f0c36c989ba91c16fe97d6cc6d08e (cherry picked from commit d0b27ff)

HDFS-15496. Add UI for deleted snapshots

dc12dce

vivekratnavel requested review from mukul1987, bharatviswa504 and bshashikant August 11, 2020 02:42

bharatviswa504 approved these changes Aug 11, 2020

View reviewed changes

Address review comments

fc9dead

bshashikant reviewed Aug 12, 2020

View reviewed changes

Address review comments

4b63f62

bshashikant approved these changes Aug 13, 2020

View reviewed changes

bshashikant merged commit cb50e3f into apache:trunk Aug 13, 2020

jojochuang pushed a commit to jojochuang/hadoop that referenced this pull request May 23, 2023

HDFS-15496. Add UI for deleted snapshots (apache#2212)

4877e03

(cherry picked from commit cb50e3f) Change-Id: I7bd4ee15468f0c36c989ba91c16fe97d6cc6d08e (cherry picked from commit d0b27ff)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HDFS-15496. Add UI for deleted snapshots #2212

HDFS-15496. Add UI for deleted snapshots #2212

vivekratnavel commented Aug 11, 2020

bshashikant commented Aug 11, 2020

hadoop-yetus commented Aug 11, 2020

bharatviswa504 left a comment

vivekratnavel commented Aug 11, 2020

vivekratnavel commented Aug 11, 2020

hadoop-yetus commented Aug 12, 2020

bshashikant commented Aug 12, 2020

bshashikant left a comment

vivekratnavel commented Aug 12, 2020

hadoop-yetus commented Aug 13, 2020

HDFS-15496. Add UI for deleted snapshots #2212

HDFS-15496. Add UI for deleted snapshots #2212

Conversation

vivekratnavel commented Aug 11, 2020

bshashikant commented Aug 11, 2020

hadoop-yetus commented Aug 11, 2020

bharatviswa504 left a comment

Choose a reason for hiding this comment

vivekratnavel commented Aug 11, 2020

vivekratnavel commented Aug 11, 2020

hadoop-yetus commented Aug 12, 2020

bshashikant commented Aug 12, 2020

bshashikant left a comment

Choose a reason for hiding this comment

vivekratnavel commented Aug 12, 2020

hadoop-yetus commented Aug 13, 2020