HDFS-15590. namenode fails to start when ordered snapshot deletion feature is disabled #2326

bshashikant · 2020-09-22T06:16:51Z

please see https://issues.apache.org/jira/browse/HDFS-15590

…ature is disabled.

hadoop-yetus · 2020-09-22T09:29:59Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 39s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ trunk Compile Tests _
+1 💚	mvninstall	34m 34s	trunk passed
+1 💚	compile	1m 40s	trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	compile	1m 16s	trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+1 💚	checkstyle	0m 48s	trunk passed
+1 💚	mvnsite	1m 21s	trunk passed
+1 💚	shadedclient	19m 19s	branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 51s	trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	javadoc	1m 30s	trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+0 🆗	spotbugs	3m 17s	Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	3m 15s	trunk passed
		_ Patch Compile Tests _
+1 💚	mvninstall	1m 14s	the patch passed
+1 💚	compile	1m 12s	the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	javac	1m 12s	the patch passed
+1 💚	compile	1m 2s	the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+1 💚	javac	1m 2s	the patch passed
-0 ⚠️	checkstyle	0m 43s	hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 20 unchanged - 0 fixed = 21 total (was 20)
+1 💚	mvnsite	1m 8s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	shadedclient	14m 2s	patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 47s	the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	javadoc	1m 23s	the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+1 💚	findbugs	3m 6s	the patch passed
		_ Other Tests _
-1 ❌	unit	97m 15s	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 42s	The patch does not generate ASF License warnings.
		190m 42s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.namenode.TestNameNodeRetryCacheMetrics
	hadoop.hdfs.server.namenode.TestDecommissioningStatus
	hadoop.hdfs.server.sps.TestExternalStoragePolicySatisfier
	hadoop.hdfs.TestSnapshotCommands
	hadoop.hdfs.web.TestWebHDFS
	hadoop.hdfs.qjournal.server.TestJournalNodeSync
	hadoop.hdfs.TestFileChecksumCompositeCrc
	hadoop.hdfs.TestFileChecksum

Subsystem	Report/Notes
Docker	ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2326/1/artifact/out/Dockerfile
GITHUB PR	#2326
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux 1286642aaefc 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `6b5d9e2`
Default Java	Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
checkstyle	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2326/1/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
unit	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2326/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2326/1/testReport/
Max. process+thread count	4242 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2326/1/console
versions	git=2.17.1 maven=3.6.0 findbugs=4.0.6
Powered by	Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

szetszwo

How about failing the second user delete request with a "Already marked as deleted" exception instead of changing the edit log loading? It is hard for editlog loading to guess if the command is valid.

bshashikant · 2020-09-23T06:01:42Z

How about failing the second user delete request with a "Already marked as deleted" exception instead of changing the edit log loading? It is hard for editlog loading to guess if the command is valid.

Thanks @szetszwo . The problem is just not with user deletes. Let's see a sequence like this:

Let's say we have 2 snapshots s1 and S2 after enabling ordered snapshot deletion
User deleted S2 ---> creates an edit log entry
User deleted S1
Snapshot Deletion GC thread will now delete S2 ---> creates an edit log entry again
Now turn off Ordered Snapshot Deletion and Restart

It will run into the same problem again. We cannot seem to avoid 2 edit log entries for same snapshot delete with Snapshot GC thread running and deleting the snapshot.

szetszwo

@bshashikant , you are right that failing the second delete won't work. Thanks.

+1 the change looks good.

hadoop-yetus · 2020-09-23T12:07:26Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 28s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+1 💚	@author	0m 1s		The patch does not contain any @author tags.
+1 💚		0m 0s	test4tests	The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	29m 57s		trunk passed
+1 💚	compile	1m 17s		trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	compile	1m 11s		trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+1 💚	checkstyle	0m 49s		trunk passed
+1 💚	mvnsite	1m 17s		trunk passed
+1 💚	shadedclient	16m 55s		branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 54s		trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	javadoc	1m 26s		trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+0 🆗	spotbugs	3m 24s		Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	3m 22s		trunk passed
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 9s		the patch passed
+1 💚	compile	1m 8s		the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	javac	1m 8s		the patch passed
+1 💚	compile	1m 0s		the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+1 💚	javac	1m 0s		the patch passed
+1 💚	checkstyle	0m 40s		the patch passed
+1 💚	mvnsite	1m 9s		the patch passed
+1 💚	whitespace	0m 0s		The patch has no whitespace issues.
+1 💚	shadedclient	15m 23s		patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 50s		the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	javadoc	1m 26s		the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+1 💚	findbugs	3m 39s		the patch passed
			_ Other Tests _
-1 ❌	unit	104m 4s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 38s		The patch does not generate ASF License warnings.
		190m 45s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.datanode.TestDataNodeUUID
	hadoop.hdfs.server.datanode.TestBPOfferService
	hadoop.hdfs.server.namenode.ha.TestHAAppend
	hadoop.hdfs.server.blockmanagement.TestReconstructStripedBlocksWithRackAwareness
	hadoop.hdfs.TestFileChecksumCompositeCrc
	hadoop.hdfs.server.balancer.TestBalancerRPCDelay
	hadoop.hdfs.server.balancer.TestBalancer
	hadoop.hdfs.server.namenode.ha.TestPipelinesFailover
	hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery
	hadoop.hdfs.TestFileChecksum

Subsystem	Report/Notes
Docker	ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2326/2/artifact/out/Dockerfile
GITHUB PR	#2326
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux 9fbfa264c0e2 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `18f9d77`
Default Java	Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2326/2/testReport/
Max. process+thread count	4173 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2326/2/console
versions	git=2.17.1 maven=3.6.0 findbugs=4.0.6
Powered by	Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

bshashikant · 2020-09-24T08:31:14Z

Thanks @szetszwo for the review. I have committed this.

…ature is disabled (apache#2326)

…ature is disabled (apache#2326) (cherry picked from commit 368f2f6) Change-Id: I78a945efa6747c34e20cf8d2d41d5f89546eebe0

HDFS-15590. namenode fails to start when ordered snapshot deletion fe…

d896e8b

…ature is disabled.

bshashikant requested review from szetszwo and mukul1987 September 22, 2020 06:16

szetszwo reviewed Sep 23, 2020

View reviewed changes

szetszwo approved these changes Sep 23, 2020

View reviewed changes

Addressed unit test failure TestSnapshotCommands#testMaxSnapshotLimit

bb28758

bshashikant merged commit 368f2f6 into apache:trunk Sep 24, 2020

bshashikant deleted the HDFS-15590 branch September 24, 2020 08:31

bilaharith pushed a commit to bilaharith/hadoop that referenced this pull request Sep 27, 2020

HDFS-15590. namenode fails to start when ordered snapshot deletion fe…

51b5fcf

…ature is disabled (apache#2326)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HDFS-15590. namenode fails to start when ordered snapshot deletion feature is disabled #2326

HDFS-15590. namenode fails to start when ordered snapshot deletion feature is disabled #2326

bshashikant commented Sep 22, 2020

hadoop-yetus commented Sep 22, 2020

szetszwo left a comment

bshashikant commented Sep 23, 2020

szetszwo left a comment

hadoop-yetus commented Sep 23, 2020

bshashikant commented Sep 24, 2020

HDFS-15590. namenode fails to start when ordered snapshot deletion feature is disabled #2326

HDFS-15590. namenode fails to start when ordered snapshot deletion feature is disabled #2326

Conversation

bshashikant commented Sep 22, 2020

hadoop-yetus commented Sep 22, 2020

szetszwo left a comment

Choose a reason for hiding this comment

bshashikant commented Sep 23, 2020

szetszwo left a comment

Choose a reason for hiding this comment

hadoop-yetus commented Sep 23, 2020

bshashikant commented Sep 24, 2020