HDDS-1449. JVM Exit in datanode while committing a key. Contributed by Mukul Kumar Singh.

[mukul1987]

https://issues.apache.org/jira/browse/HDDS-1449

The JVM exit was happening because the db instance was being closed while the other thread was adding a key to the db.

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	27	Docker mode activated.
		_ Prechecks _
+1	dupname	1	No case conflicting files found.
+1	@author	0	The patch does not contain any @author tags.
+1	test4tests	0	The patch appears to include 9 new or modified test files.
		_ trunk Compile Tests _
0	mvndep	64	Maven dependency ordering for branch
+1	mvninstall	405	trunk passed
+1	compile	199	trunk passed
+1	checkstyle	51	trunk passed
+1	mvnsite	0	trunk passed
+1	shadedclient	812	branch has no errors when building and testing our client artifacts.
+1	javadoc	112	trunk passed
0	spotbugs	242	Used deprecated FindBugs config; considering switching to SpotBugs.
+1	findbugs	426	trunk passed
		_ Patch Compile Tests _
0	mvndep	24	Maven dependency ordering for patch
+1	mvninstall	388	the patch passed
+1	compile	194	the patch passed
+1	javac	194	the patch passed
-0	checkstyle	26	hadoop-hdds: The patch generated 12 new + 0 unchanged - 0 fixed = 12 total (was 0)
-0	checkstyle	23	hadoop-ozone: The patch generated 6 new + 0 unchanged - 0 fixed = 6 total (was 0)
+1	mvnsite	0	the patch passed
+1	whitespace	0	The patch has no whitespace issues.
+1	shadedclient	623	patch has no errors when building and testing our client artifacts.
+1	javadoc	121	the patch passed
+1	findbugs	423	the patch passed
		_ Other Tests _
-1	unit	144	hadoop-hdds in the patch failed.
-1	unit	1435	hadoop-ozone in the patch failed.
+1	asflicense	39	The patch does not generate ASF License warnings.
		5681

Reason	Tests
Failed junit tests	hadoop.ozone.container.common.statemachine.commandhandler.TestCloseContainerCommandHandler
	hadoop.ozone.client.rpc.TestContainerStateMachineFailures
	hadoop.ozone.TestStorageContainerManager
	hadoop.ozone.client.rpc.TestBlockOutputStreamWithFailures
	hadoop.ozone.container.common.impl.TestContainerPersistence

Subsystem	Report/Notes
Docker	Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-825/1/artifact/out/Dockerfile
GITHUB PR	#825
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux 806581a933c1 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / de01422
Default Java	1.8.0_212
checkstyle	https://builds.apache.org/job/hadoop-multibranch/job/PR-825/1/artifact/out/diff-checkstyle-hadoop-hdds.txt
checkstyle	https://builds.apache.org/job/hadoop-multibranch/job/PR-825/1/artifact/out/diff-checkstyle-hadoop-ozone.txt
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-825/1/artifact/out/patch-unit-hadoop-hdds.txt
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-825/1/artifact/out/patch-unit-hadoop-ozone.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-825/1/testReport/
Max. process+thread count	4695 (vs. ulimit of 5500)
modules	C: hadoop-hdds/container-service hadoop-ozone/integration-test U: .
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-825/1/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

[bshashikant]

Thanks @mukul1987 for working on this. The patch overall looks good to me. Some comments :

1. When want to force evict the container cache, it should ideally not check for the reference count to be 0.
2. We should have two methods, one to actually evict the cache forcefully which will not validate the the reference count to be 0 and one which actually validates the reference count to be 0 before removing.

[mukul1987]

Thanks for the review @bshashikant.
I have changed the name of the variable to checkNoReferences in place of force. As there are no users of the force flag, I feel we can address that later.

[bshashikant]

Thanks @mukul1987 for updating the patch. The patch looks good to me but TestContainerPersistence#testDeleteBlockTwice failure seems to be related. Can you please check?

[bshashikant]

The test fails locally and will be tracked by a different jira. This change looks good to me. +1.