HBASE-24117 Shutdown AssignmentManager before ProcedureExecutor may c… #1865

Apache9 · 2020-06-08T09:25:38Z

…ause SCP to accidentally skip assigning a region

Apache-HBase · 2020-06-08T10:07:58Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 31s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	3m 43s	master passed
+1 💚	checkstyle	1m 10s	master passed
+1 💚	spotbugs	2m 6s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 27s	the patch passed
+1 💚	checkstyle	1m 4s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	11m 7s	Patch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚	spotbugs	2m 11s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 13s	The patch does not generate ASF License warnings.
		32m 39s

Subsystem	Report/Notes
Docker	Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1865/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#1865
Optional Tests	dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle
uname	Linux 480e55a470be 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `fbf0337`
Max. process+thread count	94 (vs. ulimit of 12500)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1865/1/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-06-08T12:16:19Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 56s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	5m 9s	master passed
+1 💚	compile	1m 15s	master passed
+1 💚	shadedjars	6m 42s	branch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 45s	hbase-server in master failed.
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 24s	the patch passed
+1 💚	compile	1m 17s	the patch passed
+1 💚	javac	1m 17s	the patch passed
+1 💚	shadedjars	7m 9s	patch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 44s	hbase-server in the patch failed.
		_ Other Tests _
+1 💚	unit	130m 19s	hbase-server in the patch passed.
		160m 49s

Subsystem	Report/Notes
Docker	Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1865/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#1865
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux dc0ce9231e58 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `fbf0337`
Default Java	2020-01-14
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1865/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1865/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1865/1/testReport/
Max. process+thread count	4379 (vs. ulimit of 12500)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1865/1/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-06-08T13:39:21Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 54s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	5m 5s	master passed
+1 💚	compile	1m 11s	master passed
+1 💚	shadedjars	7m 34s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 50s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 49s	the patch passed
+1 💚	compile	1m 10s	the patch passed
+1 💚	javac	1m 10s	the patch passed
+1 💚	shadedjars	6m 47s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 48s	the patch passed
		_ Other Tests _
+1 💚	unit	211m 44s	hbase-server in the patch passed.
		243m 57s

Subsystem	Report/Notes
Docker	Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1865/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#1865
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux e0132d320621 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `fbf0337`
Default Java	1.8.0_232
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1865/1/testReport/
Max. process+thread count	3692 (vs. ulimit of 12500)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1865/1/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

saintstack

Some nits you might want to fix otherwise, lets try it. Do against branch-2 because there we'll notice if the test fails since it stable.

hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java

saintstack · 2020-06-08T23:41:30Z

hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java

@@ -485,6 +486,12 @@ private void assignRegions(MasterProcedureEnv env, List<RegionInfo> regions) thr
        // UPDATE: HBCKServerCrashProcedure overrides isMatchingRegionLocation; this check can get
        // in the way of our clearing out 'Unknown Servers'.
        if (!isMatchingRegionLocation(regionNode)) {
+          // See HBASE-24117, though we have already changed the shutdown order, it is still worth
+          // double checking here to confirm that we do not skip assignment incorrectly.
+          if (!am.isRunning()) {


…ause SCP to accidentally skip assigning a region

Apache-HBase · 2020-06-09T02:38:05Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 29s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	3m 46s	master passed
+1 💚	checkstyle	1m 9s	master passed
+1 💚	spotbugs	2m 0s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 21s	the patch passed
+1 💚	checkstyle	1m 7s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	11m 14s	Patch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚	spotbugs	2m 6s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 15s	The patch does not generate ASF License warnings.
		32m 41s

Subsystem	Report/Notes
Docker	Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1865/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#1865
Optional Tests	dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle
uname	Linux 15099d0fc194 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `2131456`
Max. process+thread count	94 (vs. ulimit of 12500)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1865/2/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

…ause SCP to accidentally skip assigning a region (#1865) Signed-off-by: Michael Stack <stack@apache.org>

Apache-HBase · 2020-06-09T04:39:22Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 33s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 18s	master passed
+1 💚	compile	1m 7s	master passed
+1 💚	shadedjars	6m 26s	branch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 44s	hbase-server in master failed.
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 49s	the patch passed
+1 💚	compile	1m 16s	the patch passed
+1 💚	javac	1m 16s	the patch passed
+1 💚	shadedjars	5m 44s	patch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 39s	hbase-server in the patch failed.
		_ Other Tests _
+1 💚	unit	126m 14s	hbase-server in the patch passed.
		153m 56s

Subsystem	Report/Notes
Docker	Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1865/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#1865
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux af02f70ea5bf 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `2131456`
Default Java	2020-01-14
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1865/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1865/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1865/2/testReport/
Max. process+thread count	4126 (vs. ulimit of 12500)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1865/2/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-06-09T05:51:26Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 11s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 4s	master passed
+1 💚	compile	0m 55s	master passed
+1 💚	shadedjars	6m 3s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 37s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 49s	the patch passed
+1 💚	compile	0m 57s	the patch passed
+1 💚	javac	0m 57s	the patch passed
+1 💚	shadedjars	6m 5s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 34s	the patch passed
		_ Other Tests _
+1 💚	unit	199m 54s	hbase-server in the patch passed.
		226m 5s

Subsystem	Report/Notes
Docker	Client=19.03.11 Server=19.03.11 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1865/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#1865
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 1bbcfc0f6769 4.15.0-91-generic #92-Ubuntu SMP Fri Feb 28 11:09:48 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `2131456`
Default Java	1.8.0_232
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1865/2/testReport/
Max. process+thread count	3639 (vs. ulimit of 12500)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1865/2/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

…ause SCP to accidentally skip assigning a region (apache#1865) Signed-off-by: Michael Stack <stack@apache.org>

…ause SCP to accidentally skip assigning a region (apache#1865) Author: Duo Zhang Reason: Bug Ref: CDPD-15964 Signed-off-by: Michael Stack <stack@apache.org> Change-Id: I6bb4b16b68ea8c9e5baa39aa719447268da24c5c (cherry picked from commit 3419317)

…ause SCP to accidentally skip assigning a region (apache#1865) Signed-off-by: Michael Stack <stack@apache.org> (cherry picked from commit dee9933) Change-Id: I6bb4b16b68ea8c9e5baa39aa719447268da24c5c

Apache9 requested a review from saintstack June 8, 2020 09:25

Apache9 self-assigned this Jun 8, 2020

Apache9 added the bug label Jun 8, 2020

saintstack approved these changes Jun 8, 2020

View reviewed changes

HBASE-24117 Shutdown AssignmentManager before ProcedureExecutor may c…

3d3edeb

…ause SCP to accidentally skip assigning a region

Apache9 force-pushed the HBASE-24117 branch from fa2a071 to 3d3edeb Compare June 9, 2020 02:00

Apache9 merged commit 3d50e73 into apache:master Jun 9, 2020

asfgit pushed a commit that referenced this pull request Jun 9, 2020

HBASE-24117 Shutdown AssignmentManager before ProcedureExecutor may c…

dee9933

…ause SCP to accidentally skip assigning a region (#1865) Signed-off-by: Michael Stack <stack@apache.org>

asfgit pushed a commit that referenced this pull request Jun 9, 2020

HBASE-24117 Shutdown AssignmentManager before ProcedureExecutor may c…

4486a56

…ause SCP to accidentally skip assigning a region (#1865) Signed-off-by: Michael Stack <stack@apache.org>

asfgit pushed a commit that referenced this pull request Jun 9, 2020

HBASE-24117 Shutdown AssignmentManager before ProcedureExecutor may c…

c5dacfb

…ause SCP to accidentally skip assigning a region (#1865) Signed-off-by: Michael Stack <stack@apache.org>

HomminLee pushed a commit to HomminLee/hbase that referenced this pull request Jun 12, 2020

HBASE-24117 Shutdown AssignmentManager before ProcedureExecutor may c…

d4dfa77

…ause SCP to accidentally skip assigning a region (apache#1865) Signed-off-by: Michael Stack <stack@apache.org>

infraio pushed a commit to infraio/hbase that referenced this pull request Aug 17, 2020

HBASE-24117 Shutdown AssignmentManager before ProcedureExecutor may c…

85362f6

…ause SCP to accidentally skip assigning a region (apache#1865) Signed-off-by: Michael Stack <stack@apache.org>

clarax pushed a commit to clarax/hbase that referenced this pull request Nov 15, 2020

HBASE-24117 Shutdown AssignmentManager before ProcedureExecutor may c…

231923d

…ause SCP to accidentally skip assigning a region (apache#1865) Signed-off-by: Michael Stack <stack@apache.org>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HBASE-24117 Shutdown AssignmentManager before ProcedureExecutor may c… #1865

HBASE-24117 Shutdown AssignmentManager before ProcedureExecutor may c… #1865

Apache9 commented Jun 8, 2020

Apache-HBase commented Jun 8, 2020

Apache-HBase commented Jun 8, 2020

Apache-HBase commented Jun 8, 2020

saintstack left a comment

saintstack Jun 8, 2020

Apache-HBase commented Jun 9, 2020

Apache-HBase commented Jun 9, 2020

Apache-HBase commented Jun 9, 2020

HBASE-24117 Shutdown AssignmentManager before ProcedureExecutor may c… #1865

HBASE-24117 Shutdown AssignmentManager before ProcedureExecutor may c… #1865

Conversation

Apache9 commented Jun 8, 2020

Apache-HBase commented Jun 8, 2020

Apache-HBase commented Jun 8, 2020

Apache-HBase commented Jun 8, 2020

saintstack left a comment

Choose a reason for hiding this comment

saintstack Jun 8, 2020

Choose a reason for hiding this comment

Apache-HBase commented Jun 9, 2020

Apache-HBase commented Jun 9, 2020

Apache-HBase commented Jun 9, 2020