Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HBASE-23322 [hbck2] Simplification on HBCKSCP scheduling #855

Closed
wants to merge 1 commit into from
Closed

HBASE-23322 [hbck2] Simplification on HBCKSCP scheduling #855

wants to merge 1 commit into from

Conversation

saintstack
Copy link
Contributor

No description provided.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 54s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ branch-2 Compile Tests _
+1 💚 mvninstall 6m 16s branch-2 passed
+1 💚 compile 1m 5s branch-2 passed
+1 💚 checkstyle 1m 40s branch-2 passed
+1 💚 shadedjars 5m 22s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 43s branch-2 passed
+0 🆗 spotbugs 3m 57s Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚 findbugs 3m 54s branch-2 passed
_ Patch Compile Tests _
+1 💚 mvninstall 6m 20s the patch passed
+1 💚 compile 1m 2s the patch passed
+1 💚 javac 1m 2s the patch passed
-1 ❌ checkstyle 1m 38s hbase-server: The patch generated 2 new + 40 unchanged - 0 fixed = 42 total (was 40)
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 shadedjars 5m 20s patch has no errors when building our shaded downstream artifacts.
+1 💚 hadoopcheck 20m 19s Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
+1 💚 javadoc 0m 39s the patch passed
+1 💚 findbugs 4m 14s the patch passed
_ Other Tests _
+1 💚 unit 166m 38s hbase-server in the patch passed.
+1 💚 asflicense 0m 34s The patch does not generate ASF License warnings.
234m 49s
Subsystem Report/Notes
Docker Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-855/1/artifact/out/Dockerfile
GITHUB PR #855
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname Linux c26cbc5c7ead 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-855/out/precommit/personality/provided.sh
git revision branch-2 / 70771b6
Default Java 1.8.0_181
checkstyle https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-855/1/artifact/out/diff-checkstyle-hbase-server.txt
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-855/1/testReport/
Max. process+thread count 4639 (vs. ulimit of 10000)
modules C: hbase-server U: hbase-server
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-855/1/console
versions git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@binlijin
Copy link
Contributor

LGTM

@saintstack
Copy link
Contributor Author

Thanks @binlijin .

I'm testing on cluster. Found condition where there may be a reference still in Master to a Server though it failed its SCP. In this case, we want to force through out SCP. The code currently bypasses thinking it is mid-processing. Will add a piece so can ride over this state and get the SCP scheduled regardless. Thanks for review.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 36s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ branch-2 Compile Tests _
+1 💚 mvninstall 5m 22s branch-2 passed
+1 💚 compile 0m 56s branch-2 passed
+1 💚 checkstyle 1m 22s branch-2 passed
+1 💚 shadedjars 4m 8s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 36s branch-2 passed
+0 🆗 spotbugs 3m 36s Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚 findbugs 3m 35s branch-2 passed
_ Patch Compile Tests _
+1 💚 mvninstall 4m 48s the patch passed
+1 💚 compile 0m 56s the patch passed
+1 💚 javac 0m 56s the patch passed
-1 ❌ checkstyle 1m 19s hbase-server: The patch generated 2 new + 46 unchanged - 0 fixed = 48 total (was 46)
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 shadedjars 4m 10s patch has no errors when building our shaded downstream artifacts.
+1 💚 hadoopcheck 15m 17s Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
+1 💚 javadoc 0m 36s the patch passed
+1 💚 findbugs 3m 31s the patch passed
_ Other Tests _
+1 💚 unit 162m 58s hbase-server in the patch passed.
+1 💚 asflicense 0m 35s The patch does not generate ASF License warnings.
217m 2s
Subsystem Report/Notes
Docker Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-855/2/artifact/out/Dockerfile
GITHUB PR #855
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname Linux 8bf95434916f 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-855/out/precommit/personality/provided.sh
git revision branch-2 / 7e088da
Default Java 1.8.0_181
checkstyle https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-855/2/artifact/out/diff-checkstyle-hbase-server.txt
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-855/2/testReport/
Max. process+thread count 4753 (vs. ulimit of 10000)
modules C: hbase-server U: hbase-server
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-855/2/console
versions git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@saintstack
Copy link
Contributor Author

Pushed manually after fixing the checkstyle complaint.

@saintstack saintstack closed this Nov 21, 2019
ndimiduk
ndimiduk reviewed Nov 22, 2019
View reviewed changes
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
* SCP running.
* @return pid of scheduled SCP or {@link Procedure#NO_PROC_ID} if none scheduled.
*/
public long submitServerCrash(ServerName serverName, boolean shouldSplitWal, boolean force) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't love this force flag. I get that all we have to look for are side-effects, but seems like there should be a way of accounting the actively running procedures and at least wait for the current one to finish before starting the next. Or maybe the procedure implementations can negotiate the mutual exclusion lock between themselves? This could would unconditionally schedule the action and the action itself would refuse to run as long as another one is in flight. And then, of course, the second action might wake up and find that it has no work to do.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ndimiduk ndimiduk ndimiduk left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
4 participants
@saintstack @Apache-HBase @binlijin @ndimiduk