Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HBASE-26482 HMaster may clean wals that is replicating in rare cases #3887

Merged
merged 1 commit into from Dec 5, 2021
Merged

HBASE-26482 HMaster may clean wals that is replicating in rare cases #3887

merged 1 commit into from Dec 5, 2021

Conversation

zhengzhuobinzzb
Copy link
Contributor

No description provided.

@zhengzhuobinzzb
Copy link
Contributor Author

There is a problem. We can't update cversion of root queuesZnode atomically when hbase.zookeeper.useMulti is set false.

Now, I only fixed this problem when hbase.zookeeper.useMulti true. (default is true)

Another way to totally solve this problem: Check cversion of /hbase/replication/rs and all znodes of /hbase/replication/rs/${servername} when master clean.

But this way will cause the code of branch-1 different with master.

I don't know which is better.

@Apache9
Copy link
Contributor

Apache9 commented Nov 27, 2021

I think it is OK to have different code implementation for branch-1.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 8m 11s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ branch-1 Compile Tests _
+0 🆗 mvndep 2m 29s Maven dependency ordering for branch
+1 💚 mvninstall 8m 40s branch-1 passed
+1 💚 compile 1m 10s branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚 compile 1m 17s branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚 checkstyle 2m 24s branch-1 passed
+1 💚 shadedjars 3m 34s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 1m 1s branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚 javadoc 1m 11s branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+0 🆗 spotbugs 2m 59s Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚 findbugs 4m 45s branch-1 passed
_ Patch Compile Tests _
+0 🆗 mvndep 0m 17s Maven dependency ordering for patch
+1 💚 mvninstall 2m 23s the patch passed
+1 💚 compile 1m 5s the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚 javac 1m 5s the patch passed
+1 💚 compile 1m 18s the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚 javac 1m 18s the patch passed
+1 💚 checkstyle 2m 16s the patch passed
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 shadedjars 3m 25s patch has no errors when building our shaded downstream artifacts.
+1 💚 hadoopcheck 5m 39s Patch does not cause any errors with Hadoop 2.8.5 2.9.2.
+1 💚 javadoc 0m 53s the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚 javadoc 1m 10s the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚 findbugs 4m 51s the patch passed
_ Other Tests _
+1 💚 unit 2m 41s hbase-client in the patch passed.
-1 ❌ unit 156m 33s hbase-server in the patch failed.
+1 💚 asflicense 0m 49s The patch does not generate ASF License warnings.
222m 30s
Reason Tests
Failed junit tests hadoop.hbase.mapreduce.TestLoadIncrementalHFilesUseSecurityEndPoint
hadoop.hbase.replication.regionserver.TestReplicationSourceManagerManager
hadoop.hbase.replication.TestReplicationKillSlaveRS
hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFiles
hadoop.hbase.mapreduce.TestLoadIncrementalHFiles
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3887/1/artifact/out/Dockerfile
GITHUB PR #3887
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname Linux b86505ef4a38 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-agent/workspace/Base-PreCommit-GitHub-PR_PR-3887/out/precommit/personality/provided.sh
git revision branch-1 / 13787ae
Default Java Azul Systems, Inc.-1.7.0_272-b10
Multi-JDK versions /usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10
unit https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3887/1/artifact/out/patch-unit-hbase-server.txt
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3887/1/testReport/
Max. process+thread count 5191 (vs. ulimit of 10000)
modules C: hbase-client hbase-server U: .
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3887/1/console
versions git=2.17.1 maven=3.6.0 findbugs=3.0.1
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 11s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ branch-1 Compile Tests _
+0 🆗 mvndep 2m 29s Maven dependency ordering for branch
+1 💚 mvninstall 8m 34s branch-1 passed
+1 💚 compile 1m 7s branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚 compile 1m 16s branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚 checkstyle 2m 25s branch-1 passed
+1 💚 shadedjars 3m 35s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 1m 1s branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚 javadoc 1m 9s branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+0 🆗 spotbugs 2m 58s Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚 findbugs 4m 46s branch-1 passed
_ Patch Compile Tests _
+0 🆗 mvndep 0m 17s Maven dependency ordering for patch
+1 💚 mvninstall 2m 20s the patch passed
+1 💚 compile 1m 6s the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚 javac 1m 6s the patch passed
+1 💚 compile 1m 17s the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚 javac 1m 17s the patch passed
-1 ❌ checkstyle 1m 43s hbase-server: The patch generated 1 new + 13 unchanged - 1 fixed = 14 total (was 14)
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 shadedjars 3m 23s patch has no errors when building our shaded downstream artifacts.
+1 💚 hadoopcheck 5m 30s Patch does not cause any errors with Hadoop 2.8.5 2.9.2.
+1 💚 javadoc 0m 53s the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚 javadoc 1m 9s the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚 findbugs 4m 51s the patch passed
_ Other Tests _
+1 💚 unit 2m 46s hbase-client in the patch passed.
-1 ❌ unit 157m 14s hbase-server in the patch failed.
+1 💚 asflicense 0m 49s The patch does not generate ASF License warnings.
215m 37s
Reason Tests
Failed junit tests hadoop.hbase.mapreduce.TestLoadIncrementalHFilesUseSecurityEndPoint
hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFiles
hadoop.hbase.mapreduce.TestLoadIncrementalHFiles
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3887/2/artifact/out/Dockerfile
GITHUB PR #3887
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname Linux 8436e4fbca98 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-agent/workspace/Base-PreCommit-GitHub-PR_PR-3887/out/precommit/personality/provided.sh
git revision branch-1 / 13787ae
Default Java Azul Systems, Inc.-1.7.0_272-b10
Multi-JDK versions /usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10
checkstyle https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3887/2/artifact/out/diff-checkstyle-hbase-server.txt
unit https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3887/2/artifact/out/patch-unit-hbase-server.txt
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3887/2/testReport/
Max. process+thread count 5011 (vs. ulimit of 10000)
modules C: hbase-client hbase-server U: .
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3887/2/console
versions git=2.17.1 maven=3.6.0 findbugs=3.0.1
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@zhengzhuobinzzb
Copy link
Contributor Author

HI @Apache9, Help review in your free time, thanks~
(Test Passed in local env.)

@zhengzhuobinzzb
Copy link
Contributor Author

@Apache9 General ping for review~~

@Apache9
Copy link
Contributor

Apache9 commented Nov 29, 2021

@Reidddddd PTAL in your convenience~

@Apache9 Apache9 merged commit 8d96fc3 into apache:branch-1 Dec 5, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Apache9 Apache9 Apache9 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
3 participants
@zhengzhuobinzzb @Apache9 @Apache-HBase