Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HBASE-24603: Make Zookeeper sync() call synchronous (#1945) #1976

Merged
merged 1 commit into from
Jun 26, 2020
Merged

HBASE-24603: Make Zookeeper sync() call synchronous (#1945) #1976

merged 1 commit into from
Jun 26, 2020

Conversation

bharathv
Copy link
Contributor

Writing a test for this is tricky. There is enough coverage for
functional tests. Only concern is performance, but there is enough
logging for it to detect timed out/badly performing sync calls.

Additionally, this patch decouples the ZK event processing into it's
own thread rather than doing it in the EventThread's context. That
avoids deadlocks and stalls of the event thread.

Signed-off-by: Andrew Purtell apurtell@apache.org
Signed-off-by: Viraj Jasani vjasani@apache.org
(cherry picked from commit 84e246f)
(cherry picked from commit 2379a25)

@bharathv
HBASE-24603: Make Zookeeper sync() call synchronous (apache#1945)
b108d11 
Writing a test for this is tricky. There is enough coverage for
functional tests. Only concern is performance, but there is enough
logging for it to detect timed out/badly performing sync calls.

Additionally, this patch decouples the ZK event processing into it's
own thread rather than doing it in the EventThread's context. That
avoids deadlocks and stalls of the event thread.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 84e246f)
(cherry picked from commit 2379a25)
@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 11s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ branch-1 Compile Tests _
+0 🆗 mvndep 2m 20s Maven dependency ordering for branch
+1 💚 mvninstall 7m 58s branch-1 passed
+1 💚 compile 1m 22s branch-1 passed with JDK v1.8.0_252
+1 💚 compile 1m 33s branch-1 passed with JDK v1.7.0_262
+1 💚 checkstyle 2m 56s branch-1 passed
+1 💚 shadedjars 3m 17s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 1m 17s branch-1 passed with JDK v1.8.0_252
+1 💚 javadoc 1m 31s branch-1 passed with JDK v1.7.0_262
+0 🆗 spotbugs 2m 52s Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚 findbugs 5m 37s branch-1 passed
_ Patch Compile Tests _
+0 🆗 mvndep 0m 16s Maven dependency ordering for patch
+1 💚 mvninstall 2m 4s the patch passed
+1 💚 compile 1m 18s the patch passed with JDK v1.8.0_252
+1 💚 javac 1m 18s the patch passed
+1 💚 compile 1m 33s the patch passed with JDK v1.7.0_262
+1 💚 javac 1m 33s the patch passed
+1 💚 checkstyle 0m 30s The patch passed checkstyle in hbase-common
+1 💚 checkstyle 0m 37s hbase-client: The patch generated 0 new + 77 unchanged - 2 fixed = 77 total (was 79)
+1 💚 checkstyle 1m 43s The patch passed checkstyle in hbase-server
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 shadedjars 3m 5s patch has no errors when building our shaded downstream artifacts.
+1 💚 hadoopcheck 4m 56s Patch does not cause any errors with Hadoop 2.8.5 2.9.2.
+1 💚 javadoc 1m 9s the patch passed with JDK v1.8.0_252
+1 💚 javadoc 1m 30s the patch passed with JDK v1.7.0_262
-1 ❌ findbugs 1m 39s hbase-client generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
_ Other Tests _
+1 💚 unit 2m 42s hbase-common in the patch passed.
+1 💚 unit 2m 50s hbase-client in the patch passed.
-1 ❌ unit 167m 24s hbase-server in the patch failed.
+1 💚 asflicense 1m 21s The patch does not generate ASF License warnings.
232m 29s
Reason Tests
FindBugs module:hbase-client
Format-string method String.format(String, Object[]) called with format string "Invalid event of type {} received for path {}. Ignoring" wants 0 arguments but is given 2 in org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.processEvent(WatchedEvent) At ZooKeeperWatcher.java:with format string "Invalid event of type {} received for path {}. Ignoring" wants 0 arguments but is given 2 in org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.processEvent(WatchedEvent) At ZooKeeperWatcher.java:[line 666]
Failed junit tests hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFiles
Subsystem Report/Notes
Docker Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1976/1/artifact/out/Dockerfile
GITHUB PR #1976
JIRA Issue HBASE-24603
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname Linux 966ee6e0d634 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1976/out/precommit/personality/provided.sh
git revision branch-1 / 54c38c8
Default Java 1.7.0_262
Multi-JDK versions /usr/lib/jvm/zulu-8-amd64:1.8.0_252 /usr/lib/jvm/zulu-7-amd64:1.7.0_262
findbugs https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1976/1/artifact/out/new-findbugs-hbase-client.html
unit https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1976/1/artifact/out/patch-unit-hbase-server.txt
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1976/1/testReport/
Max. process+thread count 4624 (vs. ulimit of 10000)
modules C: hbase-common hbase-client hbase-server U: .
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1976/1/console
versions git=1.9.1 maven=3.0.5 findbugs=3.0.1
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@bharathv bharathv merged commit 32690e1 into apache:branch-1 Jun 26, 2020
@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 12m 9s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ branch-1 Compile Tests _
+0 🆗 mvndep 2m 29s Maven dependency ordering for branch
+1 💚 mvninstall 8m 25s branch-1 passed
+1 💚 compile 1m 20s branch-1 passed with JDK v1.8.0_252
+1 💚 compile 1m 32s branch-1 passed with JDK v1.7.0_262
+1 💚 checkstyle 2m 58s branch-1 passed
+1 💚 shadedjars 3m 17s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 1m 17s branch-1 passed with JDK v1.8.0_252
+1 💚 javadoc 1m 30s branch-1 passed with JDK v1.7.0_262
+0 🆗 spotbugs 2m 51s Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚 findbugs 5m 35s branch-1 passed
_ Patch Compile Tests _
+0 🆗 mvndep 0m 16s Maven dependency ordering for patch
+1 💚 mvninstall 2m 4s the patch passed
+1 💚 compile 1m 18s the patch passed with JDK v1.8.0_252
+1 💚 javac 1m 18s the patch passed
+1 💚 compile 1m 32s the patch passed with JDK v1.7.0_262
+1 💚 javac 1m 32s the patch passed
+1 💚 checkstyle 0m 29s The patch passed checkstyle in hbase-common
+1 💚 checkstyle 0m 38s hbase-client: The patch generated 0 new + 77 unchanged - 2 fixed = 77 total (was 79)
+1 💚 checkstyle 1m 44s The patch passed checkstyle in hbase-server
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 shadedjars 3m 5s patch has no errors when building our shaded downstream artifacts.
+1 💚 hadoopcheck 4m 53s Patch does not cause any errors with Hadoop 2.8.5 2.9.2.
+1 💚 javadoc 1m 7s the patch passed with JDK v1.8.0_252
+1 💚 javadoc 1m 30s the patch passed with JDK v1.7.0_262
+1 💚 findbugs 5m 59s the patch passed
_ Other Tests _
+1 💚 unit 2m 37s hbase-common in the patch passed.
+1 💚 unit 2m 37s hbase-client in the patch passed.
+1 💚 unit 146m 10s hbase-server in the patch passed.
+1 💚 asflicense 1m 6s The patch does not generate ASF License warnings.
222m 5s
Subsystem Report/Notes
Docker Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1976/2/artifact/out/Dockerfile
GITHUB PR #1976
JIRA Issue HBASE-24603
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname Linux 6ee779511641 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1976/out/precommit/personality/provided.sh
git revision branch-1 / 54c38c8
Default Java 1.7.0_262
Multi-JDK versions /usr/lib/jvm/zulu-8-amd64:1.8.0_252 /usr/lib/jvm/zulu-7-amd64:1.7.0_262
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1976/2/testReport/
Max. process+thread count 5089 (vs. ulimit of 10000)
modules C: hbase-common hbase-client hbase-server U: .
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1976/2/console
versions git=1.9.1 maven=3.0.5 findbugs=3.0.1
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
2 participants
@bharathv @Apache-HBase