HBASE-28184 Tailing the WAL is very slow if there are multiple peers #5504

shahrs87 · 2023-11-06T01:34:53Z

master branch PR: #5503
branch-2 PR: #5504
branch-2.5 PR: #5505

Apache-HBase · 2023-11-06T02:27:32Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	2m 29s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ branch-2 Compile Tests _
+1 💚	mvninstall	2m 55s	branch-2 passed
+1 💚	compile	2m 26s	branch-2 passed
+1 💚	checkstyle	0m 33s	branch-2 passed
+1 💚	spotless	0m 41s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 23s	branch-2 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 39s	the patch passed
+1 💚	compile	2m 24s	the patch passed
+1 💚	javac	2m 24s	the patch passed
+1 💚	checkstyle	0m 40s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	16m 45s	Patch does not cause any errors with Hadoop 2.10.2 or 3.2.4 3.3.6.
+1 💚	spotless	0m 39s	patch has no errors when running spotless:check.
+1 💚	spotbugs	1m 29s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 10s	The patch does not generate ASF License warnings.
		36m 44s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5504/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5504
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux d6da4f69c510 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `8b1f690`
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	80 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5504/1/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-11-06T05:46:35Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	2m 25s	Docker mode activated.
-0 ⚠️	yetus	0m 5s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ branch-2 Compile Tests _
+1 💚	mvninstall	2m 55s	branch-2 passed
+1 💚	compile	0m 43s	branch-2 passed
+1 💚	shadedjars	5m 22s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 23s	branch-2 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 39s	the patch passed
+1 💚	compile	0m 42s	the patch passed
+1 💚	javac	0m 42s	the patch passed
+1 💚	shadedjars	5m 18s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 22s	the patch passed
		_ Other Tests _
+1 💚	unit	211m 4s	hbase-server in the patch passed.
		235m 52s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5504/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5504
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux ea0873188db5 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `8b1f690`
Default Java	Eclipse Adoptium-11.0.17+8
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5504/1/testReport/
Max. process+thread count	4879 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5504/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-11-06T06:01:37Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 42s	Docker mode activated.
-0 ⚠️	yetus	0m 5s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ branch-2 Compile Tests _
+1 💚	mvninstall	2m 25s	branch-2 passed
+1 💚	compile	0m 41s	branch-2 passed
+1 💚	shadedjars	4m 43s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 24s	branch-2 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 17s	the patch passed
+1 💚	compile	0m 41s	the patch passed
+1 💚	javac	0m 41s	the patch passed
+1 💚	shadedjars	4m 39s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 23s	the patch passed
		_ Other Tests _
+1 💚	unit	229m 31s	hbase-server in the patch passed.
		251m 0s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5504/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR	#5504
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 245a19d6095b 5.4.0-163-generic #180-Ubuntu SMP Tue Sep 5 13:21:23 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `8b1f690`
Default Java	Temurin-1.8.0_352-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5504/1/testReport/
Max. process+thread count	4016 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5504/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

…-28184-branch-2

Apache-HBase · 2023-11-06T20:42:05Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 28s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ branch-2 Compile Tests _
+1 💚	mvninstall	3m 22s	branch-2 passed
+1 💚	compile	2m 50s	branch-2 passed
+1 💚	checkstyle	0m 35s	branch-2 passed
+1 💚	spotless	0m 46s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 30s	branch-2 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 50s	the patch passed
+1 💚	compile	2m 31s	the patch passed
+1 💚	javac	2m 31s	the patch passed
+1 💚	checkstyle	0m 33s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	15m 11s	Patch does not cause any errors with Hadoop 2.10.2 or 3.2.4 3.3.6.
+1 💚	spotless	0m 39s	patch has no errors when running spotless:check.
+1 💚	spotbugs	1m 29s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 8s	The patch does not generate ASF License warnings.
		35m 40s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5504/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5504
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux 29306c74e195 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `c0b5e96`
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	78 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5504/2/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-11-07T00:29:37Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 4s	Docker mode activated.
-0 ⚠️	yetus	0m 7s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ branch-2 Compile Tests _
+1 💚	mvninstall	3m 45s	branch-2 passed
+1 💚	compile	0m 55s	branch-2 passed
+1 💚	shadedjars	6m 55s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 31s	branch-2 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 25s	the patch passed
+1 💚	compile	0m 53s	the patch passed
+1 💚	javac	0m 53s	the patch passed
+1 💚	shadedjars	7m 42s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 33s	the patch passed
		_ Other Tests _
+1 💚	unit	233m 54s	hbase-server in the patch passed.
		263m 49s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5504/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR	#5504
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 0711135fc48b 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `c0b5e96`
Default Java	Temurin-1.8.0_352-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5504/2/testReport/
Max. process+thread count	4517 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5504/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-11-07T00:42:30Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 32s	Docker mode activated.
-0 ⚠️	yetus	0m 4s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ branch-2 Compile Tests _
+1 💚	mvninstall	2m 54s	branch-2 passed
+1 💚	compile	0m 48s	branch-2 passed
+1 💚	shadedjars	5m 3s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 27s	branch-2 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 41s	the patch passed
+1 💚	compile	0m 51s	the patch passed
+1 💚	javac	0m 51s	the patch passed
+1 💚	shadedjars	6m 25s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 31s	the patch passed
		_ Other Tests _
-1 ❌	unit	252m 2s	hbase-server in the patch failed.
		276m 46s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5504/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5504
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 24cf043ecffc 5.4.0-156-generic #173-Ubuntu SMP Tue Jul 11 07:25:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `c0b5e96`
Default Java	Eclipse Adoptium-11.0.17+8
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5504/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5504/2/testReport/
Max. process+thread count	4049 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5504/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

shahrs87 · 2023-11-07T18:23:25Z

[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.hbase.master.TestClusterRestartFailoverSplitWithoutZk
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 40.153 s - in org.apache.hadoop.hbase.master.TestClusterRestartFailoverSplitWithoutZk
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

TestClusterRestartFailoverSplitWithoutZk passes locally with hadoop3 and jdk 11. Merging this.

HBASE-28184 Tailing the WAL is very slow if there are multiple peers

ab2df3a

This was referenced Nov 6, 2023

HBASE-28184 Tailing the WAL is very slow if there are multiple peers #5503

Merged

HBASE-28184 Tailing the WAL is very slow if there are multiple peers #5505

Merged

shahrs87 added 2 commits November 6, 2023 11:58

Merge branch 'branch-2' of https://github.com/apache/hbase into HBASE…

fcac360

…-28184-branch-2

Addressing review comments

900a5c4

shahrs87 requested a review from virajjasani November 7, 2023 18:23

virajjasani approved these changes Nov 7, 2023

View reviewed changes

shahrs87 merged commit 3f29f8e into apache:branch-2 Nov 7, 2023
1 check failed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HBASE-28184 Tailing the WAL is very slow if there are multiple peers #5504

HBASE-28184 Tailing the WAL is very slow if there are multiple peers #5504

shahrs87 commented Nov 6, 2023 •

edited

Loading

Apache-HBase commented Nov 6, 2023

Apache-HBase commented Nov 6, 2023

Apache-HBase commented Nov 6, 2023

Apache-HBase commented Nov 6, 2023

Apache-HBase commented Nov 7, 2023

Apache-HBase commented Nov 7, 2023

shahrs87 commented Nov 7, 2023

HBASE-28184 Tailing the WAL is very slow if there are multiple peers #5504

HBASE-28184 Tailing the WAL is very slow if there are multiple peers #5504

Conversation

shahrs87 commented Nov 6, 2023 • edited Loading

Apache-HBase commented Nov 6, 2023

Apache-HBase commented Nov 6, 2023

Apache-HBase commented Nov 6, 2023

Apache-HBase commented Nov 6, 2023

Apache-HBase commented Nov 7, 2023

Apache-HBase commented Nov 7, 2023

shahrs87 commented Nov 7, 2023

shahrs87 commented Nov 6, 2023 •

edited

Loading