HBASE-28184 Tailing the WAL is very slow if there are multiple peers #5505

shahrs87 · 2023-11-06T01:46:10Z

master branch PR: #5503
branch-2 PR: #5504
branch-2.5 PR: #5505

Apache-HBase · 2023-11-06T02:21:16Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 14s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ branch-2.5 Compile Tests _
+1 💚	mvninstall	2m 47s	branch-2.5 passed
+1 💚	compile	2m 24s	branch-2.5 passed
+1 💚	checkstyle	0m 37s	branch-2.5 passed
+1 💚	spotless	0m 43s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 29s	branch-2.5 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 34s	the patch passed
+1 💚	compile	2m 21s	the patch passed
+1 💚	javac	2m 21s	the patch passed
+1 💚	checkstyle	0m 33s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	13m 10s	Patch does not cause any errors with Hadoop 2.10.2 or 3.2.4 3.3.6.
+1 💚	spotless	0m 39s	patch has no errors when running spotless:check.
+1 💚	spotbugs	1m 32s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 11s	The patch does not generate ASF License warnings.
		30m 44s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5505/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5505
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux 2abf4244b62a 5.4.0-153-generic #170-Ubuntu SMP Fri Jun 16 13:43:31 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2.5 / `695ed25`
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	77 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5505/1/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache9 · 2023-11-06T02:38:08Z

hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/WALEntryStream.java

+      fileLength = OptionalLong.empty();
+    } else {
+      // if there is only one file in queue, check whether it is still being written to
+      // we must call this before actually reading from the reader, as this method will acquire the


For branch-2.5, the way we deal with empty queue problem is different. As there is no comment before this PR, let's just add 'if there is only one file in queue, check whether it is still being written to' comment here. The comments below are for branch-2+, where we have refactored the code a lot.

Apache-HBase · 2023-11-06T05:36:38Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 41s	Docker mode activated.
-0 ⚠️	yetus	0m 5s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ branch-2.5 Compile Tests _
+1 💚	mvninstall	3m 17s	branch-2.5 passed
+1 💚	compile	0m 50s	branch-2.5 passed
+1 💚	shadedjars	4m 55s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 27s	branch-2.5 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 35s	the patch passed
+1 💚	compile	0m 47s	the patch passed
+1 💚	javac	0m 47s	the patch passed
+1 💚	shadedjars	4m 58s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 26s	the patch passed
		_ Other Tests _
+1 💚	unit	202m 36s	hbase-server in the patch passed.
		226m 2s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5505/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5505
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux f0a95d92baee 5.4.0-163-generic #180-Ubuntu SMP Tue Sep 5 13:21:23 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2.5 / `695ed25`
Default Java	Eclipse Adoptium-11.0.17+8
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5505/1/testReport/
Max. process+thread count	4247 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5505/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-11-06T05:37:41Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 42s	Docker mode activated.
-0 ⚠️	yetus	0m 5s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ branch-2.5 Compile Tests _
+1 💚	mvninstall	2m 27s	branch-2.5 passed
+1 💚	compile	0m 40s	branch-2.5 passed
+1 💚	shadedjars	4m 35s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 26s	branch-2.5 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 15s	the patch passed
+1 💚	compile	0m 41s	the patch passed
+1 💚	javac	0m 41s	the patch passed
+1 💚	shadedjars	4m 33s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 24s	the patch passed
		_ Other Tests _
+1 💚	unit	206m 9s	hbase-server in the patch passed.
		227m 4s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5505/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR	#5505
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 1e334e69e573 5.4.0-163-generic #180-Ubuntu SMP Tue Sep 5 13:21:23 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2.5 / `695ed25`
Default Java	Temurin-1.8.0_352-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5505/1/testReport/
Max. process+thread count	4127 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5505/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

…SE-28184-branch-2.5

Apache-HBase · 2023-11-06T20:48:18Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 14s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ branch-2.5 Compile Tests _
+1 💚	mvninstall	3m 57s	branch-2.5 passed
+1 💚	compile	2m 52s	branch-2.5 passed
+1 💚	checkstyle	0m 40s	branch-2.5 passed
+1 💚	spotless	0m 51s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 46s	branch-2.5 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 54s	the patch passed
+1 💚	compile	2m 56s	the patch passed
+1 💚	javac	2m 56s	the patch passed
+1 💚	checkstyle	0m 35s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	19m 31s	Patch does not cause any errors with Hadoop 2.10.2 or 3.2.4 3.3.6.
+1 💚	spotless	0m 59s	patch has no errors when running spotless:check.
+1 💚	spotbugs	2m 30s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 14s	The patch does not generate ASF License warnings.
		42m 38s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5505/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5505
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux 0624205460e3 5.4.0-153-generic #170-Ubuntu SMP Fri Jun 16 13:43:31 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2.5 / `4236b70`
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	81 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5505/2/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-11-06T23:38:48Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 41s	Docker mode activated.
-0 ⚠️	yetus	0m 4s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ branch-2.5 Compile Tests _
+1 💚	mvninstall	2m 27s	branch-2.5 passed
+1 💚	compile	0m 37s	branch-2.5 passed
+1 💚	shadedjars	4m 58s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 23s	branch-2.5 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 11s	the patch passed
+1 💚	compile	0m 37s	the patch passed
+1 💚	javac	0m 37s	the patch passed
+1 💚	shadedjars	5m 0s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 21s	the patch passed
		_ Other Tests _
+1 💚	unit	191m 43s	hbase-server in the patch passed.
		213m 2s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5505/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR	#5505
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux aea9433f373f 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2.5 / `4236b70`
Default Java	Temurin-1.8.0_352-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5505/2/testReport/
Max. process+thread count	4518 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5505/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-11-07T00:23:02Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 39s	Docker mode activated.
-0 ⚠️	yetus	0m 5s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ branch-2.5 Compile Tests _
+1 💚	mvninstall	2m 34s	branch-2.5 passed
+1 💚	compile	0m 47s	branch-2.5 passed
+1 💚	shadedjars	4m 55s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 26s	branch-2.5 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 35s	the patch passed
+1 💚	compile	0m 47s	the patch passed
+1 💚	javac	0m 47s	the patch passed
+1 💚	shadedjars	6m 4s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 32s	the patch passed
		_ Other Tests _
+1 💚	unit	233m 40s	hbase-server in the patch passed.
		257m 21s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5505/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5505
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux eff5367f6325 5.4.0-163-generic #180-Ubuntu SMP Tue Sep 5 13:21:23 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2.5 / `4236b70`
Default Java	Eclipse Adoptium-11.0.17+8
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5505/2/testReport/
Max. process+thread count	4532 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5505/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

virajjasani

+1, we also want to backport the same patch to branch-2.4?

…5505) (cherry picked from commit d383f09)

HBASE-28184 Tailing the WAL is very slow if there are multiple peers

0f30fab

This was referenced Nov 6, 2023

HBASE-28184 Tailing the WAL is very slow if there are multiple peers #5503

Merged

HBASE-28184 Tailing the WAL is very slow if there are multiple peers #5504

Merged

Apache9 reviewed Nov 6, 2023

View reviewed changes

shahrs87 added 2 commits November 6, 2023 11:59

Merge branch 'branch-2.5' of https://github.com/apache/hbase into HBA…

7cfc57f

…SE-28184-branch-2.5

Addressing review comments

78a74be

shahrs87 requested a review from virajjasani November 7, 2023 18:24

virajjasani approved these changes Nov 7, 2023

View reviewed changes

shahrs87 merged commit d383f09 into apache:branch-2.5 Nov 7, 2023
1 check passed

shahrs87 added a commit that referenced this pull request Nov 7, 2023

HBASE-28184 Tailing the WAL is very slow if there are multiple peers (#…

4227452

…5505) (cherry picked from commit d383f09)

shahrs87 mentioned this pull request Nov 14, 2023

HBASE-28184 Addendum PR #5521

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HBASE-28184 Tailing the WAL is very slow if there are multiple peers #5505

HBASE-28184 Tailing the WAL is very slow if there are multiple peers #5505

shahrs87 commented Nov 6, 2023 •

edited

Loading

Apache-HBase commented Nov 6, 2023

Apache9 Nov 6, 2023

Apache-HBase commented Nov 6, 2023

Apache-HBase commented Nov 6, 2023

Apache-HBase commented Nov 6, 2023

Apache-HBase commented Nov 6, 2023

Apache-HBase commented Nov 7, 2023

virajjasani left a comment

HBASE-28184 Tailing the WAL is very slow if there are multiple peers #5505

HBASE-28184 Tailing the WAL is very slow if there are multiple peers #5505

Conversation

shahrs87 commented Nov 6, 2023 • edited Loading

Apache-HBase commented Nov 6, 2023

Apache9 Nov 6, 2023

Choose a reason for hiding this comment

Apache-HBase commented Nov 6, 2023

Apache-HBase commented Nov 6, 2023

Apache-HBase commented Nov 6, 2023

Apache-HBase commented Nov 6, 2023

Apache-HBase commented Nov 7, 2023

virajjasani left a comment

Choose a reason for hiding this comment

shahrs87 commented Nov 6, 2023 •

edited

Loading