HBASE-26715 Blocked on SyncFuture in AsyncProtobufLogWriter#write #4184

apurtell · 2022-03-08T20:38:51Z

See the discussion on HBASE-26715

bbeaudreault

LGTM.

I was going to comment, I actually had envisioned that we'd feed the timeout in higher up the stack in the rollWriter path somewhere. It was initially concerning to me to see a wal roll timeout in a method called write. That said, I took another look at the code and realized two things:

It'd be very hard to wire that through so many layers of indirection.
The write method here is actually only used for writing the WAL header and tailer -- so it may indeed make sense to use a roll-wall-specific timeout here.

So with all of that, this is a very nitpick, but I wonder if we should rename write to writeWALMetadata or something like that. It'd just be clearer to the reader I think, but not a huge deal.

apurtell · 2022-03-08T22:29:59Z

I wonder if we should rename write to writeWALMetadata or something like that. It'd just be clearer to the reader I think, but not a huge deal.

@bbeaudreault sounds good, see c20aa7a

bbeaudreault · 2022-03-08T22:40:38Z

Thanks! Looks good.

Apache-HBase · 2022-03-08T23:42:11Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 24s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 5s	master passed
+1 💚	compile	0m 46s	master passed
+1 💚	shadedjars	3m 42s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 27s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 50s	the patch passed
+1 💚	compile	0m 45s	the patch passed
+1 💚	javac	0m 45s	the patch passed
+1 💚	shadedjars	3m 36s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 26s	the patch passed
		_ Other Tests _
-1 ❌	unit	15m 25s	hbase-server in the patch failed.
		32m 34s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4184/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#4184
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 5c8e14ddf14b 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `1047194`
Default Java	AdoptOpenJDK-11.0.10+9
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4184/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4184/1/testReport/
Max. process+thread count	466 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4184/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

virajjasani · 2022-03-09T07:26:43Z

Triggered another build.

Apache-HBase · 2022-03-09T07:57:00Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 42s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 45s	master passed
+1 💚	compile	0m 34s	master passed
+1 💚	shadedjars	3m 50s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 22s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 21s	the patch passed
+1 💚	compile	0m 33s	the patch passed
+1 💚	javac	0m 33s	the patch passed
+1 💚	shadedjars	3m 48s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 20s	the patch passed
		_ Other Tests _
-1 ❌	unit	13m 50s	hbase-server in the patch failed.
		29m 55s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4184/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#4184
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 2d6a476b9969 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `1047194`
Default Java	AdoptOpenJDK-1.8.0_282-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4184/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4184/2/testReport/
Max. process+thread count	458 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4184/2/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-03-09T07:58:31Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 19s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 1s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	2m 45s	master passed
+1 💚	compile	2m 15s	master passed
+1 💚	checkstyle	0m 38s	master passed
+1 💚	spotbugs	1m 20s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 24s	the patch passed
+1 💚	compile	2m 11s	the patch passed
+1 💚	javac	2m 11s	the patch passed
+1 💚	checkstyle	0m 36s	hbase-server: The patch generated 0 new + 2 unchanged - 1 fixed = 2 total (was 3)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	11m 42s	Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
+1 💚	spotbugs	1m 24s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 11s	The patch does not generate ASF License warnings.
		31m 30s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4184/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4184
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux fd78c57c04fd 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `1047194`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Max. process+thread count	60 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4184/2/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-03-09T08:04:58Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 31s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 30s	master passed
+1 💚	compile	0m 48s	master passed
+1 💚	shadedjars	4m 17s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 28s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 37s	the patch passed
+1 💚	compile	0m 58s	the patch passed
+1 💚	javac	0m 58s	the patch passed
+1 💚	shadedjars	4m 10s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 30s	the patch passed
		_ Other Tests _
-1 ❌	unit	18m 2s	hbase-server in the patch failed.
		37m 42s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4184/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#4184
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux d014fa5c5cf5 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `1047194`
Default Java	AdoptOpenJDK-11.0.10+9
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4184/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4184/2/testReport/
Max. process+thread count	386 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4184/2/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

apurtell · 2022-03-09T19:14:28Z

Test failures in precommit checks are a NPE due to this change, let me fix it...

Caused by: java.lang.NullPointerException
at org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.<init>(AsyncProtobufLogWriter.java:123)
... 49 more

apurtell · 2022-03-09T19:23:29Z

Rebase and fix NPE found by precommit

…_WAIT_TIMEOUT

apurtell · 2022-03-09T19:31:01Z

Make default for WAL_ROLL_WAIT_TIMEOUT into constant DEFAULT_WAL_ROLL_WAIT_TIMEOUT

Apache-HBase · 2022-03-09T20:45:29Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 39s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	4m 14s	master passed
+1 💚	compile	2m 13s	master passed
+1 💚	checkstyle	0m 36s	master passed
+1 💚	spotbugs	1m 17s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 28s	the patch passed
+1 💚	compile	2m 14s	the patch passed
+1 💚	javac	2m 14s	the patch passed
+1 💚	checkstyle	0m 34s	hbase-server: The patch generated 0 new + 2 unchanged - 1 fixed = 2 total (was 3)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	11m 43s	Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
+1 💚	spotbugs	1m 19s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 9s	The patch does not generate ASF License warnings.
		32m 29s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4184/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4184
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux 7ef6cb606bae 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `1047194`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Max. process+thread count	60 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4184/3/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-03-10T01:54:05Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 1s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 12s	master passed
+1 💚	compile	0m 47s	master passed
+1 💚	shadedjars	3m 43s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 28s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 43s	the patch passed
+1 💚	compile	0m 45s	the patch passed
+1 💚	javac	0m 45s	the patch passed
+1 💚	shadedjars	3m 34s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 26s	the patch passed
		_ Other Tests _
-1 ❌	unit	323m 29s	hbase-server in the patch failed.
		341m 21s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4184/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#4184
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux afcbf8a1c929 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `1047194`
Default Java	AdoptOpenJDK-11.0.10+9
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4184/3/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4184/3/testReport/
Max. process+thread count	3269 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4184/3/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-03-10T02:04:36Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	2m 10s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 22s	master passed
+1 💚	compile	0m 43s	master passed
+1 💚	shadedjars	4m 25s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 30s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 55s	the patch passed
+1 💚	compile	0m 40s	the patch passed
+1 💚	javac	0m 40s	the patch passed
+1 💚	shadedjars	4m 29s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 30s	the patch passed
		_ Other Tests _
-1 ❌	unit	329m 16s	hbase-server in the patch failed.
		351m 42s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4184/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#4184
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux cf03161b37be 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `1047194`
Default Java	AdoptOpenJDK-1.8.0_282-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4184/3/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4184/3/testReport/
Max. process+thread count	3519 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4184/3/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

apurtell · 2022-03-10T19:19:37Z

TestHBaseTestingUtil issue is HBASE-26824

apurtell · 2022-03-10T19:51:47Z

I have a fix pending for HBASE-26824 on #4203 . Going to merge this now.

) Co-authored-by: Bryan Beaudreault <bbeaudreault@gmail.com> Signed-off-by: Duo Zhang <zhangduo@apache.org> Signed-off-by: Viraj Jasani<virajjasani@apache.org> Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>

…ogWriter#write (apache#4184) Co-authored-by: Bryan Beaudreault <bbeaudreault@gmail.com> Signed-off-by: Duo Zhang <zhangduo@apache.org> Signed-off-by: Viraj Jasani<virajjasani@apache.org> Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>

…ache#4184) Co-authored-by: Bryan Beaudreault <bbeaudreault@gmail.com> Signed-off-by: Duo Zhang <zhangduo@apache.org> Signed-off-by: Viraj Jasani<virajjasani@apache.org> Signed-off-by: Xiaolin Ha <haxiaolin@apache.org> (cherry picked from commit bfae853) Change-Id: I5523da8a57333e4076cdbc0584c48d4833850b52

apurtell requested review from Apache9 and virajjasani March 8, 2022 20:38

bbeaudreault approved these changes Mar 8, 2022

View reviewed changes

sunhelly approved these changes Mar 9, 2022

View reviewed changes

virajjasani approved these changes Mar 9, 2022

View reviewed changes

Apache9 approved these changes Mar 9, 2022

View reviewed changes

apurtell added 3 commits March 9, 2022 11:15

HBASE-26715 Blocked on SyncFuture in AsyncProtobufLogWriter#write

3b8980f

Address review feedback

883bead

Fix NPE found by precommit checks

5c34c83

apurtell force-pushed the HBASE-26715 branch from c20aa7a to 5c34c83 Compare March 9, 2022 19:23

Make default for WAL_ROLL_WAIT_TIMEOUT into constant DEFAULT_WAL_ROLL…

b024bcf

…_WAIT_TIMEOUT

apurtell merged commit e5dbbd2 into apache:master Mar 10, 2022

apurtell deleted the HBASE-26715 branch March 10, 2022 19:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HBASE-26715 Blocked on SyncFuture in AsyncProtobufLogWriter#write #4184

HBASE-26715 Blocked on SyncFuture in AsyncProtobufLogWriter#write #4184

apurtell commented Mar 8, 2022

bbeaudreault left a comment

apurtell commented Mar 8, 2022

bbeaudreault commented Mar 8, 2022

Apache-HBase commented Mar 8, 2022

virajjasani commented Mar 9, 2022

Apache-HBase commented Mar 9, 2022

Apache-HBase commented Mar 9, 2022

Apache-HBase commented Mar 9, 2022

apurtell commented Mar 9, 2022 •

edited

apurtell commented Mar 9, 2022

apurtell commented Mar 9, 2022 •

edited

Apache-HBase commented Mar 9, 2022

Apache-HBase commented Mar 10, 2022

Apache-HBase commented Mar 10, 2022

apurtell commented Mar 10, 2022

apurtell commented Mar 10, 2022

HBASE-26715 Blocked on SyncFuture in AsyncProtobufLogWriter#write #4184

HBASE-26715 Blocked on SyncFuture in AsyncProtobufLogWriter#write #4184

Conversation

apurtell commented Mar 8, 2022

bbeaudreault left a comment

Choose a reason for hiding this comment

apurtell commented Mar 8, 2022

bbeaudreault commented Mar 8, 2022

Apache-HBase commented Mar 8, 2022

virajjasani commented Mar 9, 2022

Apache-HBase commented Mar 9, 2022

Apache-HBase commented Mar 9, 2022

Apache-HBase commented Mar 9, 2022

apurtell commented Mar 9, 2022 • edited

apurtell commented Mar 9, 2022

apurtell commented Mar 9, 2022 • edited

Apache-HBase commented Mar 9, 2022

Apache-HBase commented Mar 10, 2022

Apache-HBase commented Mar 10, 2022

apurtell commented Mar 10, 2022

apurtell commented Mar 10, 2022

apurtell commented Mar 9, 2022 •

edited

apurtell commented Mar 9, 2022 •

edited