HBASE-27398 Remove dumping of EOFException while reading WAL with ProtobufLogReader #4806

virajjasani · 2022-09-28T19:06:34Z

No description provided.

…tobufLogReader

Apache-HBase · 2022-09-28T19:45:13Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 6s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	3m 32s	master passed
+1 💚	compile	2m 16s	master passed
+1 💚	checkstyle	0m 32s	master passed
+1 💚	spotless	0m 49s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 21s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 12s	the patch passed
+1 💚	compile	2m 15s	the patch passed
+1 💚	javac	2m 15s	the patch passed
+1 💚	checkstyle	0m 31s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	7m 59s	Patch does not cause any errors with Hadoop 3.2.4 3.3.4.
-1 ❌	spotless	0m 33s	patch has 26 errors when running spotless:check, run spotless:apply to fix.
+1 💚	spotbugs	1m 26s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 12s	The patch does not generate ASF License warnings.
		29m 59s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4806/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4806
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux dd6e7acc0123 5.4.0-122-generic #138-Ubuntu SMP Wed Jun 22 15:00:31 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `23a5633`
Default Java	Temurin-1.8.0_345-b01
spotless	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4806/1/artifact/yetus-general-check/output/patch-spotless.txt
Max. process+thread count	64 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4806/1/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.7.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-09-28T23:11:28Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 8s	Docker mode activated.
-0 ⚠️	yetus	0m 4s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 24s	master passed
+1 💚	compile	0m 39s	master passed
+1 💚	shadedjars	3m 49s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 26s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 18s	the patch passed
+1 💚	compile	0m 40s	the patch passed
+1 💚	javac	0m 40s	the patch passed
+1 💚	shadedjars	3m 48s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 25s	the patch passed
		_ Other Tests _
+1 💚	unit	218m 23s	hbase-server in the patch passed.
		236m 9s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4806/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#4806
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux b634435e81c8 5.4.0-124-generic #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `23a5633`
Default Java	Temurin-1.8.0_345-b01
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4806/1/testReport/
Max. process+thread count	2557 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4806/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-09-28T23:26:02Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 1s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 12s	master passed
+1 💚	compile	0m 47s	master passed
+1 💚	shadedjars	3m 54s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 29s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 38s	the patch passed
+1 💚	compile	0m 46s	the patch passed
+1 💚	javac	0m 46s	the patch passed
+1 💚	shadedjars	3m 46s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 26s	the patch passed
		_ Other Tests _
+1 💚	unit	232m 5s	hbase-server in the patch passed.
		250m 46s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4806/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#4806
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 9f9211d18ece 5.4.0-124-generic #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `23a5633`
Default Java	Eclipse Adoptium-11.0.16.1+1
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4806/1/testReport/
Max. process+thread count	2682 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4806/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-09-28T23:58:41Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	2m 28s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	2m 26s	master passed
+1 💚	compile	2m 16s	master passed
+1 💚	checkstyle	0m 32s	master passed
+1 💚	spotless	0m 42s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 19s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 7s	the patch passed
+1 💚	compile	2m 12s	the patch passed
+1 💚	javac	2m 12s	the patch passed
+1 💚	checkstyle	0m 31s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	7m 55s	Patch does not cause any errors with Hadoop 3.2.4 3.3.4.
+1 💚	spotless	0m 37s	patch has no errors when running spotless:check.
+1 💚	spotbugs	1m 23s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 11s	The patch does not generate ASF License warnings.
		29m 29s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4806/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4806
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux d55a1526cb76 5.4.0-124-generic #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `23a5633`
Default Java	Temurin-1.8.0_345-b01
Max. process+thread count	60 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4806/2/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.7.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache9 · 2022-09-29T00:58:30Z

Please wait a bit more time before merging. @ddupg is still trying to fix the EOF related problem in #4756, I'm not sure if removing the exception stacktrace will have negative impact there, especially that this is just a debug log, not info...

Thanks.

Apache9

OK, after reading the related code, I do not think we should remove the dumping here.

We can throw EOFException at many places in the try block, and they all have a fat message to describe what is going on, and if we remove the dumping of EOFException below, there is no way for us to print out these messages.

IF this is too excessive, I think we should try to avoid throwing too many EOFExceptions for normal cases, for example, if we are tailing a WAL which is currently being written, then at the last position we will get an EOF, and if we want to catch up fast, we will get a lot EOF, which is expected, so we should try to reduce these noises, instead of purging the output for all the EOFException.

Thanks.

Apache-HBase · 2022-09-29T03:08:14Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 48s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 50s	master passed
+1 💚	compile	0m 47s	master passed
+1 💚	shadedjars	3m 47s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 29s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 35s	the patch passed
+1 💚	compile	0m 47s	the patch passed
+1 💚	javac	0m 47s	the patch passed
+1 💚	shadedjars	3m 45s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 26s	the patch passed
		_ Other Tests _
+1 💚	unit	201m 41s	hbase-server in the patch passed.
		219m 34s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4806/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#4806
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux cbe7262f8d17 5.4.0-109-generic #123-Ubuntu SMP Fri Apr 8 09:10:54 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `23a5633`
Default Java	Eclipse Adoptium-11.0.16.1+1
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4806/2/testReport/
Max. process+thread count	2567 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4806/2/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-09-29T03:21:22Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 7s	Docker mode activated.
-0 ⚠️	yetus	0m 4s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 14s	master passed
+1 💚	compile	0m 39s	master passed
+1 💚	shadedjars	3m 52s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 26s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 13s	the patch passed
+1 💚	compile	0m 40s	the patch passed
+1 💚	javac	0m 40s	the patch passed
+1 💚	shadedjars	3m 48s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 25s	the patch passed
		_ Other Tests _
+1 💚	unit	215m 56s	hbase-server in the patch passed.
		232m 47s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4806/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#4806
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 49f9e8147af7 5.4.0-124-generic #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `23a5633`
Default Java	Temurin-1.8.0_345-b01
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4806/2/testReport/
Max. process+thread count	2465 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4806/2/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

virajjasani · 2022-09-29T04:21:55Z

hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/ProtobufLogReader.java

@@ -446,7 +444,7 @@ protected boolean readNext(Entry entry) throws IOException {
        // Else restore our position to original location in hope that next time through we will
        // read successfully.
        LOG.debug("Encountered a malformed edit, seeking back to last good position in file, "
-          + "from {} to {}", inputStream.getPos(), originalPosition, eof);
+          + "from {} to {}", inputStream.getPos(), originalPosition);


@Apache9 how about this change (line 447)? This change can be done if not the one above where we are throwing EOF (line 430-431)?

apurtell · 2022-10-05T00:03:42Z

This was -1ed by @Apache9 , so closing it

HBASE-27398 Remove dumping of EOFException while reading WAL with Pro…

a3070a9

…tobufLogReader

virajjasani requested a review from apurtell September 28, 2022 19:06

apurtell approved these changes Sep 28, 2022

View reviewed changes

spotless:apply

6d61b69

Apache9 requested changes Sep 29, 2022

View reviewed changes

virajjasani commented Sep 29, 2022

View reviewed changes

virajjasani requested a review from wchevreuil September 29, 2022 04:22

apurtell closed this Oct 5, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HBASE-27398 Remove dumping of EOFException while reading WAL with ProtobufLogReader #4806

HBASE-27398 Remove dumping of EOFException while reading WAL with ProtobufLogReader #4806

virajjasani commented Sep 28, 2022

Apache-HBase commented Sep 28, 2022

Apache-HBase commented Sep 28, 2022

Apache-HBase commented Sep 28, 2022

Apache-HBase commented Sep 28, 2022

Apache9 commented Sep 29, 2022

Apache9 left a comment

Apache-HBase commented Sep 29, 2022

Apache-HBase commented Sep 29, 2022

virajjasani Sep 29, 2022

apurtell commented Oct 5, 2022

HBASE-27398 Remove dumping of EOFException while reading WAL with ProtobufLogReader #4806

HBASE-27398 Remove dumping of EOFException while reading WAL with ProtobufLogReader #4806

Conversation

virajjasani commented Sep 28, 2022

Apache-HBase commented Sep 28, 2022

Apache-HBase commented Sep 28, 2022

Apache-HBase commented Sep 28, 2022

Apache-HBase commented Sep 28, 2022

Apache9 commented Sep 29, 2022

Apache9 left a comment

Choose a reason for hiding this comment

Apache-HBase commented Sep 29, 2022

Apache-HBase commented Sep 29, 2022

virajjasani Sep 29, 2022

Choose a reason for hiding this comment

apurtell commented Oct 5, 2022