HBASE-27019 Minor compression performance improvements #4420

apurtell · 2022-05-11T01:13:09Z

TRACE level logging is expensive enough to warrant removal. They were useful during development but are now just overhead.

Also we unnecessarily create new compressor and decompressor instances in the reset() methods for the Aircompressor and Lz4 codecs. Remove.

Apache-HBase · 2022-05-11T01:55:11Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 6s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 11s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 20s	master passed
+1 💚	compile	1m 20s	master passed
+1 💚	shadedjars	3m 42s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	1m 7s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 13s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 13s	the patch passed
+1 💚	compile	1m 20s	the patch passed
+1 💚	javac	1m 20s	the patch passed
+1 💚	shadedjars	3m 39s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	1m 6s	the patch passed
		_ Other Tests _
+1 💚	unit	0m 43s	hbase-compression-aircompressor in the patch passed.
+1 💚	unit	0m 35s	hbase-compression-brotli in the patch passed.
+1 💚	unit	0m 29s	hbase-compression-lz4 in the patch passed.
+1 💚	unit	0m 31s	hbase-compression-snappy in the patch passed.
+1 💚	unit	0m 54s	hbase-compression-xz in the patch passed.
+1 💚	unit	0m 31s	hbase-compression-zstd in the patch passed.
		23m 45s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4420/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#4420
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 0fa4c07e72d6 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `f6e9d3e`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4420/1/testReport/
Max. process+thread count	798 (vs. ulimit of 30000)
modules	C: hbase-compression/hbase-compression-aircompressor hbase-compression/hbase-compression-brotli hbase-compression/hbase-compression-lz4 hbase-compression/hbase-compression-snappy hbase-compression/hbase-compression-xz hbase-compression/hbase-compression-zstd U: hbase-compression
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4420/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-05-11T01:57:09Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 6s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 11s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 46s	master passed
+1 💚	compile	1m 26s	master passed
+1 💚	shadedjars	3m 42s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	1m 15s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 14s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 34s	the patch passed
+1 💚	compile	1m 26s	the patch passed
+1 💚	javac	1m 26s	the patch passed
+1 💚	shadedjars	3m 45s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	1m 12s	the patch passed
		_ Other Tests _
+1 💚	unit	0m 50s	hbase-compression-aircompressor in the patch passed.
+1 💚	unit	0m 35s	hbase-compression-brotli in the patch passed.
+1 💚	unit	0m 30s	hbase-compression-lz4 in the patch passed.
+1 💚	unit	0m 29s	hbase-compression-snappy in the patch passed.
+1 💚	unit	0m 57s	hbase-compression-xz in the patch passed.
+1 💚	unit	0m 31s	hbase-compression-zstd in the patch passed.
		25m 12s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4420/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#4420
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 519488f20e92 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `f6e9d3e`
Default Java	AdoptOpenJDK-11.0.10+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4420/1/testReport/
Max. process+thread count	603 (vs. ulimit of 30000)
modules	C: hbase-compression/hbase-compression-aircompressor hbase-compression/hbase-compression-brotli hbase-compression/hbase-compression-lz4 hbase-compression/hbase-compression-snappy hbase-compression/hbase-compression-xz hbase-compression/hbase-compression-zstd U: hbase-compression
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4420/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-05-11T02:06:24Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 7s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+0 🆗	mvndep	0m 12s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 22s	master passed
+1 💚	compile	1m 42s	master passed
+1 💚	checkstyle	0m 48s	master passed
-1 ❌	spotless	0m 37s	branch has 68 errors when running spotless:check, run spotless:apply to fix.
+1 💚	spotbugs	1m 48s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 11s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 11s	the patch passed
+1 💚	compile	1m 40s	the patch passed
+1 💚	javac	1m 40s	the patch passed
+1 💚	checkstyle	0m 47s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	11m 37s	Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
-1 ❌	spotless	0m 36s	patch has 68 errors when running spotless:check, run spotless:apply to fix.
+1 💚	spotbugs	2m 24s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 54s	The patch does not generate ASF License warnings.
		34m 58s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4420/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4420
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux c341815e694c 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `f6e9d3e`
Default Java	AdoptOpenJDK-1.8.0_282-b08
spotless	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4420/1/artifact/yetus-general-check/output/branch-spotless.txt
spotless	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4420/1/artifact/yetus-general-check/output/patch-spotless.txt
Max. process+thread count	69 (vs. ulimit of 30000)
modules	C: hbase-compression/hbase-compression-aircompressor hbase-compression/hbase-compression-brotli hbase-compression/hbase-compression-lz4 hbase-compression/hbase-compression-snappy hbase-compression/hbase-compression-xz hbase-compression/hbase-compression-zstd U: hbase-compression
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4420/1/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

TRACE level logging is expensive enough to warrant removal. They were useful during development but are now just overhead. Also we unnecessarily create new compressor and decompressor instances in the reset() methods for the Aircompressor and Lz4 codecs. Remove.

Apache-HBase · 2022-05-13T22:09:14Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 7s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 11s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 28s	master passed
+1 💚	compile	1m 21s	master passed
+1 💚	shadedjars	3m 40s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	1m 8s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 13s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 14s	the patch passed
+1 💚	compile	1m 18s	the patch passed
+1 💚	javac	1m 18s	the patch passed
+1 💚	shadedjars	3m 41s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	1m 6s	the patch passed
		_ Other Tests _
+1 💚	unit	0m 43s	hbase-compression-aircompressor in the patch passed.
+1 💚	unit	0m 35s	hbase-compression-brotli in the patch passed.
+1 💚	unit	0m 30s	hbase-compression-lz4 in the patch passed.
+1 💚	unit	0m 30s	hbase-compression-snappy in the patch passed.
+1 💚	unit	0m 55s	hbase-compression-xz in the patch passed.
+1 💚	unit	0m 32s	hbase-compression-zstd in the patch passed.
		23m 56s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4420/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#4420
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 152ea08b3339 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `0eb1a4e`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4420/2/testReport/
Max. process+thread count	776 (vs. ulimit of 30000)
modules	C: hbase-compression/hbase-compression-aircompressor hbase-compression/hbase-compression-brotli hbase-compression/hbase-compression-lz4 hbase-compression/hbase-compression-snappy hbase-compression/hbase-compression-xz hbase-compression/hbase-compression-zstd U: hbase-compression
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4420/2/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-05-13T22:10:35Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 4s	Docker mode activated.
-0 ⚠️	yetus	0m 4s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 13s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 46s	master passed
+1 💚	compile	1m 29s	master passed
+1 💚	shadedjars	3m 44s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	1m 14s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 14s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 33s	the patch passed
+1 💚	compile	1m 26s	the patch passed
+1 💚	javac	1m 26s	the patch passed
+1 💚	shadedjars	3m 42s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	1m 13s	the patch passed
		_ Other Tests _
+1 💚	unit	0m 48s	hbase-compression-aircompressor in the patch passed.
+1 💚	unit	0m 35s	hbase-compression-brotli in the patch passed.
+1 💚	unit	0m 29s	hbase-compression-lz4 in the patch passed.
+1 💚	unit	0m 30s	hbase-compression-snappy in the patch passed.
+1 💚	unit	0m 56s	hbase-compression-xz in the patch passed.
+1 💚	unit	0m 31s	hbase-compression-zstd in the patch passed.
		25m 1s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4420/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#4420
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 83b413716a96 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `0eb1a4e`
Default Java	AdoptOpenJDK-11.0.10+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4420/2/testReport/
Max. process+thread count	637 (vs. ulimit of 30000)
modules	C: hbase-compression/hbase-compression-aircompressor hbase-compression/hbase-compression-brotli hbase-compression/hbase-compression-lz4 hbase-compression/hbase-compression-snappy hbase-compression/hbase-compression-xz hbase-compression/hbase-compression-zstd U: hbase-compression
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4420/2/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-05-13T22:19:57Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 25s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+0 🆗	mvndep	0m 11s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 23s	master passed
+1 💚	compile	1m 43s	master passed
+1 💚	checkstyle	0m 47s	master passed
+1 💚	spotless	0m 42s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 47s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 11s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 12s	the patch passed
+1 💚	compile	1m 39s	the patch passed
+1 💚	javac	1m 39s	the patch passed
+1 💚	checkstyle	0m 46s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	11m 46s	Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
+1 💚	spotless	0m 41s	patch has no errors when running spotless:check.
+1 💚	spotbugs	2m 22s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 52s	The patch does not generate ASF License warnings.
		34m 36s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4420/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4420
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux cae56a57ae04 5.4.0-96-generic #109-Ubuntu SMP Wed Jan 12 16:49:16 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `0eb1a4e`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Max. process+thread count	65 (vs. ulimit of 30000)
modules	C: hbase-compression/hbase-compression-aircompressor hbase-compression/hbase-compression-brotli hbase-compression/hbase-compression-lz4 hbase-compression/hbase-compression-snappy hbase-compression/hbase-compression-xz hbase-compression/hbase-compression-zstd U: hbase-compression
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4420/2/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

TRACE level logging is expensive enough to warrant removal. They were useful during development but are now just overhead. Also we unnecessarily create new compressor and decompressor instances in the reset() methods for the Aircompressor and Lz4 codecs. Remove. Signed-off-by: Viraj Jasani <vjasani@apache.org> Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>

apurtell requested review from Apache9 and virajjasani May 11, 2022 01:13

sunhelly approved these changes May 11, 2022

View reviewed changes

virajjasani approved these changes May 11, 2022

View reviewed changes

apurtell force-pushed the HBASE-27019 branch from 4841e5f to a874956 Compare May 13, 2022 21:32

apurtell merged commit 369adcd into apache:master May 14, 2022

apurtell deleted the HBASE-27019 branch May 14, 2022 01:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HBASE-27019 Minor compression performance improvements #4420

HBASE-27019 Minor compression performance improvements #4420

apurtell commented May 11, 2022

Apache-HBase commented May 11, 2022

Apache-HBase commented May 11, 2022

Apache-HBase commented May 11, 2022

Apache-HBase commented May 13, 2022

Apache-HBase commented May 13, 2022

Apache-HBase commented May 13, 2022

HBASE-27019 Minor compression performance improvements #4420

HBASE-27019 Minor compression performance improvements #4420

Conversation

apurtell commented May 11, 2022

Apache-HBase commented May 11, 2022

Apache-HBase commented May 11, 2022

Apache-HBase commented May 11, 2022

Apache-HBase commented May 13, 2022

Apache-HBase commented May 13, 2022

Apache-HBase commented May 13, 2022