HBASE-27796 Improve MemcachedBlockCache #5181

apurtell · 2023-04-14T21:29:32Z

Changes:

Track and log better stats.
Fix use of memcached API such that we are not accidentally immediately expiring keys.
Use a default retention period of 30 days, the max per memcached spec.
Use set instead of add to store keys.
Gracefully handle failures to cache and read timeouts.
Add unit tests using jmemcached as a test dependency.

Testing performed:

New unit tests pass
Stats logging of hit rate matches memcached side stats
After manually killing the memcached server reads continue to be served, the load client LTT reports no errors, and there are no errors logged in the regionserver logs except for memcached client, as expected. Memcached client errors stop once memcached is restarted.

The stats logging looks liks

2023-04-14 14:15:06,767 INFO  [MemcachedBlockCacheStatsExecutor] hfile.MemcachedBlockCache:
cached=275237, notCached=0, cacheErrors=10, timeouts=23, reads=474686, hits=193928, hitRatio=40.85%,
cachingAccesses=330164, cachingHits=137181, cachingHitsRatio=41.55%, evictions=0, evicted=0,
evictedPerRun=0.0

Track and log better stats. Fix use of memcached API such that we are not accidentally immediately expiring keys. Use a default retention period of 30 days, the max per memcached spec. Use set instead of add to store keys. Gracefully handle failures to cache and read timeouts. Add unit tests using jmemcached as a test dependency.

Apache-HBase · 2023-04-14T21:57:26Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 41s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 0s	master passed
+1 💚	compile	0m 13s	master passed
+1 💚	shadedjars	4m 15s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 12s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 53s	the patch passed
+1 💚	compile	0m 13s	the patch passed
+1 💚	javac	0m 13s	the patch passed
+1 💚	shadedjars	4m 15s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 10s	the patch passed
		_ Other Tests _
+1 💚	unit	0m 16s	hbase-external-blockcache in the patch passed.
		17m 13s

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5181/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5181
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 4eec755ed2f7 5.4.0-144-generic #161-Ubuntu SMP Fri Feb 3 14:49:04 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `856fa14`
Default Java	Temurin-1.8.0_352-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5181/1/testReport/
Max. process+thread count	66 (vs. ulimit of 30000)
modules	C: hbase-external-blockcache U: hbase-external-blockcache
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5181/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-04-14T22:04:04Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 25s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 44s	master passed
+1 💚	compile	0m 16s	master passed
+1 💚	shadedjars	5m 40s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 12s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 34s	the patch passed
+1 💚	compile	0m 16s	the patch passed
+1 💚	javac	0m 16s	the patch passed
+1 💚	shadedjars	5m 47s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 11s	the patch passed
		_ Other Tests _
+1 💚	unit	0m 21s	hbase-external-blockcache in the patch passed.
		23m 33s

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5181/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5181
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux ef6e6a0ad314 5.4.0-1097-aws #105~18.04.1-Ubuntu SMP Mon Feb 13 17:50:57 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `856fa14`
Default Java	Eclipse Adoptium-11.0.17+8
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5181/1/testReport/
Max. process+thread count	85 (vs. ulimit of 30000)
modules	C: hbase-external-blockcache U: hbase-external-blockcache
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5181/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-04-14T22:15:59Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 5s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	4m 10s	master passed
+1 💚	compile	0m 18s	master passed
+1 💚	checkstyle	0m 9s	master passed
+1 💚	spotless	0m 50s	branch has no errors when running spotless:check.
+1 💚	spotbugs	0m 24s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 10s	the patch passed
+1 💚	compile	0m 15s	the patch passed
-0 ⚠️	javac	0m 15s	hbase-external-blockcache generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1)
+1 💚	checkstyle	0m 7s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	xml	0m 1s	The patch has no ill-formed XML file.
+1 💚	hadoopcheck	14m 17s	Patch does not cause any errors with Hadoop 3.2.4 3.3.4.
+1 💚	spotless	0m 49s	patch has no errors when running spotless:check.
+1 💚	spotbugs	0m 30s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 10s	The patch does not generate ASF License warnings.
		35m 46s

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5181/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5181
Optional Tests	dupname asflicense javac hadoopcheck spotless xml compile spotbugs hbaseanti checkstyle
uname	Linux 1b4b86352a0d 5.4.0-144-generic #161-Ubuntu SMP Fri Feb 3 14:49:04 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `856fa14`
Default Java	Eclipse Adoptium-11.0.17+8
javac	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5181/1/artifact/yetus-general-check/output/diff-compile-javac-hbase-external-blockcache.txt
Max. process+thread count	82 (vs. ulimit of 30000)
modules	C: hbase-external-blockcache U: hbase-external-blockcache
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5181/1/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

hbase-external-blockcache/pom.xml

...-external-blockcache/src/main/java/org/apache/hadoop/hbase/io/hfile/MemcachedBlockCache.java

...ernal-blockcache/src/test/java/org/apache/hadoop/hbase/io/hfile/TestMemcachedBlockCache.java

...-external-blockcache/src/main/java/org/apache/hadoop/hbase/io/hfile/MemcachedBlockCache.java

…lues (they aren't)

Apache-HBase · 2023-04-17T17:07:27Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 19s	Docker mode activated.
-0 ⚠️	yetus	0m 4s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 3s	master passed
+1 💚	compile	0m 12s	master passed
+1 💚	shadedjars	4m 18s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 12s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 51s	the patch passed
+1 💚	compile	0m 13s	the patch passed
+1 💚	javac	0m 13s	the patch passed
+1 💚	shadedjars	4m 18s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 11s	the patch passed
		_ Other Tests _
+1 💚	unit	0m 17s	hbase-external-blockcache in the patch passed.
		17m 2s

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5181/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5181
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux f107921d9912 5.4.0-144-generic #161-Ubuntu SMP Fri Feb 3 14:49:04 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `31c4aea`
Default Java	Temurin-1.8.0_352-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5181/2/testReport/
Max. process+thread count	67 (vs. ulimit of 30000)
modules	C: hbase-external-blockcache U: hbase-external-blockcache
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5181/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-04-17T17:26:33Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 47s	Docker mode activated.
-0 ⚠️	yetus	0m 4s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	5m 51s	master passed
+1 💚	compile	0m 29s	master passed
+1 💚	shadedjars	9m 12s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 35s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	8m 0s	the patch passed
+1 💚	compile	0m 33s	the patch passed
+1 💚	javac	0m 33s	the patch passed
+1 💚	shadedjars	7m 42s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 15s	the patch passed
		_ Other Tests _
+1 💚	unit	0m 25s	hbase-external-blockcache in the patch passed.
		35m 18s

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5181/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5181
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux a42ab1a9938b 5.4.0-1097-aws #105~18.04.1-Ubuntu SMP Mon Feb 13 17:50:57 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `31c4aea`
Default Java	Eclipse Adoptium-11.0.17+8
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5181/2/testReport/
Max. process+thread count	84 (vs. ulimit of 30000)
modules	C: hbase-external-blockcache U: hbase-external-blockcache
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5181/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-04-17T17:31:36Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 13s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	4m 16s	master passed
+1 💚	compile	0m 16s	master passed
+1 💚	checkstyle	0m 9s	master passed
+1 💚	spotless	0m 44s	branch has no errors when running spotless:check.
+1 💚	spotbugs	0m 22s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	5m 9s	the patch passed
+1 💚	compile	0m 21s	the patch passed
+1 💚	javac	0m 21s	the patch passed
+1 💚	checkstyle	0m 8s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	xml	0m 2s	The patch has no ill-formed XML file.
+1 💚	hadoopcheck	15m 57s	Patch does not cause any errors with Hadoop 3.2.4 3.3.4.
+1 💚	spotless	1m 0s	patch has no errors when running spotless:check.
+1 💚	spotbugs	0m 35s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 12s	The patch does not generate ASF License warnings.
		41m 9s

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5181/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5181
Optional Tests	dupname asflicense javac hadoopcheck spotless xml compile spotbugs hbaseanti checkstyle
uname	Linux 41b33edccbfa 5.4.0-144-generic #161-Ubuntu SMP Fri Feb 3 14:49:04 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `31c4aea`
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	84 (vs. ulimit of 30000)
modules	C: hbase-external-blockcache U: hbase-external-blockcache
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5181/2/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Track and log better stats. Fix use of memcached API such that we are not accidentally immediately expiring keys. Use a default retention period of 30 days, the max per memcached spec. Use set instead of add to store keys. Gracefully handle failures to cache and read timeouts. Add unit tests using jmemcached as a test dependency. Signed-off-by: Viraj Jasani <vjasani@apache.org>

apurtell requested review from Apache9 and virajjasani April 14, 2023 21:29

virajjasani reviewed Apr 15, 2023

View reviewed changes

apurtell added 2 commits April 17, 2023 09:36

Address review feedback and fix the test category

ddd9ceb

Silence unhelpful error prone warnings about ignored future return va…

0c3e0b6

…lues (they aren't)

virajjasani approved these changes Apr 17, 2023

View reviewed changes

apurtell merged commit d56e7f2 into apache:master Apr 17, 2023
1 check passed

apurtell deleted the HBASE-27796 branch April 17, 2023 20:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HBASE-27796 Improve MemcachedBlockCache #5181

HBASE-27796 Improve MemcachedBlockCache #5181

apurtell commented Apr 14, 2023

Apache-HBase commented Apr 14, 2023

Apache-HBase commented Apr 14, 2023

Apache-HBase commented Apr 14, 2023

Apache-HBase commented Apr 17, 2023

Apache-HBase commented Apr 17, 2023

Apache-HBase commented Apr 17, 2023

HBASE-27796 Improve MemcachedBlockCache #5181

HBASE-27796 Improve MemcachedBlockCache #5181

Conversation

apurtell commented Apr 14, 2023

Apache-HBase commented Apr 14, 2023

Apache-HBase commented Apr 14, 2023

Apache-HBase commented Apr 14, 2023

Apache-HBase commented Apr 17, 2023

Apache-HBase commented Apr 17, 2023

Apache-HBase commented Apr 17, 2023