Caffeine cache extension #3028

drcrallen · 2016-05-26T20:23:51Z

A new local cache implementation using Caffeine as the cache.

We currently use this in production as l1 of a hybrid cache and memcached as the l2. The graph above shows the node mean TopN query time for production servers with and without using this cache in l1 (caffeine = false implies straight-to-memcached).

This extension requires java 8 to run, and therefore has a special profile that activates in the druid.io parent pom only when java8 is detected. This detection will eventually have to be improved to account for java9+

Packaging is accomplished by calling pull-deps a second time during the distribution build IFF the caller is using java8. Otherwise it retains the prior behavior.

In order to get this to work correctly, I had to put the distribution target in its own special rule set that will put it at the END of all other modules, including java8-only ones. If anyone else has a cleaner way of doing this, please let me know.

This is a contribution of https://github.com/metamx/druid-cache-caffeine

fjy · 2016-05-26T20:47:17Z

extensions-core/caffeine-cache/README.md

@@ -0,0 +1,41 @@
+Druid Caffeine Cache


layout doc_page

Fixed, and in general cleaned up the readme, and moved it to the correct location.

gianm · 2016-05-26T20:50:31Z

@drcrallen do you have any comparison of caffeine vs the existing local cache?

Also what does this part mean?

I have not yet figured out a good way to get the distribution to switch for jdk7 vs 8.

If that is just about the tarball, IMO it's ok for us to just have one distribution, with most modules targeting 1.7 but this module targeting 1.8. If the pom achieves that then I think it is good the way it is.

drcrallen · 2016-05-26T20:54:12Z

@gianm I didn't touch the distribution packaging extension. I think your suggestion is reasonable but would like to tackle distribution of this extension in a separate PR with appropriate community discussion and feedback.

drcrallen · 2016-05-26T20:56:55Z

For local cache I have no comparison because its lock contention falls flat on its face in our production cluster. See #1836 and related.

drcrallen · 2016-05-26T20:58:27Z

Also, this fixes #2411 (will add to master comment when I'm not on mobile)

gianm · 2016-05-26T21:02:19Z

@gianm I didn't touch the distribution packaging extension. I think your suggestion is reasonable but would like to tackle distribution of this extension in a separate PR with appropriate community discussion and feedback.

As long as we can get this figured out before the next release, that sounds fine. It's marked extensions-core (which SGTM) but as a core extension it should be in the tarball.

For local cache I have no comparison because its lock contention falls flat on its face in our production cluster. See #1836 and related.

Would it be possible to do synthetic benchmarks? I'm not gonna insist but I'm just curious and others might be too.

gianm · 2016-05-26T21:03:57Z

extensions-core/caffeine-cache/README.md

+
+### Jars
+
+For the sake of sanity if you are manually making an extension directory, release requires the following jars: 


What does "for the sake of sanity" mean in this context?

I can edit it to be clearer. I was trying to indicate this extension should be packaged with two jars (as of 0.9.0)

IMO before this is in a release we should figure out how to make it part of the tarball, meaning this section and the earlier "How to use" won't be necessary.

gianm · 2016-05-26T21:48:46Z

extensions-core/caffeine-cache/README.md

+
+# Versioning
+
+The versioning works like this: `druid_version.patch_set`. Such that `druid_version` is the version of druid the extension was compiled against, and `patch_set` is the "version" of the extension.


We don't do this with any other extensions. Any reason to have this here?

ah, no, need to remove.

gianm · 2016-05-26T21:50:17Z

👍 on the code, although would like to figure out the packaging before release if possible. For a core extension "it's in the tarball, just update loadList" is a better story than "build it yourself or use pull-deps".

drcrallen · 2016-05-26T21:54:57Z

@gianm I'll tag it for 0.9.2 for now. Hopefully I can bang around the distribution stuff a bit after 0.9.1 is qualified.

drcrallen · 2016-05-26T21:56:47Z

Anyone who REALLY REALLY wants to use this NOW can use https://github.com/metamx/druid-cache-caffeine .

One of the advantages of being in druid proper is the built-in pacakging, so I'm ok if it is desired to be solved before merging.

ben-manes · 2016-06-15T21:22:13Z

@drcrallen Can you try upgrading to 2.3.1? I assume you haven't looked into the failure, but perhaps it was due to ben-manes/caffeine#90.

drcrallen · 2016-06-15T21:28:21Z

@ben-manes thanks for the info. Yes I'll update for this PR

For our prod servers we're using single-threaded executor since the moral of the story in general is "don't use java common FJP"

ben-manes · 2016-06-15T21:38:39Z

In that case it was my fault. The bug was caused by all of the executor's task writing back to the cache. This would happen on the completion of a future (to assign a weight) or refresh to insert the new value. Previously the write buffer was unbounded, but when we switched to a bounded buffer I forgot to handle the case of all executor tasks performing a write-back. That meant none could proceed as they waited for an asynchronous task queued up, but with no threads available. This was surprising to have hit outside of a stress test given the available buffer slack, but it happened to a user.

It was an easy fix and also improved eviction throughput (2.5M/s => 4M/s). I don't think you'd hit this bug given your usage patterns, though.

drcrallen · 2016-06-22T00:38:11Z

Failed due to ben-manes/caffeine#93

See ben-manes/caffeine#93 (comment) for an explanation

fjy · 2016-07-01T21:27:41Z

docs/content/development/extensions-core/caffeine-cache.md

+Druid Caffeine Cache
+--------------------
+
+A highly performant local cache implementation for Druid based on [Caffeine](https://github.com/ben-manes/caffeine). Requires a JRE with a fix for https://bugs.openjdk.java.net/browse/JDK-8078490


can we add a note caution and just mention this extension only works for java 8?

fjy · 2016-07-01T23:27:57Z

docs/content/development/extensions.md

@@ -28,7 +28,8 @@ Core extensions are maintained by Druid committers.
 |druid-kafka-eight|Kafka ingest firehose (high level consumer).|[link](../development/extensions-core/kafka-eight-firehose.html)|
 |druid-kafka-extraction-namespace|Kafka-based namespaced lookup. Requires namespace lookup extension.|[link](../development/extensions-core/kafka-extraction-namespace.html)|
 |druid-lookups-cached-global|A module for [lookups](../querying/lookups.html) providing a jvm-global eager caching for lookups. It provides JDBC and URI implementations for fetching lookup data.|[link](../development/extensions-core/lookups-cached-global.html)|
-|druid-s3-extensions|Interfacing with data in AWS S3, and using S3 as deep storage.|[link](../development/extensions-core/s3.html)|
+|druid-s3-extensions|Interfacing with data in AWS S3, and using S3 as deep storage.|[link](../development/extensions-core/caffeine-cache.html)|
+|druid-caffeine-cache|A local cache implementation backed by Caffeine.|[link](../development/extensions-core/s3.html)|


minor nit, but can we sort these lexicographically?

sure, fixed

fjy · 2016-07-01T23:29:23Z

👍

drcrallen · 2016-07-01T23:32:40Z

@gianm I did some packaging magic, please check again

gianm · 2016-07-06T22:42:49Z

👍

Initial commit of caffeine cache

155729f

drcrallen added the Improvement label May 26, 2016

drcrallen changed the title ~~Initial commit of caffeine cache~~ Caffeine cache extension May 26, 2016

fjy reviewed May 26, 2016
View reviewed changes

gianm reviewed May 26, 2016
View reviewed changes

drcrallen added 2 commits May 26, 2016 14:46

Address code comments

eca2df2

Move and fixup README.md a bit

e580a27

gianm reviewed May 26, 2016
View reviewed changes

Improve caffeine readme information

e9d1025

drcrallen added this to the 0.9.2 milestone May 26, 2016

Cleanup caffeine pom

3cd5eba

Bump caffeine to 2.3.1

c6f9606

drcrallen added 2 commits June 21, 2016 13:36

Merge remote-tracking branch 'druid/master' into addCaffeineCache

37e3217

Bump druid version to 0.9.2-SNAPSHOT

b584da9

Make test not fail randomly.

c5869ae

See ben-manes/caffeine#93 (comment) for an explanation

This was referenced Jun 28, 2016

Better MapCache concurrency #1849

Closed

Local cache needs something better than LRU #1787

Closed

fjy reviewed Jul 1, 2016
View reviewed changes

drcrallen added 4 commits July 1, 2016 15:58

Merge remote-tracking branch 'druid/master' into addCaffeineCache

d653ffc

Fix distribution and documentation

64c3ea2

Add caffeine to extensions.md

6d15b72

Fix links in extensions.md

c00469a

fjy reviewed Jul 1, 2016
View reviewed changes

Lexicographic

e894bfa

gianm merged commit 3f1681c into apache:master Jul 6, 2016

drcrallen deleted the addCaffeineCache branch July 6, 2016 22:51

gianm mentioned this pull request Sep 9, 2016

updated dependency on sketches-core #3443

Merged

gianm mentioned this pull request Sep 23, 2016

Druid 0.9.2 release notes #3503

Closed

vogievetsky mentioned this pull request Aug 2, 2018

MapCache needs re-written #1836

Closed

jon-wei mentioned this pull request Dec 19, 2018

tranquility kafka version 52.0 #6752

Closed

ben-manes mentioned this pull request Mar 4, 2019

Admission window and the main body of cache on different servers ben-manes/caffeine#303

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Caffeine cache extension #3028

Caffeine cache extension #3028

drcrallen commented May 26, 2016 •

edited

Loading

fjy May 26, 2016

drcrallen May 26, 2016

gianm commented May 26, 2016

drcrallen commented May 26, 2016

drcrallen commented May 26, 2016

drcrallen commented May 26, 2016

gianm commented May 26, 2016

gianm May 26, 2016 •

edited

Loading

drcrallen May 26, 2016

gianm May 26, 2016

gianm May 26, 2016

drcrallen May 26, 2016

gianm commented May 26, 2016

drcrallen commented May 26, 2016

drcrallen commented May 26, 2016

ben-manes commented Jun 15, 2016

drcrallen commented Jun 15, 2016

ben-manes commented Jun 15, 2016

drcrallen commented Jun 22, 2016

fjy Jul 1, 2016

drcrallen Jul 1, 2016

fjy Jul 1, 2016

drcrallen Jul 1, 2016

fjy commented Jul 1, 2016

drcrallen commented Jul 1, 2016

gianm commented Jul 6, 2016


		### Jars

		For the sake of sanity if you are manually making an extension directory, release requires the following jars:


		# Versioning

		The versioning works like this: `druid_version.patch_set`. Such that `druid_version` is the version of druid the extension was compiled against, and `patch_set` is the "version" of the extension.

Caffeine cache extension #3028

Caffeine cache extension #3028

Conversation

drcrallen commented May 26, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gianm commented May 26, 2016

drcrallen commented May 26, 2016

drcrallen commented May 26, 2016

drcrallen commented May 26, 2016

gianm commented May 26, 2016

gianm May 26, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gianm commented May 26, 2016

drcrallen commented May 26, 2016

drcrallen commented May 26, 2016

ben-manes commented Jun 15, 2016

drcrallen commented Jun 15, 2016

ben-manes commented Jun 15, 2016

drcrallen commented Jun 22, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fjy commented Jul 1, 2016

drcrallen commented Jul 1, 2016

gianm commented Jul 6, 2016

drcrallen commented May 26, 2016 •

edited

Loading

gianm May 26, 2016 •

edited

Loading