AdaptiveCache

[avibryant]

This is an attempt to fix the problem of sometimes having to lower the capacity used by SummingCache when experiencing OOM errors during MapsideReduce in Scalding.

The general idea is that along with a SummingCache, we keep a second, "sentinel" cache that sums the evicted items, but holds onto these values with soft references. If we're ever experiencing memory pressure, the GC will clear these references and notify us. But if the sentinel cache has grown to size X without that happening, that's good evidence that we should be ok to grow our main SummingCache by something close to X (since, had we used that larger capacity to begin with, we empirically would have had enough memory to cache all of the values that we evicted instead).

[johnynek]

can we make the member's private? Especially the vars?

[avibryant]

oops, yep, fixed

[avibryant]

@johnynek added a test and I think addressed your other issues
@reconditesea added a putWithStats which is like (and uses) putWithHits but also includes a Boolean as to whether it grew, so that this can also be fed into a hadoop counter or whatever. (Alternatively we could feed in the amount by which the capacity grew; since for now it always starts at 1 and doubles, they're basically equivalent).

[reconditesea]

LGTM.

[avibryant]

Thanks. I'm running some stress tests with it and seeing some OOMs that I'm trying to figure out, so don't merge yet; will update.

[avibryant]

@johnynek @reconditesea ok, it turned out that using a SoftReference for every key in the map created too much memory pressure and GC overhead dealing with the refs themselves. I've much simplified it to use a single SoftReference for the entire HashMap, and am just careful to never hold a strong reference to the map while doing anything that's likely to allocate much. It now performs well in stress tests.

[avibryant]

Anything else you'd like from this pre-merge?

[reconditesea]

Nit pick: call the case class CacheStats? Nothing major, just it might be confused with the Scalding Stat and these two are likely to be used together when recording these metrics to Hadoop counters.

[avibryant]

@reconditesea re CacheStats, no problem, done.

[reconditesea]

Seems there is a conflicts so cannot be automatically merged.
Merge when green.

[johnynek]

Looks good to me. We can merge this when it is clean (not sure why this is not clean now, I don't know what has conflicted).

[ianoc]

@avibryant bump fyi, just blocked on non-clean merge

[DanielleSucher]

Ping! This should merge cleanly now.

[ianoc]

Merged, thanks