Roadmap

Current

• Evaluate eviction policies that improve upon LRU
  • Acquire traces from production workloads to understand the trade-offs
    • SNIA
    • Internet Traffic Archive
    • UMass Trace Repository
    • Contributions appreciated!
  • Decide between LIRS and EdenQueue
    • Comparable hit rates across all traces
    • LIRS: Complexity with high non-resident memory usage
    • EQ: Optimize sketch algorithm to reduce memory and cpu cost
      • Reducing to 1/16th is straightforward; to 1/64th is possible
      • Doug Lea supports adopting this approach if memory can be tamed
• Benchmark arena spacing vs padding for better false sharing avoidance
• Explore better fork-join scheduling for maintenance ("chewing up threads")
  • This only happened on a synthetic load test so FJ was thrashing
  • To combat that, our benchmarks use a single-threaded FJ pool
  • A little smarter scheduling should resolve this quirk

Version 2.0

A major revision that includes API incompatible changes.

• Potentially adopt alternative eviction policy
  • Based on simulation analysis and algorithm compatibilities
  • Allow hot reads to skip maintenance work (maybe 2-3x speedup?)
• RemovalListener
  • Use friendlier typed lambda syntax to chain as (k, v, cause) -> ...
  • RemovalNotification retained as helper, but not used in the API
• AsyncLocalCache
  • Consider extending Cache<K, CompletableFuture<V>> interface
    • Cache: Conflict on get(k, k -> v) with get(k, k -> future)
    • LoadingCache: Conflicts on getAll returning future<map> vs map<k, future>
  • Add asMap() view of key -> future
• Stats
  • Show disabled stats as either -1 or provide a flag on Policy
• Tracing
  • Remove deprecated methods
  • Use snake case for system properties (consistency with Typesafe Config library)
  • This package may be removed entirely after integrating a optimal replacement policy
• CacheLoader
  • Revisit having loads throw checked exceptions (forgot why it was decided against)
• JCache
  • Rename CopyStrategy to Copier
• Misc
  • Remove NonReentrantLock (embed usage)
  • Consider removal of UnsafeAccess (or wait until VarHandles rewrite?)
  • Review ConcurrentLinkedStack for promotion out of Beta status

Future

• Collections
  • MPSC linked array queue [slides]
• Simulator
  • Add weight and expiration support
• Consider encoding the linked lists as arrays
  • 64-bit reference becomes 32-bit int (only if compressed OOPS is not enabled)
  • Better caching effects due to contiguous data
  • Cons: Limits the size (2B entries), cost is minimal due to maintenance done using ForkJoinPool