Redis storage

Ccache 4.4+ has support for using the Redis protocol to communicate with a server for the purpose of sharing cache results with others. This page contains hints on how to set up such a server for use with ccache.

See the Remote storage backends section in the ccache manual for details on how to configure ccache.

See also HTTP storage.

Redis

Redis is an open source data store server that can be used as an LRU cache for remote ccache storage.

1. Add something like this to the Redis configuration:

   # Allow up to 2 GB of cache:
   maxmemory 2G
   # Configure LRU (least recently used) eviction:
   maxmemory-policy allkeys-lru

2. Add remote_storage = redis://YOUR_SERVER to your ccache configuration (or set CCACHE_REMOTE_STORAGE=redis://YOUR_SERVER if you're using environment variables).

NOTE: Since Redis is an in-memory database it may not be a good idea to use a single server for a large caches but instead set up a server cluster. That can be done with the shards attribute described in Remote storage backends.

NOTE: Since Redis is single-threaded, you might need to run more than one instance per server. See for instance 13 Years Later – Does Redis Need a New Architecture? which recommends not having more than 25 GB per Redis process.

TODO: Maybe write some recommendation about persistence?

Options:

• RDB (Redis Database): The RDB persistence performs point-in-time snapshots of your dataset at specified intervals.

• AOF (Append Only File): The AOF persistence logs every write operation received by the server, that will be played again at server startup, reconstructing the original dataset.

• None (No Persistence): The database is kept in-memory only, and deleted when the server is restarted for any reason.

• NDS (Naive Disk Store): The NDS persistence is implemented by persisting every change to a key to an on-disk database, through a periodic "flush" of all keys that have changed (controlled by the same logic as RDB dumps).

Note:

The NDS is based on a patched version of Redis. There are also other alternative Redis-compatible disk-based databases.

Aerospike

Aerospike is an open source database that can be used for remote ccache storage, both memory and disk based.

• https://github.com/aerospike/aerospike-server

One can use the Redis backend with the "skyhook" proxy, or a custom ccache Aerospike backend using libaerospike.

• https://github.com/aerospike-community/skyhook

Alternative servers

Most alternative servers use LevelDB or RocksDB for the disk storage. Some also offer memory-mapped databases.

Redis Enterprise

Commercial Redis (not Open Source) had something called "Redis on Flash".

The feature is now called Auto Tiering, and changed from RocksDB to Speedb.

References:

• https://redis.io/blog/introducing-auto-tiering/

• https://redis.io/blog/redis-on-flash-now-3-7x-faster/

Redis with NDS

See REDIS-NDS and LMDB (liblmdb).

Updated version REDIS-NDS (2.6->7.0)

References:

• Redis Rethought: Exciting Extremes with Larger-Than-Memory Datasets

• Second strike with Lightning! (replacing Kyoto Cabinet with LMDB)

• In-Memory Microbenchmark

• On-Disk Microbenchmark

KeyDB

TODO: https://github.com/Snapchat/KeyDB

SSDB

TODO: https://github.com/ideawu/ssdb

Ardb

TODO: https://github.com/yinqiwen/ardb

KvRocks

TODO: https://github.com/KvrocksLabs/kvrocks

RocksDB-Server

TODO: https://github.com/tidwall/rocksdb-server

Tendis

TODO: https://github.com/Tencent/Tendis

High Availability

High availability and cross-datacenter replication:

Redis Cluster

TODO: https://redis.io/docs/manual/scaling/

Redis Sentinel

TODO: https://redis.io/docs/manual/sentinel/

DynomiteDB

TODO: https://github.com/Netflix/dynomite

Proxy

Twemproxy

As an alternative to ccache's own client-side partitioning support via the shards attribute, you can use proxy-assisted partitioning with a local proxy like twemproxy (nutcracker). Configuration looks something like this:

pool:
  listen: 127.0.0.1:6379
  preconnect: true
  redis: true
  servers:
    - 1.2.3.4:6379:3 server1
    - 5.6.7.8:6379:4 server2

With this nutcracker.yml, connections to localhost (127.0.0.1) will be forwarded to server1 (1.2.3.4) and server2 (5.6.7.8). The weights (3 and 4 in the example) can be either 1 for all servers (even distribution) or the allocated memory for each in case of different sizes.

Redis Cluster Proxy

Redis Cluster is a deployment topology that needs client support. It has both master nodes and replica nodes in the cluster, always communicating with each other.

This client support is not available in ccache directly. However, you can use redis-cluster-proxy to communicate with such a cluster. It will handle the connection to the cluster and provides a regular address such as 127.0.0.1:7777.

Example:

redis-cluster-proxy --bind 127.0.0.1 --port 7777 127.0.0.1:7000