Skip to content

A redis cache library, with producer resilience easily configurable

License

Notifications You must be signed in to change notification settings

joao-fontenele/stable-cache

Repository files navigation

Stable Cache

npm Dependency Status devDependency Status Conventional Commits Test Coverage Maintainability Gitter chat

This lib makes extensive use of a concept called producer. A producer is the source of truth of a cached value. Producers are only called when needed to set a cache, and are usually not needed when there's a cache hit.

The proposal of this library is to make it easier to manage producers and how keys are fetched/saved to redis.

Features:

  • there's easily configurable resilience against producer failures:
  • optionally set redis ttl for produced values
  • optionally ignoring cached values and forcing values refresh
  • optionally refreshing keys in background. Useful when you want keys to be updated before their ttl would evict them.
  • optionally producing values in background, while returning the cached value, even on cache misses
  • builtin RTA for prometheus

Usage

Cache instance

If the circuitBreaker config is not provided this cache instance simply will not have a circuit breaker flow, and will still be usable.

I recommend to instantiate a cache for each service you consume, or else the circuit breaker would be shared. Which in turn could mean that a failing service would open the circuit for working services too.

const Redis = require('ioredis');
const { Cache } = require('stable-cache');

const redis = new Redis({
  port: 6379,
  host: 'redis', // localhost
});

const cache = new Cache({
  redis, // required to inject a redis client
  options: { // everything is optional from here
    name: 'someService', // name of the service the cache will handle, useful for RTA
    circuitBreaker: { // circuit breaker config for the service
      // percentage of failing producer requests to trigger the circuit breaker
      threshold: 0.2,
      // time window to be considered by the threshold, in milliseconds
      duration: 10000,
      // attempt to half open the circuit after this time in milliseconds
      halfOpenAfter: 20000,
      // don't open the circuit if there's less than this amount of RPS.
      // This avoids opening the circuit, during failures in low load periods.
      minimumRps: 5,
    },
  },
});

Cache.get method

Only the key argument is required. Other options are used to augment the producer flow. See more usage examples at lib/examples/cache-get-usage.js

Cache get options are:

NOTE: circuit breaker policy is configured only when instantiating the Cache class.

Option Description Default Behavior Default Value
producer A function that will be called when the source of the truth of the key is needed, has the signature function(): Promise<string> No producer to call null
ttl Time in milliseconds for the redis key ttl, in case the producer resolves No ttl set null
producerRetry Object configuration for the exponential backoff No retry null
producerRetry.maxDelay Max amount of delay between retry attempts, in milliseconds - 30000
producerRetry.maxAttempts Max amount of retries for the producer - 10
producerRetry.initialDelay Initial delay, in milliseconds before the retry - 128
producerTimeout Timeout in milliseconds for the producer. Even if there's a timeout, if the producer eventually resolves, the key will be set in background so it would be best if you configure a greater timeout on the producer itself, together with this config No timeout null
returnEarlyFromCache Whether to return the cached value, and make the producer call on background, on cache miss Await the producer on cache miss false
overrideCache Whether to ignore cached values and request producer, regardless of cache hits Don't ignore cache hits, and call producer only if needed false
shouldRefreshKey A callback that if returns true, calls the producer and sets the key on background. Has the signature function(key, currentTTL, options): boolean No automatic refresh of keys null

Example of all the configs:

function producer() {
  return new Promise((resolve) => {
    setTimeout(() => resolve('some-value'), 2000);
  });
}

cache.get(
  'some^key', // required redis key to get
  { // optional configs
    producer,
    ttl: 1000,
    producerRetry: {
      maxDelay: 30000,
      maxAttempts: 10,
      initialDelay: 128
    },
    producerTimeout: 1000,
    returnEarlyFromCache: false,
    overrideCache: false,
    shouldRefreshKey(key, currentTTL, options) {
      if (!options.ttl || currentTTL <= 0) {
        return false;
      }
      // options.ttl is the usually configured ttl for the given key
      const halfLife = options.ttl / 2;
      // refresh key if currentTTL is 50% below the initial ttl
      return currentTTL <= halfLife;
    }
  },
);

Cache.set method

Sets a key with a value, and receives an optional ttl for the key.

Cache set options:

Option Description Default Behavior Default Value
ttl Time in milliseconds for the redis key ttl No ttl set null

Example:

// sets `some^key` with value `a-value` with ttl of 30 seconds
cache.set(
  'some^key', // required key
  'a-value', // required value
  { // optional configs
    ttl: 30000, // 30 seconds
  }
);

Prometheus Exporter

There's an exporter for prometheus that exposes the following metrics

  • cache_results_count: counts cache hits/misses, with labels: ['service', 'operation', 'result']
  • cache_operations_duration_seconds: histogram for cache RT, with labels: ['service', 'operation']
  • cache_operations_count: counts cache operations made, with labels: ['service', 'operation']
  • producer_operations_duration_seconds: histogram with RT for producer calls, with labels: ['service']
  • producer_operations_result_count: counts the successes/errors for producer calls, with labels: ['service', 'result']
  • producer_circuit_break_state: gauge for the state of a service circuit breaker. A value of 0 means the circuit is closed (working normally), and a value of 1 means the circuit is open (fail fast is activated).

What labels mean?

  • service: is the cache name option
  • cache operation: type of redis operation, e.g. set, get.
  • cache result: cache hit or miss
  • producer result: success, or error

Usage

Prometheus exporter options for the constructor are:

Option Description Default Behavior Default Value
prefix Prefix to be added to every exported metric No prefix added ''
registers Array of prometheus registers to which metrics should be exported Use default prom-client register [Prometheus.register]
cacheBuckets Array of numbers, representing the histogram buckets for cache RT Use default value Prometheus.exponentialBuckets(0.05, 2, 8)
producerBuckets Array of numbers, representing the histogram buckets for producer RT Use default value Prometheus.exponentialBuckets(0.1, 2, 8)

Example:

const Prometheus = require('prom-client');
const { PrometheusExporter } = require('stable-cache');

const exporter = new PrometheusExporter({
  prefix: 'my_app_',
  registers: [Prometheus.register],
  cacheBuckets: Prometheus.exponentialBuckets(0.05, 2, 8),
  producerBuckets: Prometheus.exponentialBuckets(0.1, 2, 8),
});

exporter.collectMetrics(); // start collecting metrics