Expire Hash

TL;DR

Redis natively doesn’t support TTLs on redis hash fields (secondary keys). Dynomite client (Dyno) adds a new set of APIs to provide this functionality by maintaining additional metadata, transparent to the application.

Background

Redis hash stores a mapping of keys to values. A hash-key is associated with a hash and all its {field, value} pairs share a common TTL (timeout). On timeout expiry, the hash and all its items are deleted. Redis natively doesn’t support individual hash fields (secondary keys) with dedicated timeouts.

Design

Expire hash is purely a composite data type created and maintained by dyno client by providing a new set of APIs. Neither Jedis client, Dynomite Server or Redis are aware of this new data-type nor provide additional support in creating and maintaining it.

Expire Hash is a redis hash that supports TTL on Redis hash fields (secondary keys). Expire hash is composed of Redis hash to store data and Redis Sorted set to store secondary key TTL metadata. Applications will be able to directly control the expiration of hash fields through APIs. Both data and metadata is tracked in different redis data types and updated appropriately upon invoking relevant APIs. Each hash will have its fields’ TTL metadata stored in dedicated redis sorted set. TTLs are maintained as expiration timestamps since epoch, and will be stored as a score for each member in the sorted set.

Storage

Having a sorted set for metadata will introduce additional storage overhead. Also, the latency in reads/writes to expire hash will be higher due to metadata updates. Additional latencies are a result of removing expired fields.

Expire hash will have a lazy space reclamation where secondary keys are not space reclaimed immediately upon TTL expiry. These keys will be reclaimed either on a subsequent READ operation on the expire hash or upon hash-key TTL expiry. Metadata associated with the expire hash needs no additional clean-up when a TTL is set on the expire hash. TTL metadata relevant to individual secondary keys are space reclaimed along with its secondary key.

API

A new set of expire hash APIs will be added to dyno client for applications. Note that read operations can also result in updating hash fields and its metadata. Although redis hash APIs can mutate or fetch data from expire hash, an application using expire hash is expected to use only the new APIs both for mutation and read operations. Using redis hash commands on mixed hash can result in undefined outcomes and not supported.

Here is a list of supported APIs.

Example

Set the hashtag config.

dyno.{APP_NAME}.hashtag={}

Create a dyno client and use the expire hash APIs to Write, Read, modify and delete hash fields.

dynoClient = new DynoJedisClient.Builder()
				.withApplicationName(APP_NAME)
				.withDynomiteClusterName(DYNO_CLUSTER)
				.withDiscoveryClient(discoveryClient)
				.build();

client.ehset(hashKey, field, value, ttl);
.
.
client.ehget(hashKey, field);