TriCache

tricache is an extremely fast three-tier Node.js cache library. It serves warm reads at 2.81 million operations per second from a single thread — over 100× faster than a localhost Redis round-trip and below any network latency floor. When L1 fills, evicted entries spill to a local NVMe disk tier rather than being dropped, keeping hit rates high without unbounded RAM growth. Cache misses that reach L2 (Redis or Valkey) are automatically coalesced: no matter how many concurrent callers miss the same key, fetchFn fires exactly once. See the performance section for full numbers. Stale-While-Revalidate, AES-256-GCM at-rest encryption, pub/sub fleet-wide invalidation, OOM guard, cold-start snapshots, and Prometheus metrics are also supported through optional configuration — with zero required fields to get started.

✨ Features

Feature	Detail
Adaptive eviction	LFU × LRU × priority score + Count-Min Sketch cross-eviction frequency; reservoir-sampled O(1) hot path; category limits prevent any prefix monopolising RAM
Count-Min Sketch	4 × 512 `Uint16Array` (4 KB) tracks historical access frequency across eviction boundaries — same-priority burst keys cannot displace long-resident entries; 78 % survival rate in benchmark flood tests
WASM Bloom filter	562-byte binary inlined as Base64 — O(k=7) guaranteed-miss detection, no filesystem access, pure-JS fallback
msgpackr serialization	All entries packed with msgpackr — uniform binary format, no JSON at any payload size
Stale-While-Revalidate	Serve stale instantly, revalidate in background — zero added latency on cache hit
Stale-if-error	Extend a stale entry's TTL when SWR revalidation fails — no errors served during upstream outages
Thundering-herd prevention	Inflight `Promise` registry — only one `fetchFn` call per key regardless of concurrency
Pub/sub invalidation backplane	Redis pub/sub channel propagates deletes across all instances in real time
Tag-based invalidation	Tag entries on write; `invalidateTag('catalog')` evicts all matching entries from L1, disk, and Redis atomically
Batch read	`mget()` collects L1 hits, calls `fetchFn` only for misses, preserves ordering
Batch write	`mset()` / `mdel()` write or delete many keys in a single `Promise.all` call
TTL jitter	`ttlJitterFactor` spreads expirations across a configurable ± window — prevents thundering-cliff mass-expiry
OpenTelemetry spans	Structural `ICacheTracer` / `ICacheSpan` interfaces — pass any OTEL-compatible tracer; no peer dep required
L2 circuit breaker	Suspends Redis after N consecutive failures; auto-probes after cooldown; state visible in `metrics()`
`warmFromL2(pattern)`	Scan Redis and pre-populate L1 at startup; returns count loaded; no-op when Redis unavailable
OOM guard	Polls `heapUsed/heapTotal` on a timer; emergency-evicts coldest L1 entries before the process crashes
Cold-start snapshot	L1 serialised to disk on `SIGTERM`/`SIGINT`, reloaded on next startup — warm cache, cold process
AES-256-GCM encryption	L2 (Redis) values, disk spill files, and snapshots encrypted at rest; zero-downtime key rotation via `previousEncryptionKey`
Prometheus metrics	`cache.metrics()` + `CacheService.toPrometheusText()` — drop into any `/metrics` endpoint
Distributed counter	`cache.increment()` backed by Redis `INCR` for distributed rate limiting; in-process fallback when Redis is disabled
Pluggable logger	Bring your own `pino`, `winston`, etc.
L2 read-only mode	`l2WriteMode: 'read-only'` reads from Redis but skips all writes — canary deploys, read replicas
Eviction callback	`onEviction(key, reason)` fires on every L1 eviction with a typed reason string
Negative caching (`notFoundTtl`)	Cache `null`/`undefined` fetchFn results for a configurable TTL — prevents hammering upstream on repeated misses
`setIfAbsent()`	Atomic "set if not cached" — L1 `has()` check → Redis `SET NX EX` → L1 set on success; returns `true` if written, `false` if already present
Refresh-ahead	Proactively recompute an entry in the background when remaining TTL falls below a configured fraction — zero-latency freshness
XFetch probabilistic early expiry	Probabilistic background recompute keyed to last fetch duration and `xfetchBeta` — optimal protection against expiry spikes under load
Adaptive TTL	Tracks per-key fetch latency in a rolling ring buffer; once ≥ 5 samples are collected, automatically sets TTL = `p95LatencyMs × multiplier`. Expensive keys get cached longer; cheap keys stay close to their base TTL — no manual TTL tuning required
`hotKeys(n)`	Returns top N keys by Count-Min Sketch access frequency with size — no full Map scan
`dependsOn` cascade invalidation	Tag entries with parent keys; deleting a parent automatically evicts all declared dependents from L1
`onHit` / `onMiss` callbacks	Per-operation hit/miss hooks with tier info (`'l1'` \| `'disk'` \| `'l2'`) — no wait for the metrics interval
`frozen` mode	Dev-time mutation guard — `Object.freeze()` applied recursively to every L1 hit so accidental mutations throw immediately
`tags` in `get()` opts	Attach tags at read time; when `fetchFn` populates the entry on a miss the tags are registered automatically

📦 Install

npm install tricache
# or
pnpm add tricache

🚀 Quick start

import { CacheService, CachePriority } from 'tricache';

// Get (or create) the process-level singleton
const cache = CacheService.create({
  redisHost: 'my-redis.example.com',   // omit or set NODE_ENV!=production to disable L2
});

// Get-or-fetch with a 5-minute TTL
const user = await cache.get(
  `user:${userId}`,
  () => db.users.findById(userId),
  300,
);

// Explicit set
await cache.set(`user:${userId}`, user, 300);

// Delete one key
await cache.delete(`user:${userId}`);

// Delete by glob pattern
await cache.delete(`user:${userId}:*`);

// Stale-While-Revalidate: serve stale for up to 30 s while refreshing in background
const dashboard = await cache.get(
  `dashboard:${orgId}`,
  () => analytics.buildDashboard(orgId),
  300,
  { swr: 30 },
);

// Distributed rate-limiting counter
const hits = await cache.increment(`ratelimit:${ip}`, 60 /* TTL seconds */);

// Check if a key is cached (fast, no fetch)
const isCached = cache.has(`user:${userId}`);

// Batch read
const [userA, userB] = await cache.mget(
  [`user:${userIdA}`, `user:${userIdB}`],
  (missKeys) => db.users.findByIds(missKeys).then(rowsToMap),
  300,
);

// Batch write
await cache.mset({
  [`user:${userIdA}`]: { value: userA, ttl: 300 },
  [`user:${userIdB}`]: { value: userB, ttl: 300 },
});

// Batch delete
await cache.mdel([`user:${userIdA}`, `user:${userIdB}`]);

// Warm L1 from Redis at startup
const loaded = await cache.warmFromL2('user:*');
console.log(`Pre-warmed ${loaded} user entries`);

// Or auto-warm at construction + gate traffic with ready()
const cache2 = CacheService.create({ warmKeys: 'user:*' });
await cache2.ready(); // resolves once warm-up completes — ideal for k8s readiness probes

// Atomic set-if-absent — returns true if written, false if key already cached
const written = await cache.setIfAbsent(`session:${id}`, sessionData, 3600);

// Dependency cascade: deleting 'org:42' automatically evicts 'org:42:config'
await cache.set('org:42:config', config, 300, undefined, { dependsOn: ['org:42'] });
await cache.delete('org:42'); // also evicts org:42:config

// Top 10 hottest keys by Count-Min Sketch frequency
const hot = cache.hotKeys(10);
console.log(hot); // [{ key: 'user:1', hits: 842, sizeBytes: 512 }, ...]

// Tag entries for group invalidation
await cache.set(`product:${id}`, product, 300, undefined, { tags: ['catalog'] });
await cache.invalidateTag('catalog'); // evict all catalog entries

// Health check with tier latencies
const { l1, disk, l2 } = await cache.ping();

// Prometheus metrics
const snap = cache.metrics();
console.log(CacheService.toPrometheusText(snap));

⚙️ Configuration

All options are optional — sensible defaults apply.

CacheService.create({
  // ── Namespace ─────────────────────────────────────────────────────────
  // Isolates keys, disk dir, snapshot file, and Redis backplane channel.
  // Two instances with different namespaces are fully independent.
  namespace: 'my-app',

  // ── Logger ────────────────────────────────────────────────────────────
  logger: pinoLogger,               // default: console warn/error only

  // ── L1 (in-memory) ───────────────────────────────────────────────────
  l1MaxBytes:   200 * 1024 * 1024,  // 200 MB total RAM cap (default)
  l1MaxEntries: 2_000,              // max entries in L1 (default)
  l1EvictionWatermark: 0.9,         // proactive eviction fires at 90 % of l1MaxEntries / l1MaxBytes (default)
                                    // lower to 0.8 to reduce GC pressure on heap-bound workloads
  categoryLimits: {
    // per-prefix limits — keys are matched by startsWith()
    'user:':      { maxEntries: 500,  maxSizeBytes: 50  * 1024 * 1024 },
    'analytics:': { maxEntries: 100,  maxSizeBytes: 20  * 1024 * 1024 },
    'default':    { maxEntries: 1000, maxSizeBytes: 100 * 1024 * 1024 },
  },

  // ── L1.5 (disk spill) ────────────────────────────────────────────────
  diskCacheDir:      '/tmp/my-app-cache',  // default: os.tmpdir()/tricache-disk
  diskMaxBytes:      500 * 1024 * 1024,   // 500 MB (default)
  diskEntryMaxBytes: 10  * 1024 * 1024,   // 10 MB per entry (default)

  // ── L2 (Redis / Valkey) ──────────────────────────────────────────────
  redisHost:    'my-redis.example.com',   // or REDIS_HOST env var
  redisPort:    6379,
  redisTls:     true,                     // default: true when NODE_ENV=production
  disableRedis: false,                    // default: true when NODE_ENV!=production

  // ── Invalidation backplane ───────────────────────────────────────────
  // Redis pub/sub channel that propagates deletes to all instances.
  // Enabled by default when Redis is active.
  invalidationBackplane: true,

  // ── OOM guard ────────────────────────────────────────────────────────
  oomProtection:      true,   // enabled by default
  oomHeapThreshold:   0.85,   // evict when heapUsed/heapTotal > 85 %
  oomCheckIntervalMs: 10_000, // poll every 10 s
  oomEvictPercent:    0.20,   // evict coldest 20 % of L1 per trigger

  // ── Encryption ───────────────────────────────────────────────────────
  // base64-encoded 32-byte key; or set CACHE_ENCRYPTION_KEY env var.
  // node -e "console.log(require('crypto').randomBytes(32).toString('base64'))"
  encryptionKey: process.env.CACHE_ENCRYPTION_KEY,

  // Zero-downtime key rotation — remove after all old entries have expired
  previousEncryptionKey:  process.env.PREV_ENCRYPTION_KEY,
  previousEncryptionMode: 'aes-256-gcm', // defaults to current encryptionMode

  // ── L2 write mode ────────────────────────────────────────────────────
  // 'read-write' (default) — reads and writes to Redis
  // 'read-only'            — reads from Redis, skips all writes (canary / replica)
  l2WriteMode: 'read-write',

  // ── Stale-if-error ───────────────────────────────────────────────────
  // Extra seconds to extend a stale L1 entry's expiry when a SWR fetchFn fails.
  // Prevents serving errors while the upstream is temporarily down.
  staleIfError: 300, // keep stale for 5 more minutes on revalidation error

  // ── Eviction callback ────────────────────────────────────────────────
  // Called synchronously whenever L1 evicts a key.
  // reason: 'capacity' | 'category' | 'rebalance' | 'oom' | 'ttl' | 'manual'
  onEviction: (key, reason) => metrics.increment(`cache.eviction.${reason}`),

  // ── TTL jitter ────────────────────────────────────────────────────────
  // Multiply each TTL by a random factor in [1-j, 1+j] to spread expiry.
  // Prevents mass-expiry stampedes ("thundering cliff").
  // Range [0, 1]; default 0 (no jitter).
  ttlJitterFactor: 0.15,  // ± 15 % spread

  // ── Adaptive TTL ──────────────────────────────────────────────────────
  // When true, tricache tracks per-key fetch latency in a rolling ring
  // buffer and derives an optimal TTL from the p95 fetch duration:
  //   adaptedTtl = clamp(p95LatencyMs × multiplier, min, max)
  // The caller-supplied ttlSeconds is used until ≥ 5 samples are collected,
  // then the library takes over TTL management autonomously.
  adaptiveTtl:            true,
  adaptiveTtlMin:         10,      // floor: never assign TTL below 10 s (default)
  adaptiveTtlMax:         86400,   // ceiling: never exceed 24 h (default)
  adaptiveTtlMultiplier:  20,      // p95Ms × 20 = TTL in seconds (default)

  // ── OpenTelemetry tracer ──────────────────────────────────────────────
  // Pass any @opentelemetry/api-compatible tracer. No peer dependency.
  // Spans: 'tricache.get' | 'tricache.set' | 'tricache.delete'
  // Attributes: cache.key_prefix, cache.hit ('l1'|'disk'|'l2'|'miss')
  tracer: trace.getTracer('my-app'),

  // ── L2 circuit breaker ────────────────────────────────────────────────
  // Opens after N consecutive Redis errors; probes after cooldown ms.
  // State visible in cache.metrics().l2CircuitBreaker.state
  l2CircuitBreakerThreshold:  5,      // default
  l2CircuitBreakerCooldownMs: 30_000, // default

  // ── Negative caching ──────────────────────────────────────────────────
  // Cache null/undefined fetchFn results for this many seconds globally.
  // Prevents repeated upstream calls for keys that genuinely don't exist.
  // Can be overridden per-call via opts.notFoundTtl in cache.get().
  notFoundTtl: 30, // seconds; 0 = disabled (default)

  // ── Startup warm-up ───────────────────────────────────────────────────
  // Auto-call warmFromL2(pattern) at construction time.
  // cache.ready() resolves once warm-up finishes — use as a k8s readiness gate.
  // No-op when Redis is disabled or unreachable.
  warmKeys: 'user:*',

  // ── Prometheus instance label ─────────────────────────────────────────
  // Adds an `instance` label to every metric in toPrometheusText().
  instanceName: 'api-us-east-1',

  // ── Cold-start snapshot ──────────────────────────────────────────────
  snapshotPath:              '/tmp/my-app-cache-snapshot.msgpack',
  snapshotMaxAgeMs:          2 * 60 * 60 * 1000,  // 2 hours (default)
  forbiddenSnapshotPrefixes: ['auth:', 'session:', 'mfa:', 'rate_limit:'],

  // ── Metrics callback ─────────────────────────────────────────────────
  metricsIntervalMs: 60_000,                       // emit every 60 s (default)
  onMetrics: (m) => myMonitoring.record(m),        // optional push callback

  // ── Per-operation hooks ───────────────────────────────────────────────
  // onHit fires on every L1, disk, or L2 hit with the caller-facing key (no prefix)
  // and the tier that served it. Lower latency than waiting for onMetrics.
  onHit:  (key, tier) => cloudwatch.putMetricData({ key, tier }),

  // onMiss fires when all three tiers are exhausted — before fetchFn is called.
  onMiss: (key) => cloudwatch.putMetricData({ key }),

  // ── Development mutation guard ────────────────────────────────────────
  // When true, every L1 hit value is deep-frozen before being returned.
  // Mutation attempts throw TypeError immediately in development.
  // Do NOT enable in production — deep-freezing large objects has measurable overhead.
  frozen: process.env.NODE_ENV !== 'production',
});

Environment variables

Variable	Purpose
`REDIS_HOST`	Redis/Valkey hostname (used when `redisHost` option is not set)
`CACHE_ENCRYPTION_KEY`	Base64-encoded 32-byte AES-256-GCM key
`NODE_ENV`	When `!== 'production'`, L2 Redis and TLS are disabled by default

📖 API reference

`CacheService.create(options?)` → `CacheService`

Returns the process-level singleton. Options are only applied on the first call per namespace — subsequent calls return the existing instance.

`CacheService.createAsync(optionsOrPromise)` → `Promise<CacheService>`

Async factory that resolves a Promise<CacheOptions> before constructing the singleton. Useful when config is fetched from a secret store at startup.

const cache = await CacheService.createAsync(fetchSecretsFromVault());

`CacheService.reset(options?)` → `CacheService`

Destroys the existing singleton and creates a fresh one. Useful in tests.

`cache.get<T>(key, fetchFn, ttlSeconds?, opts?)` → `Promise<T>`

Get from cache or call fetchFn on a miss. The inflight map ensures fetchFn fires at most once per key regardless of concurrency.

Reference semantics: on an L1 hit, the returned value is the live JS object stored in the entry — not a deep copy. Mutating it will corrupt the cached entry. Deep-clone at the call site if you need an independent copy.

Parameter	Type	Default	Description
`key`	`string`	—	Cache key
`fetchFn`	`() => Promise<T>`	—	Called on a miss; result is cached
`ttlSeconds`	`number`	`300`	Hard TTL in seconds
`opts.swr`	`number`	`0`	Stale-While-Revalidate grace seconds
`opts.priority`	`CachePriority`	auto-inferred	Eviction priority override
`opts.refreshAhead`	`number`	—	Fraction `(0, 1]` of TTL — triggers background recompute when `remaining ≤ ttl × (1 - refreshAhead)`
`opts.xfetchBeta`	`number`	—	XFetch β ≥ 0 — scales probabilistic early recompute by last fetch duration; higher = recompute earlier
`opts.notFoundTtl`	`number`	—	Per-call TTL in seconds for `null`/`undefined` results (overrides global `notFoundTtl`)
`opts.tags`	`string[]`	—	Tags to register when `fetchFn` populates the entry on a miss; no-op on L1/L2 hits where tags are already registered

`cache.set<T>(key, data, ttlSeconds?, priority?, opts?)` → `Promise<void>`

Writes to L1 and (in production) L2. Publishes an invalidation to the backplane.

Parameter	Type	Default	Description
`opts.tags`	`string[]`	`[]`	Associate tags with this entry for group invalidation
`opts.dependsOn`	`string[]`	`[]`	Parent keys — when any parent is deleted, this entry is automatically evicted from L1

await cache.set('product:1', data, 60, undefined, { tags: ['catalog', 'featured'] });

// Cascade invalidation: evicting 'org:42' also evicts 'org:42:members'
await cache.set('org:42:members', members, 300, undefined, { dependsOn: ['org:42'] });
await cache.delete('org:42'); // org:42:members is evicted too

`cache.mget<T>(keys, fetchFn, ttl?, priority?)` → `Promise<(T | undefined)[]>`

Batch read. Returns L1-cached values for hot keys; calls fetchFn only with the keys that missed. Preserves input ordering.

ttl accepts a plain number (uniform TTL) or a function (key: string) => number (per-key TTL). The function is only called for miss keys — L1 hits are unaffected.

// Uniform TTL
const [userA, userB] = await cache.mget(
  ['user:1', 'user:2'],
  (missKeys) => db.users.findByIds(missKeys).then(rowsToMap),
  300,
);

// Per-key TTL — heterogeneous data in one batch call
const results = await cache.mget(
  ['user:1', 'config:global', 'feature:flags'],
  fetchFn,
  (key) => key.startsWith('config:') ? 3600 : 300,
);

`cache.mset<T>(entries)` → `Promise<void>`

Write multiple entries in a single call. Each entry accepts value, ttl, priority, and tags.

await cache.mset({
  'user:1': { value: alice, ttl: 300, priority: CachePriority.HIGH, tags: ['users'] },
  'user:2': { value: bob,   ttl: 300 },
});

`cache.mdel(keys)` → `Promise<void>`

Delete multiple keys in a single call. No-op for keys that do not exist.

await cache.mdel(['user:1', 'user:2', 'user:3']);

`cache.warmFromL2(pattern)` → `Promise<number>`

Scan Redis for keys matching a glob pattern (e.g. 'user:*') and load their values into L1 with a 10-minute TTL. Returns the number of keys loaded. Returns 0 immediately when Redis is disabled or unreachable — safe to call unconditionally at startup.

// In your application startup
const loaded = await cache.warmFromL2('user:*');
console.log(`Pre-warmed ${loaded} user entries from Redis`);

`cache.ready()` → `Promise<void>`

Returns a Promise that resolves once the cache is fully initialised. Without warmKeys, resolves immediately. With warmKeys, resolves once the automatic warmFromL2 call completes.

Designed for k8s readiness probes — await before accepting traffic, then never call again:

const cache = CacheService.create({ warmKeys: 'user:*' });

// k8s readiness probe endpoint
app.get('/ready', async (_req, res) => {
  await cache.ready();
  res.sendStatus(200);
});

`cache.has(key)` → `boolean`

Return true if the key exists in L1 and has not expired. Bloom-filter fast-path — no fetch, no disk or Redis round-trip.

`cache.ttl(key)` → `number | null`

Return the remaining TTL in seconds for a key currently held in L1. Returns null if the key is absent or expired. Does not fetch or consume the value.

const remaining = cache.ttl('user:123'); // e.g. 247 (seconds left)
if (remaining !== null && remaining < 30) await cache.touch('user:123', 300);

`cache.touch(key, newTtlSeconds)` → `Promise<boolean>`

Extend the TTL of a key in L1 (and fire-and-forget EXPIRE in Redis) without reading or re-fetching its value. Returns false if the key is absent or already expired.

`cache.getIfFresh<T>(key)` → `T | null`

Return the L1 value only if it is fresh (not yet in the SWR grace window). Returns null when absent, expired, or stale — without triggering a revalidation.

const fresh = cache.getIfFresh<User>('user:123');
if (fresh !== null) return fresh; // serve from L1, no network hop

`cache.setIfAbsent<T>(key, value, ttlSeconds?)` → `Promise<boolean>`

Atomically write value only if key is not already cached. Checks L1 first, then attempts a Redis SET NX EX. Returns true if the value was written, false if a live entry already existed.

Useful for distributed lock-style writes, session initialisation, or any pattern where you must not overwrite an already-cached value.

const written = await cache.setIfAbsent(`session:${id}`, sessionData, 3600);
if (!written) {
  // session already exists — do not overwrite
}

`cache.hotKeys(n?)` → `Array<{ key: string; hits: number; sizeBytes: number }>`

Returns the top n live L1 keys ranked by Count-Min Sketch access frequency. Namespace prefix is stripped from each key. Expired entries are excluded. Default n = 10.

const hot = cache.hotKeys(5);
// [
//   { key: 'user:1',    hits: 1024, sizeBytes: 512 },
//   { key: 'product:7', hits:  893, sizeBytes: 256 },
//   ...
// ]

`cache.invalidateTag(tag)` → `Promise<void>`

Evict all entries associated with a tag from L1, disk, and Redis.

await cache.set('product:1', data, 60, undefined, { tags: ['catalog'] });
await cache.set('product:2', data, 60, undefined, { tags: ['catalog'] });
await cache.invalidateTag('catalog'); // evicts both entries

`cache.ping()` → `Promise<CachePingResult>`

Measure L1 / disk / Redis latency in milliseconds. Returns { l1, disk, l2 } — l2 is null when Redis is disabled. Suitable for health-check endpoints.

app.get('/health', async (_req, res) => {
  const { l1, disk, l2 } = await cache.ping();
  res.json({ status: 'ok', latencyMs: { l1, disk, l2 } });
});

`cache.drainToL2()` → `Promise<number>`

Pipeline all live L1 entries to Redis in a single round-trip. Returns the number of keys written. Useful for warming a new Redis node or zero-downtime failover.

`cache.delete(key)` → `Promise<void>`

Deletes one exact key or a glob pattern (user:abc:*). Propagates to disk, Redis, and all backplane peers.

`cache.clear(prefix?)` → `Promise<void>`

Flush all entries, or only those whose key starts with prefix. Propagates to disk and Redis.

await cache.clear();           // flush everything
await cache.clear('session:'); // flush only session keys

`cache.rebalance()` → `void`

Evict L1 entries that now violate the current category or global capacity limits. Useful when categoryLimits are tightened after startup — normally, existing entries are not re-evaluated until they expire naturally.

// Tighten analytics limit at runtime, then immediately enforce it
cache.options.categoryLimits['analytics:'].maxEntries = 50;
cache.rebalance();

`cache.increment(key, ttlSeconds?)` → `Promise<number>`

Redis INCR — atomically increments a counter, setting TTL on first write. When Redis is disabled, maintains an in-process counter with the same TTL semantics so rate-limiting works in dev/test.

`cache.metrics()` → `CacheMetrics`

Returns a full metrics snapshot including hit rates, bloom filter stats, backplane counters, OOM eviction history, and tier sizes.

`CacheService.toPrometheusText(metrics, prefix?, instanceName?)` → `string`

Converts a CacheMetrics snapshot to Prometheus text exposition format. Pass instanceName to add an instance label alongside namespace.

app.get('/metrics', (_req, res) => {
  res.type('text/plain').send(
    CacheService.toPrometheusText(cache.metrics(), 'tricache', 'api-us-east-1'),
  );
});

`cache.stats()` → `{ l1, disk }`

Lightweight L1 and disk stats without the full metrics breakdown.

`cache.writeSnapshot(altPath?)` / `cache.loadSnapshot()`

Manual snapshot control. Called automatically on SIGTERM/SIGINT — only needed when you manage shutdown yourself. writeSnapshot() accepts an optional path to write to an alternate location without touching the configured default snapshot file.

// Graceful-shutdown hook — write to a dated backup path
process.on('SIGTERM', async () => {
  await cache.writeSnapshot(`/backups/cache-${Date.now()}.snap`);
  process.exit(0);
});

`cache.keys()` → `Generator<string>`

Lazily yields the key for every live (non-expired) L1 entry. Namespace prefix is stripped automatically. Uses a dedicated generator that skips intermediate tuple allocation.

for (const key of cache.keys()) console.log(key);

`cache.values<T>()` → `Generator<T>`

Lazily yields the cached value for every live L1 entry. Returns the live deserialized object (same reference semantics as get()). Uses yield* delegation — no intermediate generator frame.

for (const session of cache.values<Session>()) evict(session);

`cache.entries<T>()` → `Generator<[string, T]>`

Lazily yields [key, value] pairs for every live L1 entry. Key has namespace prefix stripped.

for (const [key, user] of cache.entries<User>()) sync(key, user);

JIT note: all three generators iterate SmartMemoryCache.cache (a single Map). V8 maintains per-call-site type feedback; having three generator functions share the same Map means no single one gets the full monomorphic specialization budget. In practice the throughput impact is ≤5 % relative to each running in isolation. See BENCHMARKS.md for numbers.

`cache.destroy()` → `Promise<void>`

Closes the Redis connection, unsubscribes the backplane, and stops all background timers.

🎯 Priority levels

import { CachePriority } from 'tricache';

CachePriority.LOW      // 1 — analytics, reports — evicted first
CachePriority.NORMAL   // 2 — general application data (default)
CachePriority.HIGH     // 3 — user profiles, config — evicted last
CachePriority.CRITICAL // 4 — never evicted while valid (auth tokens, sessions)

Priority is auto-inferred from the key when not specified:

Key contains	Inferred priority
`auth:` or `session:`	`CRITICAL`
`user:`, `org:`, or `profile:`	`HIGH`
`analytics:`, `report:`, or `stats:`	`LOW`
anything else	`NORMAL`

🧠 Eviction algorithm

L1 eviction uses reservoir sampling — an O(n) single pass samples 16 candidates, then sorts only those 16 (O(1)). Each candidate is scored:

score = priority × 1000 + min(hits, 100) × 10 + ttlRemaining/60s − age/60s

Higher score = kept longer
CRITICAL entries are excluded from sampling while valid
When a category limit is breached, entries from that category receive a score penalty

🪵 Pluggable logger

Bring your own structured logger — tricache doesn't care if it's pino, winston, or console.

import pino from 'pino';
const logger = pino();

CacheService.create({
  logger: {
    debug: (msg, meta) => logger.debug(meta ?? {}, msg),
    info:  (msg, meta) => logger.info(meta  ?? {}, msg),
    warn:  (msg, meta) => logger.warn(meta  ?? {}, msg),
    error: (msg, meta, err) => logger.error({ ...(meta ?? {}), err }, msg),
  },
});

🔐 Encryption

AES-256-GCM for L2 (Redis) values, disk spill files, and cold-start snapshots. Three modes are available via encryptionMode:

Mode	Key length	Notes
`aes-256-gcm`	32 bytes	Default. Authenticated encryption (AEAD).
`aes-128-gcm`	16 bytes	~15% faster than AES-256. Same AEAD guarantees.
`aes-128-ctr`	16 bytes	Fastest cipher mode. AES-NI keystream, no auth tag. Use when integrity is guaranteed elsewhere (TLS, HMAC).
`xor`	any (≥ 16 bytes recommended)	NOT cryptographic. XOR obfuscation only. Dev/non-sensitive data.

Key generation:

# AES-256 (32 bytes)
node -e "console.log(require('crypto').randomBytes(32).toString('base64'))"

# AES-128 / AES-128-CTR (16 bytes)
node -e "console.log(require('crypto').randomBytes(16).toString('base64'))"

# XOR — any length, minimum 16 bytes recommended
node -e "console.log(require('crypto').randomBytes(32).toString('base64'))"

// AES-256-GCM (default)
CacheService.create({ encryptionKey: '<base64-32-bytes>' });

// AES-128-GCM
CacheService.create({ encryptionKey: '<base64-16-bytes>', encryptionMode: 'aes-128-gcm' });

// AES-128-CTR (fastest cipher, no auth tag)
CacheService.create({ encryptionKey: '<base64-16-bytes>', encryptionMode: 'aes-128-ctr' });

// XOR obfuscation (NOT cryptographic — dev/non-sensitive only)
CacheService.create({ encryptionKey: '<base64-key>', encryptionMode: 'xor' });

// or use the env var: CACHE_ENCRYPTION_KEY=<base64-key>

Mode	Redis format	Disk / snapshot format
`aes-256-gcm`	`enc:v1:<base64(IV[12]\|Tag[16]\|CT)>`	`TRIC1ENC\|IV[12]\|Tag[16]\|CT[N]`
`aes-128-gcm`	`a128:v1:<base64(IV[12]\|Tag[16]\|CT)>`	`TRIC1128\|IV[12]\|Tag[16]\|CT[N]`
`aes-128-ctr`	`ctr:v1:<base64(IV[16]\|CT)>`	`TRIC1CTR\|IV[16]\|CT[N]`
`xor`	`xor:v1:<base64(key⊕data)>`	`TRIC1XOR\|key⊕data[N]`

Existing plaintext values are read transparently during key rotation.

Zero-downtime key rotation

Set previousEncryptionKey to your old key while rolling out a new one. The cache tries the current key first; if decryption fails it transparently retries with the previous key. Remove previousEncryptionKey once all old entries have expired.

CacheService.create({
  encryptionKey:         process.env.NEW_ENCRYPTION_KEY, // new AES-256 key
  previousEncryptionKey: process.env.OLD_ENCRYPTION_KEY, // fallback for old entries
  // previousEncryptionMode defaults to current encryptionMode
});

⚡ WASM Bloom filter

A 100,000-bit filter with k=7 hash probes:

At the default l1MaxEntries: 2,000 — false-positive rate ≈ 0.01%
At rated capacity (~18,000 entries) — false-positive rate ≈ 1%
The filter rebuilds automatically when stale bits from deleted/expired entries accumulate

Mechanics:

mightContain(key) === false → guaranteed miss — the Map lookup is skipped entirely
mightContain(key) === true → probable hit — the Map is checked to confirm

The 562-byte WASM binary is inlined as Base64 — zero filesystem access at runtime. Falls back to a pure-JS implementation if WebAssembly is unavailable.

📊 Performance

Measured on a single Node.js thread (no await on synchronous paths):

L1 SmartMemoryCache

Operation	Throughput	Latency	Notes
`get` — hot hit (8K entries)	2.81 M/s	356 ns	bloom → Map lookup → return cached value
`get` — cold miss	7.14 M/s	140 ns	bloom gates → early return
`set` — tiny payload	899 K/s	1.11 µs	pack() + Map.set + bloom.add
`set` — small payload (≈ 512 B)	554 K/s	1.81 µs	pack() same unified path, larger payload
`set` — large payload (≥ 512 B)	205.3 K/s	4.87 µs	pack() larger payload
`set` — CRITICAL priority	730.1 K/s	1.37 µs	same set path; skipped in eviction sort
`delete` — exact key	5.36 M/s	186 ns	Map.delete
`deletePattern` — glob wildcard	7.2 K/s	138 µs	O(n) Map scan
Count-Min Sketch estimate	3.37 M/s	297 ns	4 row lookups — called on every `get()` hit and `set()`

Iterator interface (L1 live entries, 500 entries)

Method	Throughput	Latency	Notes
`cache.keys()`	26.6 K/s	37.53 µs	no `[key,entry]` tuple allocation
`cache.values()`	35.5 K/s	28.19 µs	`yield*` delegation
`cache.entries()`	24.0 K/s	41.73 µs	`[strippedKey, value]` pairs
raw `Map` iteration (baseline)	277.2 K/s	3.61 µs	no expiry check, no generator overhead

CacheService (end-to-end)

Operation	Throughput	Latency	Notes
`get` — L1 warm hit	2.03 M/s	491 ns	inflight check → l1.get → return cached value
`get` — SWR stale serve	1.78 M/s	562 ns	serves stale; revalidates async
`get` — miss + fetchFn	13.7 K/s	73 µs	Promise microtask + l1.set
`set`	28.7 K/s	34.86 µs	l1.set + disk.save (fire-and-forget)
`delete` — exact key	7.3 K/s	137.82 µs	l1.delete + disk.delete + backplane
`delete` — glob `*`	687 K/s	1.46 µs	l1.deletePattern O(n) + disk glob

Encryption (IV pool, pre-allocated output buffers)

Mode	Payload	Encrypt	Decrypt
AES-256-GCM	64 B	140.4 K/s / 7.12 µs	155.5 K/s / 6.43 µs
AES-256-GCM	512 B	103.1 K/s / 9.70 µs	142.9 K/s / 6.99 µs
AES-256-GCM	4 KB	58.4 K/s / 17.12 µs	48.0 K/s / 20.84 µs
AES-128-GCM	64 B	148.8 K/s / 6.72 µs	173.0 K/s / 5.78 µs
AES-128-GCM	512 B	135.7 K/s / 7.37 µs	158.8 K/s / 6.30 µs
AES-128-GCM	4 KB	70.2 K/s / 14.24 µs	53.2 K/s / 18.79 µs
AES-128-CTR	64 B	187.9 K/s / 5.32 µs	196.9 K/s / 5.08 µs
AES-128-CTR	512 B	183.5 K/s / 5.45 µs	185.6 K/s / 5.39 µs
AES-128-CTR	4 KB	78.4 K/s / 12.75 µs	71.8 K/s / 13.93 µs
XOR (obfuscation only)	64 B	2.43 M/s / 412 ns	2.10 M/s / 476 ns
XOR (obfuscation only)	512 B	665.5 K/s / 1.50 µs	715.3 K/s / 1.40 µs
XOR (obfuscation only)	4 KB	114.5 K/s / 8.73 µs	77.6 K/s / 12.89 µs

AES and XOR string-path numbers shown (Redis L2). Buffer path (disk/snapshot) is 5–20% faster — no base64 overhead.
AES-128-GCM is 5–50% faster than AES-256-GCM depending on payload (gap widens at mid-range sizes on AES-NI hardware).
AES-128-CTR removes the GHASH MAC step: ~50% faster than AES-128-GCM at small payloads; use only when integrity is guaranteed by transport.
XOR numbers are for the buffer path (32-bit word-level XOR, 4 bytes/iteration). XOR dominates at small payloads (no cipher setup) and remains ~2× faster than AES at 4 KB.

See BENCHMARKS.md for the full breakdown: bloom filter cost, serialization by payload size, eviction pressure, concurrency analysis, multi-tenancy isolation, and a realistic 80/15/5 read/miss/write workload.

🤝 Contributing

Bug reports and pull requests are welcome!

Fork the repo and create a feature branch
Run pnpm test — all tests must pass
Run pnpm bench if you touch a hot path and include before/after numbers in your PR
Open your PR against master

New to the codebase? Start with src/cache-service.ts for the public API and src/smart-memory-cache.ts for the L1 engine.

🛡️ Security

Found a vulnerability? Please don't open a public issue. Report it privately via GitHub Security Advisories so it can be patched before disclosure.

For encryption key generation and rotation best practices, see the Encryption section.

📄 License

MIT

Name		Name	Last commit message	Last commit date
Latest commit History 37 Commits
.github/workflows		.github/workflows
bench		bench
public		public
src		src
tests		tests
.gitignore		.gitignore
BENCHMARKS.md		BENCHMARKS.md
CHANGELOG.md		CHANGELOG.md
CONTRIBUTING.md		CONTRIBUTING.md
LICENSE		LICENSE
README.md		README.md
SECURITY.md		SECURITY.md
package.json		package.json
pnpm-lock.yaml		pnpm-lock.yaml
pnpm-workspace.yaml		pnpm-workspace.yaml
tsconfig.json		tsconfig.json
vitest.config.ts		vitest.config.ts

Folders and files

Latest commit

History

Repository files navigation

TriCache

✨ Features

📦 Install

🚀 Quick start

⚙️ Configuration

Environment variables

📖 API reference

CacheService.create(options?) → CacheService

CacheService.createAsync(optionsOrPromise) → Promise<CacheService>

CacheService.reset(options?) → CacheService

cache.get<T>(key, fetchFn, ttlSeconds?, opts?) → Promise<T>

cache.set<T>(key, data, ttlSeconds?, priority?, opts?) → Promise<void>

cache.mget<T>(keys, fetchFn, ttl?, priority?) → Promise<(T | undefined)[]>

cache.mset<T>(entries) → Promise<void>

cache.mdel(keys) → Promise<void>

cache.warmFromL2(pattern) → Promise<number>

cache.ready() → Promise<void>

cache.has(key) → boolean

cache.ttl(key) → number | null

cache.touch(key, newTtlSeconds) → Promise<boolean>

cache.getIfFresh<T>(key) → T | null

cache.setIfAbsent<T>(key, value, ttlSeconds?) → Promise<boolean>

cache.hotKeys(n?) → Array<{ key: string; hits: number; sizeBytes: number }>

cache.invalidateTag(tag) → Promise<void>

cache.ping() → Promise<CachePingResult>

cache.drainToL2() → Promise<number>

cache.delete(key) → Promise<void>

cache.clear(prefix?) → Promise<void>

cache.rebalance() → void

cache.increment(key, ttlSeconds?) → Promise<number>

cache.metrics() → CacheMetrics

CacheService.toPrometheusText(metrics, prefix?, instanceName?) → string

cache.stats() → { l1, disk }

cache.writeSnapshot(altPath?) / cache.loadSnapshot()

cache.keys() → Generator<string>

cache.values<T>() → Generator<T>

cache.entries<T>() → Generator<[string, T]>

cache.destroy() → Promise<void>

🎯 Priority levels

🧠 Eviction algorithm

🪵 Pluggable logger

🔐 Encryption

Zero-downtime key rotation

⚡ WASM Bloom filter

📊 Performance

🤝 Contributing

🛡️ Security

📄 License

About

Topics

Resources

License

Contributing

Security policy

Uh oh!

Stars

Watchers

Forks

Releases 7

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

`CacheService.create(options?)` → `CacheService`

`CacheService.createAsync(optionsOrPromise)` → `Promise<CacheService>`

`CacheService.reset(options?)` → `CacheService`

`cache.get<T>(key, fetchFn, ttlSeconds?, opts?)` → `Promise<T>`

`cache.set<T>(key, data, ttlSeconds?, priority?, opts?)` → `Promise<void>`

`cache.mget<T>(keys, fetchFn, ttl?, priority?)` → `Promise<(T | undefined)[]>`

`cache.mset<T>(entries)` → `Promise<void>`

`cache.mdel(keys)` → `Promise<void>`

`cache.warmFromL2(pattern)` → `Promise<number>`

`cache.ready()` → `Promise<void>`

`cache.has(key)` → `boolean`

`cache.ttl(key)` → `number | null`

`cache.touch(key, newTtlSeconds)` → `Promise<boolean>`

`cache.getIfFresh<T>(key)` → `T | null`

`cache.setIfAbsent<T>(key, value, ttlSeconds?)` → `Promise<boolean>`

`cache.hotKeys(n?)` → `Array<{ key: string; hits: number; sizeBytes: number }>`

`cache.invalidateTag(tag)` → `Promise<void>`

`cache.ping()` → `Promise<CachePingResult>`

`cache.drainToL2()` → `Promise<number>`

`cache.delete(key)` → `Promise<void>`

`cache.clear(prefix?)` → `Promise<void>`

`cache.rebalance()` → `void`

`cache.increment(key, ttlSeconds?)` → `Promise<number>`

`cache.metrics()` → `CacheMetrics`

`CacheService.toPrometheusText(metrics, prefix?, instanceName?)` → `string`

`cache.stats()` → `{ l1, disk }`

`cache.writeSnapshot(altPath?)` / `cache.loadSnapshot()`

`cache.keys()` → `Generator<string>`

`cache.values<T>()` → `Generator<T>`

`cache.entries<T>()` → `Generator<[string, T]>`

`cache.destroy()` → `Promise<void>`

Packages