Skip to content

Add unified cache infrastructure with NATS KV and in-memory LRU backends #2951

Description

@migmartri

Problem

The control plane has three ad-hoc caching mechanisms with inconsistent patterns:

  1. Membership cache — package-level expirable.LRU with 1s TTL
  2. JWT claims cache — local expirable.LRU inside middleware with 10s TTL
  3. Remote policy/group caches — package-level map + sync.Mutex with no TTL (unbounded growth, latent memory leak)

These caches are not configurable, use global state, and are limited to single-process in-memory storage with no option to share cache state across replicas.

Proposal

Introduce a unified Cache[T any] interface in pkg/cache/ with two backends:

  • NATS JetStream KV — used when NATS is configured, enables shared caching across replicas
  • In-memory LRU — automatic fallback when NATS is not configured

A single constructor auto-selects the backend based on whether a NATS connection is provided. All existing ad-hoc caches are migrated to use this abstraction.

Key properties:

  • Generic interface: Get, Set, Delete, Purge
  • TTL-based expiration on both backends
  • Structured debug/warn logging
  • No external framework dependencies in the interface — reusable across projects
  • Eliminates package-level global cache state

Spec: docs/superpowers/specs/2026-03-28-cache-infrastructure-design.md

Metadata

Metadata

Assignees

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions