Cache TTL not honored in getOrExecute with in-memory storage (results never expire)

[Tim-Hoare]

Summary

CacheManager.getOrExecute() does not honor TTL when using InMemoryStorage. Once an entry is cached, the function passed to getOrExecute is never re-executed past the TTL — the stale value is served until the entry is evicted by LRU or the process restarts.

This silently breaks data freshness for any AppKit app using the analytics plugin without a Lakebase persistent cache, since the analytics plugin caches every query result via getOrExecute (packages/appkit/src/plugins/analytics/analytics.ts, defaults enabled: true, ttl: 3600).

In our deployed Databricks App we observed query results staying cached for >19 hours despite the default 1-hour TTL, until we set cache: { enabled: false } in createApp.

Root cause

Two pieces of code interact:

1. InMemoryStorage.get() (packages/appkit/src/cache/storage/memory.ts) returns entries unconditionally, without checking expiry. This is documented as intentional — the test should return expired entry on get (expiry check is done by CacheManager) makes the contract explicit: expiry handling is the CacheManager's responsibility.

2. CacheManager.getOrExecute() (packages/appkit/src/cache/index.ts) calls this.storage.get(cacheKey) directly and returns cached.value if non-null, without an expiry check:

   const cached = await this.storage.get<T>(cacheKey);
   if (cached !== null) {
     // hit — returned immediately, expiry never checked
     return cached.value as T;
   }

   By contrast, the standalone CacheManager.get() method does check Date.now() > entry.expiry. So the expiry data is being written correctly; only the getOrExecute path skips the check.

The probabilistic cleanupExpired path doesn't help either — it's gated by if (!this.storage.isPersistent()) return;, so it never runs against in-memory storage.

Net result: with the in-memory backend, cached entries returned by getOrExecute live until LRU eviction at maxSize (default 1000) or process restart.

Reproduction

Standalone Node script using the published @databricks/appkit@0.11.0:

import { InMemoryStorage } from "@databricks/appkit/dist/cache/storage/memory.js";
import { CacheManager } from "@databricks/appkit/dist/cache/index.js";

const cm = await CacheManager.getInstance({
  enabled: true,
  ttl: 1,
  storage: new InMemoryStorage({ maxSize: 100 }),
});

let executions = 0;
const fn = async () => { executions++; return `result-${executions}`; };

const r1 = await cm.getOrExecute(["k"], fn, "u", { ttl: 1 });
await new Promise(r => setTimeout(r, 2500));   // wait past 1s TTL
const r2 = await cm.getOrExecute(["k"], fn, "u", { ttl: 1 });

console.log({ r1, r2, executions });
// Actual:   { r1: 'result-1', r2: 'result-1', executions: 1 }
// Expected: { r1: 'result-1', r2: 'result-2', executions: 2 }

fn is provably not re-executed after the TTL has elapsed.

For completeness, a parallel call to the standalone cm.get() for the same expired key correctly returns null, confirming the issue is isolated to the getOrExecute code path.

Suggested fix

Either:

• In CacheManager.getOrExecute, replace the raw storage.get lookup with the expiry-aware logic from CacheManager.get (check Date.now() > entry.expiry, delete the entry, treat as miss). Inlining is straightforward; alternatively factor out a getValid() helper used by both.

• Or move the expiry check into the storage implementations and have all consumers go through it.

Either fix should be paired with a regression test in cache-manager.test.ts's describe("getOrExecute", ...) block — the existing TTL test ("should respect TTL expiry") only covers the set / get pair, not getOrExecute, which is why this slipped through.

I'm happy to put up a PR if the maintainers have a preferred direction.

Environment

• @databricks/appkit@0.11.0
• Node 20 (issue is logic-level, not runtime-specific)
• Bug reproduces on main as well — same code paths.

Workaround

Disable the cache globally:

await createApp({
  plugins: [server(...), analytics({})],
  cache: { enabled: false },
});

This short-circuits getOrExecute (if (!this.config.enabled) return fn();) and bypasses the bug entirely.