Problem
Background workers (distillation, curation, cache warming) use cached OAuth Bearer tokens from resolveAuth() in auth.ts. When tokens expire between client requests, all background work fails silently with 401 errors — no Sentry alert, no retry, no credential refresh.
Observed: 401 errors in Sentry breadcrumbs alongside prompt overflow errors (LOREAI-GATEWAY-V). The two issues are independent but co-occur in long sessions.
Impact
- Distillations deferred (not lost): messages stay
distilled=0 in SQLite, retried on next trigger
- Curations deferred:
turnsSinceCuration is not reset on failure, retried on next threshold
- Cache warming fails silently: prompt cache goes cold, next request incurs full cache write cost
- No Sentry visibility: 401s return
null in llm-adapter.ts:451 without captureException — operators only see them in breadcrumbs
Root Cause
AuthCredential in auth.ts:21-23 stores only { scheme, value } — no expiry tracking, no refresh mechanism, no staleness detection. Credentials are set once during handleConversationTurn() and never updated until the next client request arrives.
In llm-adapter.ts:41, TRANSIENT_CODES = [429, 500, 502, 503, 529] — 401 is non-transient, so the worker returns null immediately with no retry.
Suggested Fix
1. Detect and alert on 401 specifically in LLM adapter (llm-adapter.ts)
Add a specific 401/403 handler before the generic non-transient path. Log as error and Sentry.captureException() so 401s are visible in alerts, not just breadcrumbs.
2. Re-resolve credentials on 401 with one retry (llm-adapter.ts)
On 401, call getAuth() again — the credential store may have been updated by a concurrent client request. If the credential changed, retry once with the fresh token. This is a re-resolve, not a retry with the same stale credential.
3. Mark stale credentials in the session auth store (auth.ts)
Add a _stale flag to AuthCredential. When a 401 is detected, mark the credential as stale. resolveAuth() should skip stale credentials and fall back to global. Prevents all workers from hammering the API with the same expired token.
4. Cache warmer 401 awareness (cache-warmer.ts)
Detect 401 specifically and skip future warmups for that session until a fresh credential arrives (similar to how batch-queue.ts handles auth errors).
Key Files
packages/gateway/src/auth.ts — credential storage and resolution
packages/gateway/src/llm-adapter.ts — worker LLM client, error handling
packages/gateway/src/pipeline.ts — credential capture, background work dispatch
packages/gateway/src/cache-warmer.ts — warmup credential usage
packages/gateway/src/batch-queue.ts — reference for 401 handling pattern
Problem
Background workers (distillation, curation, cache warming) use cached OAuth Bearer tokens from
resolveAuth()inauth.ts. When tokens expire between client requests, all background work fails silently with 401 errors — no Sentry alert, no retry, no credential refresh.Observed: 401 errors in Sentry breadcrumbs alongside prompt overflow errors (LOREAI-GATEWAY-V). The two issues are independent but co-occur in long sessions.
Impact
distilled=0in SQLite, retried on next triggerturnsSinceCurationis not reset on failure, retried on next thresholdnullinllm-adapter.ts:451withoutcaptureException— operators only see them in breadcrumbsRoot Cause
AuthCredentialinauth.ts:21-23stores only{ scheme, value }— no expiry tracking, no refresh mechanism, no staleness detection. Credentials are set once duringhandleConversationTurn()and never updated until the next client request arrives.In
llm-adapter.ts:41,TRANSIENT_CODES = [429, 500, 502, 503, 529]— 401 is non-transient, so the worker returnsnullimmediately with no retry.Suggested Fix
1. Detect and alert on 401 specifically in LLM adapter (
llm-adapter.ts)Add a specific 401/403 handler before the generic non-transient path. Log as error and
Sentry.captureException()so 401s are visible in alerts, not just breadcrumbs.2. Re-resolve credentials on 401 with one retry (
llm-adapter.ts)On 401, call
getAuth()again — the credential store may have been updated by a concurrent client request. If the credential changed, retry once with the fresh token. This is a re-resolve, not a retry with the same stale credential.3. Mark stale credentials in the session auth store (
auth.ts)Add a
_staleflag toAuthCredential. When a 401 is detected, mark the credential as stale.resolveAuth()should skip stale credentials and fall back to global. Prevents all workers from hammering the API with the same expired token.4. Cache warmer 401 awareness (
cache-warmer.ts)Detect 401 specifically and skip future warmups for that session until a fresh credential arrives (similar to how
batch-queue.tshandles auth errors).Key Files
packages/gateway/src/auth.ts— credential storage and resolutionpackages/gateway/src/llm-adapter.ts— worker LLM client, error handlingpackages/gateway/src/pipeline.ts— credential capture, background work dispatchpackages/gateway/src/cache-warmer.ts— warmup credential usagepackages/gateway/src/batch-queue.ts— reference for 401 handling pattern