Resuming a long-lived session fails with 401 "OAuth access token has expired" and login: does not fix it; new sessions work

[chrispatil]

Version: prime-agent 0.10.0 (0.10.0-darwin-arm64-e418bdf6…), macOS arm64, Anthropic OAuth login

Related: #3617 reports the same failure (auto-closed, reporter not vouched). Posting here per the README's bug-report route, with an independent reproduction.

Symptom

Resuming a session that had been held by a worker process for ~3 days returned:

⚠ Error: Retry failed after 1 attempts: Provider authentication failed (authentication_error, 401):
OAuth access token has expired. Re-authenticate to continue. [request_id: req_011Cg82mVFppxWFx48V5UgnK]

Running login completed successfully and rewrote ~/.prime/agent/auth.json (mtime updated, new expires ~8 h out), but resuming the same session produced the identical error. Starting a new session at the same time worked normally.

What I observed

• The daemon and three prime-agent worker processes had been running since the session was last active (3 days). The resumed session attached to one of those old workers.
• Each new session spawned a fresh prime-agent worker, which authenticated fine.
• Killing the old worker processes (daemon left running) and resuming again fixed it: a new worker was spawned for the session and the fresh token was used. No transcript data was lost.

Expected

After login succeeds, resuming any session should use the new credentials. At minimum, the error message should not tell the user to re-authenticate when re-authenticating has no effect on the process that is failing.

Hypothesis

Workers hold their own in-memory copy of the access/refresh pair loaded at worker start. If refresh tokens are rotated on use, then once any process (another worker, or login) refreshes and rewrites auth.json, every other long-lived worker's cached refresh token is revoked. When that worker's access token expires, its refresh fails and it reports the 401. Nothing re-reads auth.json in a running worker, so login can't repair it.

I couldn't verify this from source (the binary is compiled), but #3617's observation 4 — a refresh in one session immediately producing 401: OAuth access token has been revoked in older workers — is what this predicts.

Suggested fixes (any one would do)

1. On a 401 from the provider, re-read auth.json and retry once before surfacing the error.
2. Have the daemon own credentials and hand tokens to workers on request, so there is one copy to refresh.
3. If neither, make --resume spawn a fresh worker when the existing worker's credentials are stale, or at least say in the error "restart the worker" rather than "re-authenticate".

Workaround

ps aux | grep 'prime-agent worker'
kill <old worker pids>
prime-agent --resume

[chrispatil]

Linking #3575, which reports the same root cause with a revoked token instead of an expired one: a long-lived worker keeps using its in-memory token after auth.json has a fresh one.

One point to add to my report: this isn't only a problem when resuming an idle session. On 0.10.1 it stopped a session that was actively running a multi-hour task. #3393 makes /reload re-read the credential store, which may help with recovery. But a session shouldn't need a manual /reload or a worker restart to keep running past a token refresh. Running for hours without supervision is a main reason to use Prime Agent for long tasks.

[chrispatil]

Follow-up with a root cause, found with prime-agent and checked against the v0.10.1 source. Line numbers are at the v0.10.1 tag.

Cause: a running daemon session resolves its request key once, when the session is built (crates/pa-daemon/src/agent_engine/lifecycle.rs:1036). switchable_stream_fn then clones that captured key for every request (crates/pa-core/src/session_engine/provider_adapter.rs:101-119). The single auth retry reads the same slot, so it re-sends the rejected token (provider_retry.rs:152, turn/model.rs:363-368). AuthStorage itself reads auth.json fresh, but the main turn never asks it again. 0.9.8 looked up the key before every request (sdk.ts:301-313), which matches my history: no expired or revoked 401s on 0.9.x, recurring ones since 0.10.0. As far as I can tell, #3651 rebinds keys only on /login and /logout, so it would not cover a token that expires or rotates mid-session.

What I'm running locally: a patched v0.10.1 build that does two things:

• It re-resolves the live target's key from the store before each request, so an expired token is refreshed first.
• On an auth 401, it recovers only the credential that the failed request was sent with: it uses a newer token from disk, or forces a refresh if disk still has the rejected one, then retries once.

A review of that patch found two refresh bugs that already exist in v0.10.1 and that the patch would have hit more often:

1. The refresh flight is released before the new credential is saved (crates/pa-core/src/auth/manager/lookup.rs:185-204 vs :213-229). So one process can spend a single-use refresh token twice, and separate workers have no shared guard at all.
2. The save step only keeps a credential that is usable OAuth (lookup.rs:215-225). So a refresh that finishes after a logout, or after a switch to an API key, writes the old OAuth credential back.

The patch fixes both: it holds the flight plus a per-provider lock file through the save, and the save only goes through if the stored credential hasn't changed. Tests cover rotation before a turn, expiry in a running session, revoked tokens, and two processes sharing one auth.json. The full pa-core and pa-daemon suites show no failures beyond those that also fail on unmodified v0.10.1 on macOS.

Result so far: a session whose worker sat idle for about 22 hours refreshed its token and continued without a 401. Before I moved that session to the patched build, it got 401 has been revoked on 0.10.1 right after the patched daemon refreshed the shared token, which is the #3575 case. I haven't yet tested a session working without a break past an expiry. Print/headless mode and inline RLM children still capture the key at build time, and I didn't change them.

I'm not a vouched contributor, so I haven't opened a PR. I'm happy to share the patch as a diff or gist if it's useful.

[baudehlo]

Same bug as #3575, seen from the revoked-token side. The root cause above explains both. The open fixes are #3392 and #3651; neither is merged or released yet (still reproducing on v0.10.2).