[Bug]: ENOTSUP when acquiring session lock file: implementation uses hard link (fs.link), breaks on SMB/NFS/virtiofs

[david-garcia-garcia]

Bug type

Crash (process/app exits or hangs)

Beta release blocker

No

Summary

Summary

When OpenClaw refreshes or coordinates work around the on-disk cache for aggregated session usage / cost data, it acquires a lock file under the agent sessions directory (e.g. ~/.openclaw/agents/<agentId>/sessions/). That lock is implemented with fs.promises.link() (a POSIX hard link) from a uniquely named temp file to the final *.lock path.

On several common filesystem and mount types, link(2) is not supported and Node reports ENOTSUP, so lock acquisition fails outright instead of competing cleanly for the lock.

Error (example)

Error: ENOTSUP: operation not supported on socket, link '/home/node/.openclaw/agents/main/sessions/.usage-cost-cache.json.lock.<pid>.<hrtime>.tmp' -> '/home/node/.openclaw/agents/main/sessions/.usage-cost-cache.json.lock': code=ENOTSUP

(Paths vary by agentId and user; the important part is link … *.lock and code=ENOTSUP.)

Expected behavior

Lock acquisition should work on typical local POSIX filesystems and common container writable layers, and should use a portable primitive (or fallback) on environments where hard links are unavailable (SMB/CIFS, some NFS setups, virtiofs/9p shared folders, certain FUSE stacks, etc.).

Actual behavior

link() fails with ENOTSUP, so the refresh lock cannot be created and dependent code paths error.

Root cause

In src/infra/session-cost-usage.ts, writeUsageCostCacheLockAtomically:

1. Writes the lock payload to a temp file with exclusive create (writeFile + flag: "wx").
2. Calls fs.promises.link(tempPath, lockPath) so concurrent processes get EEXIST if the lock already exists.

Step 2 requires hard-link support. Many network and shared-folder filesystems return ENOTSUP for link.

The lock file sits next to the usage cache: .usage-cost-cache.json → .usage-cost-cache.json.lock (see resolveUsageCostCacheLockPath / USAGE_COST_CACHE_FILE in the same module).

Note: The "operation not supported on socket" fragment in the message is often misleading; the salient point is ENOTSUP on link, not that the path is a socket.

Environments where this shows up

• Docker (or similar) with a bind mount to a host directory on SMB, exFAT, or another FS without hard links.
• VM shared folders (virtiofs, 9p).
• NFS setups where link is unsupported or rejected.

Suggested fix directions (for maintainers)

1. Portable exclusive create: open(lockPath, O_CREAT | O_EXCL | O_WRONLY) (or equivalent); on EEXIST, treat as “busy” and reuse existing stale-lock cleanup logic.
2. Fallback: try link first, on ENOTSUP / known unsupported errors fall back to O_EXCL create.
3. Docs: if behavior can’t be fully unified, document that ~/.openclaw (or at least agents/*/sessions) must live on a filesystem that supports hard links until the implementation changes.

Workaround (operators)

Until fixed: keep ~/.openclaw on a normal local filesystem inside the container/VM (e.g. a Docker volume on ext4/xfs), not an SMB/NFS/host shared-folder mount.

Related surface

Failure is at lock creation for the usage-cost cache refresh, not necessarily at reading/writing .usage-cost-cache.json itself (cache body uses replaceFileAtomic from @openclaw/fs-safe/atomic).

Steps to reproduce

Steps to reproduce (ENOTSUP / hard-link session lock)

1. Use a directory on a filesystem that does not support hard links (any one of these is enough):

   • Docker Desktop (Windows/macOS): bind-mount host data into the container where the host path is on SMB/CIFS, NTFS accessed via a share, or another limited FS.
   • Linux: place (or bind-mount) ~/.openclaw on SMB (//server/share/...), NFS with restricted link semantics, or exFAT.
   • VM: guest ~/.openclaw on a virtiofs / VirtualBox shared folder / 9p mount.

2. Optional sanity check on that same directory (proves the FS, not OpenClaw):

   mkdir -p /path/to/openclaw-test
   echo test > /path/to/openclaw-test/a && ln /path/to/openclaw-test/a /path/to/openclaw-test/b || true

   If ln fails with “Operation not supported” (or similar), you are in the failing class of filesystems.

3. Point OpenClaw’s state at that directory, e.g.:

   • Container: -v /your/smb/mapped/path:/home/node/.openclaw (adjust target path to the user your image runs as).
   • Bare metal: ensure the agent’s sessions tree resolves under that mount (typically ~/.openclaw/agents/<agentId>/sessions/).

4. Trigger a usage-cost cache refresh, e.g. open any UI/API path that loads cost or usage summaries from cache with refresh, or any flow that refreshes the on-disk usage cache for session transcripts (exact surface depends on OpenClaw version; gateway plus session usage/cost views are typical).

5. Observe logs or stderr for an error containing:

   • link
   • .usage-cost-cache.json.lock
   • ENOTSUP
     (wording may include “operation not supported on socket”.)

Example error:

Error: ENOTSUP: operation not supported on socket, link '.../.usage-cost-cache.json.lock.<pid>.<hrtime>.tmp' -> '.../.usage-cost-cache.json.lock': code=ENOTSUP

Expected behavior

No dependency on filesystem specifics

Actual behavior

Crash

OpenClaw version

v2026.5.10-beta.5

Operating system

Ubuntu 24.04

Install method

custom

Model

N/A

Provider / routing chain

N/A

Additional provider/model setup details

No response

Logs, screenshots, and evidence

Error: ENOTSUP: operation not supported on socket, link '/home/node/.openclaw/agents/main/sessions/.usage-cost-cache.json.lock.167.222258340581245.tmp' -> '/home/node/.openclaw/agents/main/sessions/.usage-cost-cache.json.lock': code=ENOTSUP

Impact and severity

No response

Additional information

No response

[clawsweeper]

Codex review: keeping this open for maintainer follow-up; there is still a little grit to resolve.

Summary
Keep open: current main still publishes the usage-cost cache lock with a hard link and rethrows unsupported-link errors; the active closing fix is still an open PR, so this issue should remain the tracker until that work merges or is replaced.

Reproducibility: yes. source-level: current main calls fs.promises.link during usage-cost refresh and only handles EEXIST, so ENOTSUP from an unsupported-hardlink mount escapes the refresh path. I did not mount a live SMB/NFS/virtiofs filesystem in this read-only review.

Next step
No repair lane: #81388 already references this issue and implements the same fallback, so maintainers should review, land, or replace that PR instead of opening another automated fix.

Review details

Best possible solution:

Land a narrow exclusive-create fallback in src/infra/session-cost-usage.ts with regression coverage and real affected-filesystem proof, then let the merged PR close this issue.

Do we have a high-confidence way to reproduce the issue?

Yes, source-level: current main calls fs.promises.link during usage-cost refresh and only handles EEXIST, so ENOTSUP from an unsupported-hardlink mount escapes the refresh path. I did not mount a live SMB/NFS/virtiofs filesystem in this read-only review.

Is this the best way to solve the issue?

Yes: preserving the existing lock semantics while falling back to exclusive create for known unsupported-hardlink errors is the narrow maintainable fix. The existing open PR is the right implementation path to review before closing this issue.

What I checked:

• Current lock publication still uses hard links: writeUsageCostCacheLockAtomically writes a temp lock payload, then publishes it with fs.promises.link(tempPath, lockPath). (src/infra/session-cost-usage.ts:236, f22c26a6cdcd)
• Unsupported link errors still propagate: acquireUsageCostCacheRefreshLock only handles EEXIST; any other error code from lock publication, including ENOTSUP, is thrown. (src/infra/session-cost-usage.ts:301, f22c26a6cdcd)
• Refresh path reaches the lock: refreshCostUsageCache acquires the usage-cost cache refresh lock before scanning transcript usage files. (src/infra/session-cost-usage.ts:1097, f22c26a6cdcd)
• Gateway usage path can trigger sync refresh: The gateway usage cache path calls loadCostUsageSummaryFromCache with requestRefresh: true and refreshMode: "sync-when-empty", matching a user-visible refresh path into the lock. (src/gateway/server-methods/usage.ts:742, f22c26a6cdcd)
• No fallback or regression coverage exists on current main: A focused source/test search found the hard-link call but no ENOTSUP, EOPNOTSUPP, EXDEV, or hard-link fallback handling in the session-cost usage implementation or tests. (src/infra/session-cost-usage.test.ts, f22c26a6cdcd)
• Open closing PR is the active implementation candidate: GitHub reports fix(session-cost-usage): fall back to O_EXCL write when fs.link is unsupported (#81089) #81388 as an open, mergeable closing PR for this issue; the similar fallback PRs are closed and unmerged. (2d5cb9055495)

Likely related people:

• @Marvinthebored: GitHub commit history shows the durable transcript aggregate cache commit introduced the usage-cost cache lock helper, hard-link publish path, and gateway cache routing involved in this issue. (role: lock helper introducer; confidence: high; commits: a64b30705fab; files: src/infra/session-cost-usage.ts, src/infra/session-cost-usage.test.ts, src/gateway/server-methods/usage.ts)
• @steipete: Recent path history shows repeated session-cost usage/cache maintenance, fs-safe extraction, and type extraction work in the same implementation and tests. (role: recent area contributor; confidence: high; commits: 0e17e55be902, c256503ea129, 2daf3d332ff0; files: src/infra/session-cost-usage.ts, src/infra/session-cost-usage.test.ts, src/gateway/server-methods/usage.ts)
• @Takhoffman: The token usage dashboard commit touched the same session-cost usage implementation, tests, and gateway usage surface that route into this lock path. (role: adjacent usage dashboard contributor; confidence: medium; commits: 8a352c8f9dfd; files: src/infra/session-cost-usage.ts, src/infra/session-cost-usage.test.ts, src/gateway/server-methods/usage.ts)

Remaining risk / open question:

• No live SMB/NFS/virtiofs filesystem was mounted during this read-only review, so reproduction is source-level rather than runtime on the affected filesystem.
• The active fix PR still has a real-environment proof question in discussion, so closing should wait for a merged fix or an explicit maintainer replacement path.

Codex review notes: model gpt-5.5, reasoning high; reviewed against f22c26a6cdcd.