Skip to content

Discord second message fails: false stale-snapshot from skillsSnapshot hydration mismatch #96698

Closed

Description

@wiidz

Summary

On Discord (and likely any channel using reply session initialization), the first inbound message succeeds but the second message always fails with:

reply session initialization conflicted for agent:main:discord:channel:<channelId>

Root cause is a false positive stale-snapshot in optimistic concurrency: createReplySessionInitializationRevision() serializes the full session entry via JSON.stringify, but snapshot load and commit compare different shapes of skillsSnapshot.

Environment

  • OpenClaw 2026.6.10 (commit a96418c)
  • Channel: Discord
  • Single agent (main), minimal config
  • Repro: send two messages in the same Discord channel; first OK, second fails

Root cause

loadReplySessionInitializationSnapshot() loads the store with default hydration, which persists skillsSnapshot as { skills: [...], promptRef: "..." }.

Inside commitReplySessionInitialization(), the writer's in-memory store is loaded with hydrated skill prompt blobs, so the same field becomes { skills: [...], prompt: "<full text>" }.

Revision comparison then fails even when updatedAt / lastInteractionAt are identical. Diagnostic logs showed:

  • fieldDiff.changedKeys: ['skillsSnapshot']
  • firstDiffIndex around 1036
  • JSON diff at that index: "skills" (expected) vs "prompt" (actual)

After the first agent run writes a full skillsSnapshot, the second inbound turn's init snapshot no longer matches the commit-time revision built from the hydrated writer cache → retry once → hard conflict.

Suggested fix

Use the same persisted shape for revision on both paths:

  1. loadReplySessionInitializationSnapshot: loadSessionStore(..., { hydrateSkillPromptRefs: false })
  2. commitReplySessionInitialization: when building the revision for comparison, re-read from disk with hydrateSkillPromptRefs: false instead of using the writer cache entry (which may be hydrated).

Alternative: change revision to exclude or normalize skillsSnapshot.prompt vs promptRef.

Secondary issue (optional hardening)

In src/channels/session.ts, recordInboundSession used void metaTask before updateLastRoute, allowing concurrent writes to the same session store. Changing to await metaTask avoids an additional race (not the primary cause of the every-second-message failure).

Verified locally

With the two changes above, consecutive Discord messages succeed (no stale-snapshot, model fetch 200 on second turn).

Activity

  1. clawsweeper commented on Jun 25, 2026

    @clawsweeper
    Contributor

    Codex review: keeping this open for maintainer follow-up; there is still a little grit to resolve. Reviewed June 25, 2026, 3:54 AM ET / 07:54 UTC.

    Summary
    Keep open: current main still has a source-level path where reply-session initialization revisions compare whole JSON-stringified session entries while skill prompt blob hydration can change the in-memory skillsSnapshot shape/order. The open Telegram PR overlaps the generic stale-snapshot symptom but does not address this reported hydration mismatch, so this issue remains actionable.

    Reproducibility: yes. at source level: the current revision guard JSON-stringifies the full entry, while skill snapshot creation and blob hydration can preserve the same data with different skillsSnapshot shape/order. I did not run the live Discord two-message flow in this read-only review.

    Next step
    This is a narrow core session-state repair with clear source evidence, likely files, and a focused test path; no product decision or new config surface is needed.

    Review details

    Best possible solution:

    Normalize reply-session initialization revisions at the session accessor/storage boundary so persisted and hydrated skillsSnapshot entries compare by the same durable shape, with a focused regression test for prompt blob hydration.

    Do we have a high-confidence way to reproduce the issue?

    Yes at source level: the current revision guard JSON-stringifies the full entry, while skill snapshot creation and blob hydration can preserve the same data with different skillsSnapshot shape/order. I did not run the live Discord two-message flow in this read-only review.

    Is this the best way to solve the issue?

    Yes, the reporter's suggested persisted-shape or normalized revision comparison is the narrowest maintainable fix. The optional recordInboundSession ordering hardening should be added only with a separate focused test if it proves necessary.

    AGENTS.md: found and applied where relevant.

    Remaining risk / open question:

    • I did not run a live Discord reproduction in this read-only review; the confidence comes from the source path plus the reporter's local verification.
    • A fix should coordinate with fix(telegram): topic replies stall after session conflicts #96550 if it remains open, because both touch reply-session stale-snapshot handling even though the root causes differ.

    Codex review notes: model internal, reasoning high; reviewed against 31e941c3fc32.

    Label changes

    Label changes:

    • add P1: The report describes a currently broken Discord channel workflow where the second inbound message fails before a reply can be generated.
    • add impact:message-loss: The failed reply-session initialization prevents the user's second Discord message from reaching a successful agent reply path.
    • add impact:session-state: The root problem is stale-snapshot detection over session metadata, specifically the persisted versus hydrated skillsSnapshot state.
    • add issue-rating: 🦞 diamond lobster: Current issue advisory state selects this label.
    • add clawsweeper:source-repro: Current issue advisory state selects this label.
    • add clawsweeper:linked-pr-open: Current issue advisory state selects this label.
    • add clawsweeper:no-new-fix-pr: Current issue advisory state selects this label.
    • add clawsweeper:queueable-fix: Current issue advisory state selects this label.
    • add clawsweeper:fix-shape-clear: Current issue advisory state selects this label.

    Label justifications:

    • P1: The report describes a currently broken Discord channel workflow where the second inbound message fails before a reply can be generated.
    • impact:message-loss: The failed reply-session initialization prevents the user's second Discord message from reaching a successful agent reply path.
    • impact:session-state: The root problem is stale-snapshot detection over session metadata, specifically the persisted versus hydrated skillsSnapshot state.
    Evidence reviewed

    Acceptance criteria:

    • node scripts/run-vitest.mjs src/config/sessions/session-accessor.test.ts -t "reply session initialization"
    • node scripts/run-vitest.mjs src/config/sessions/store.skills-stripping.test.ts
    • node scripts/run-vitest.mjs src/auto-reply/reply/session.test.ts -t "initSessionState"
    • git diff --check

    What I checked:

    • Current revision guard is whole-entry and order-sensitive: createReplySessionInitializationRevision() returns JSON.stringify(entry ?? null), so equivalent session entries with different hydrated/persisted skillsSnapshot key order or shape can compare unequal. (src/config/sessions/session-accessor.ts:1158, 31e941c3fc32)
    • Current initialization compare still fails closed on revision mismatch: loadReplySessionInitializationSnapshot() reads the store with default skill-prompt hydration, and commitReplySessionInitialization() builds a second revision from the writer store before returning reason: "stale-snapshot" on mismatch. (src/config/sessions/session-accessor.ts:1695, 31e941c3fc32)
    • Skill snapshot creation and hydration can produce different JSON order: Workspace snapshots are built with prompt before skills, while prompt blob hydration strips promptRef from the persisted snapshot and appends prompt after the rest of the object. (src/skills/loading/workspace.ts:1411, 31e941c3fc32)
    • Discord inbound turns use the shared channel turn/session path: Discord calls dispatchChannelInboundReply with recordInboundSession, and the channel turn kernel records the inbound session before dispatching the reply run, so shared reply-session initialization failures can block a visible Discord reply. (extensions/discord/src/monitor/message-handler.process.ts:932, 31e941c3fc32)
    • Adjacent open PR is only a partial overlap: fix(telegram): topic replies stall after session conflicts #96550 increases stale-snapshot retries and adds Telegram spool backoff, but its diff does not normalize skillsSnapshot revision comparison in session-accessor.ts. (8e24cb1c873f)
    • Feature history points at the guarded initialization refactor and prompt blob persistence: GitHub history shows jalehman merged the guarded reply-session initialization in 95e37f8, while steipete introduced persisted skill prompt blobs in edd4c62. (src/config/sessions/session-accessor.ts:1691, 95e37f8e9517)

    Likely related people:

    • jalehman: Introduced and recently maintained the guarded reply-session initialization/session accessor boundary that now performs the stale-snapshot revision comparison. (role: recent feature-history contributor; confidence: high; commits: 95e37f8e9517, 7e2b2d2987b2, b58e6e0734f2; files: src/config/sessions/session-accessor.ts, src/auto-reply/reply/session.ts)
    • steipete: Introduced persisted skill prompt blob projection/hydration, the storage behavior whose hydrated shape participates in the false revision mismatch. (role: adjacent storage contributor; confidence: high; commits: edd4c62da1c2, d93394e29b1f, 0e3f7a82fd12; files: src/config/sessions/skill-prompt-blobs.ts, src/config/sessions/store-load.ts, src/config/sessions/store.ts)
    • vincentkoc: Recent work repaired prompt blob fast-update behavior in the same persisted skillsSnapshot storage area. (role: adjacent prompt-blob repair contributor; confidence: medium; commits: 4d13055ca564; files: src/config/sessions/skill-prompt-blobs.ts, src/config/sessions/store.ts)
    How this review workflow works
    • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
    • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
    • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
    • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
    • Maintainers can also comment @clawsweeper review to request a fresh review only.
    • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
    • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
    • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.
  2. added
    P1High-priority user-facing bug, regression, or broken workflow.
    impact:message-lossChannel message delivery can be lost, duplicated, or misrouted.
    impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.
    no-staleExclude from stale automation
    issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.
    clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.
    clawsweeper:linked-pr-openClawSweeper found an open linked pull request for this issue.
    clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.
    clawsweeper:queueable-fixClawSweeper marked this issue as an existing queue_fix_pr work candidate.
    clawsweeper:fix-shape-clearClawSweeper found a clear likely implementation shape for this issue.
    on Jun 25, 2026
  3. elliott-dandelion-cult commented on Jun 27, 2026

    @elliott-dandelion-cult

    Field observation from the 191a7af989 deployed proof cycle (2026-06-27): this class is still operationally relevant on Discord, and /new must be treated as a workaround, not an acceptable steady-state cure.

    Observed shape in cohort runtime:

    • A prince session (Ronan) was considered for /new because the Discord reply path had previously shown reply session initialization conflicted behavior on the shared channel session.
    • A restart was not treated as a root-cause proof; /new would only rotate/replace the affected session state.
    • Proof artifacts on Ronan were later peer-checkable from another seat, so the proof-lane blocker was not "Ronan's brain needs nuking"; it was a runtime/session-state failure mode plus local rendering/session health trouble.

    Boundary for tracking:

    • Do not normalize /new as the cure for this bug family. It destroys/rotates conversational continuity and can hide the reply-session conflict root cause.
    • If this recurs, capture the exact reply session initialization conflicted log window, session-state shape, and whether persisted-vs-hydrated entry comparison is involved.
    • openclaw/openclaw#96936 remains the likely fix/verification lane, but it needs direct behavior proof that the affected Discord/Telegram reply-init flow no longer requires session reset.

    This is a field note to keep the bug tracked instead of sweeping the unhealthy state to /new.

  4. self-assigned this
    on Jun 29, 2026
  5. joshavant commented on Jun 29, 2026

    @joshavant
    Collaborator

    Thanks @wiidz for the clear original report and root-cause notes.

    This was fixed by #97657, merged in 75db48a. The landed fix keeps reply-session runtime snapshots hydrated, but computes the optimistic revision from the canonical persisted session-entry shape. That makes persisted skillsSnapshot.promptRef, hydrated skillsSnapshot.prompt, and runtime-only resolvedSkills compare as the same durable session state, while still rejecting real stale writes.

    The fix was verified with focused session-store coverage plus live Discord before/after proof: the reverted-fix run reproduced the second-message stale-snapshot conflict, and the fixed run successfully handled consecutive Discord replies after skillsSnapshot.promptRef was present.

    Closing as fixed by #97657.

  6. locked as resolved and limited conversation to collaborators on Jul 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

P1High-priority user-facing bug, regression, or broken workflow.clawsweeper:fix-shape-clearClawSweeper found a clear likely implementation shape for this issue.clawsweeper:linked-pr-openClawSweeper found an open linked pull request for this issue.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.clawsweeper:queueable-fixClawSweeper marked this issue as an existing queue_fix_pr work candidate.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.impact:message-lossChannel message delivery can be lost, duplicated, or misrouted.impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.no-staleExclude from stale automation

Type

No type

Fields

Priority

None yet

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions