Skip to content

Session system_prompt persists as null after a turn → every subsequent turn rebuilds, prefix cache misses permanently #63713

Description

@gmoranxyz

Version

hermes-agent 0.18.0 @ 30e947e · Python gateway (systemd user unit)

Summary

On continuing gateway sessions, conversation_loop.py:346-352 repeatedly warns:

Stored system prompt for session <id> is null; rebuilding from scratch this
turn. Prefix cache will miss until the rebuild persists. Investigate the
previous turn's update_system_prompt write path.

Once a session's stored system_prompt is null/empty, it stays that way: every
turn rebuilds the prompt and the prefix cache misses on every request for the life
of the session.

Observed

Recurs across turns on the same session id (e.g. a long-lived Matrix DM session,
174+ messages), and on multiple gateways. The stored prompt is never repaired.

Suspected root cause

Likely the same un-awaited AsyncSessionDB call documented in #63712: the previous
turn's update_system_prompt write is issued on the async session-DB wrapper
without await, so it's dropped and never persists. Sessions that have had a
/model switch are prime suspects.

Impact

Continuous prefix-cache misses → higher token cost and latency on every turn of an
affected session. No incorrect output, but a persistent cost/perf regression.

Suggested fix

Ensure the update_system_prompt write path is awaited and persists; add a
read-back assertion (or repair-on-detect) so a null stored prompt is rewritten
once rather than rebuilt every turn.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P0Critical — data loss, security, crash loopcomp/agentCore agent runtime: loop, agent_init, prompt builder, context-compression, responses endpointsweeper:implemented-on-mainSweeper: behavior already present on current mainsweeper:risk-cachingSweeper risk: may break/degrade prompt caching or cache-key stability (invariant)sweeper:risk-session-stateSweeper risk: may lose/corrupt/mis-associate session or context statetype/bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions