Skip to content

fix(memory): store only what a person sent as a conversation memory - #5313

Open
yh928 wants to merge 3 commits into
tinyhumansai:mainfrom
yh928:fix/autosave-user-turns-only
Open

fix(memory): store only what a person sent as a conversation memory#5313
yh928 wants to merge 3 commits into
tinyhumansai:mainfrom
yh928:fix/autosave-user-turns-only

Conversation

@yh928

@yh928 yh928 commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Summary

  • An internal agent no longer stores its own prompt as the user's conversation memory.
  • AgentTurnOrigin::is_user_authored names the distinction the codebase already had: WebChat / ExternalChannel carry what a person sent, every other origin carries host text.
  • Gated at the session, so a new internal agent cannot forget to opt out.
  • Existing rows are left alone — this is the write path only.

Problem

Session's autosave persists the turn's "user message" as a MemoryCategory::Conversation document keyed user_msg:<uuid>, gated only on config.memory.auto_save. An internal agent built with Agent::from_config_for_agent(...) inherits that flag — and its "user message" is the prompt the host wrote for it.

Found live, in the global namespace of a workspace with two namespaces:

category: "conversation"   key: "user_msg:f7cf6a07-…"
content:  "Maintain the existing goals list. Call goals_list first, then make the
           MINIMAL set of changes (goals_add / goals_edit / goals_delete) justified
           by the context below. Do not churn goals that are still valid.
           …
           ## Context
           Recent conversation recap (segment seg-18c746e3…): The user asked twice to
           search Gmail for emails from Colorado…"

That is memory_goals::enrich's prompt, verbatim, stored as if the user had typed it. It then competes for slots in every later recall, which is what made a namespace-wide memory search read like a transcript dump.

The blast radius is every config-built internal agent, not just goals: the flag comes from config, so a caller has to remember to turn it off, and forgetting is invisible until the store is inspected.

Solution

Gate the autosave on the turn's origin. memory_goals::enrich already runs under AgentTurnOrigin::TrustedAutomation { source: Subconscious }; a live chat turn runs under WebChat (web chat / TUI) or ExternalChannel (Telegram, Discord, …).

is_user_authored is an allowlist, for the same reason the permission gate uses one: a new origin is a turn nobody has classified yet, and mistaking host text for a user message writes it where the user's own words belong, indistinguishable afterwards. Cli is excluded on the strength of its own doc comment — "command-line / sub-agent / one-off internal invocation" — and an unscoped Unknown is excluded because turn_origin already documents that every entry point must scope a real origin.

Tradeoff: a caller that relays a person's text without scoping an origin stops autosaving. That is the same contract the permission gate enforces, and both production user paths scope one today (web_chat/ops.rsWebChat, channels/runtime/dispatch/processor.rsExternalChannel).

Submission Checklist

  • Tests added or updated (happy path + at least one failure / edge case) — an_automation_turn_does_not_store_its_prompt_as_the_users_memory, an_unscoped_turn_stores_no_user_message, and the existing round-trip test now scopes WebChat as production does
  • Diff coverage ≥ 80% — the changed lines are the origin predicate and the autosave gate, both covered in each direction by the tests above
  • Coverage matrix updated — N/A: behaviour-only change
  • All affected feature IDs from the matrix are listed under ## RelatedN/A: behaviour-only change
  • No new external network dependencies introduced
  • Manual smoke checklist updated if this touches release-cut surfaces — N/A: no release-cut surface
  • Linked issue closed via Closes #NNN in the ## Related section

Impact

Desktop / CLI / channels: unchanged for user turns. Internal automation turns (cron, subconscious, goal continuation, workflow) stop writing user_msg:* conversation documents, so the memory store keeps only what a person sent and recall stops competing with prompt boilerplate. No migration: rows written before this change are left in place deliberately.

Related

AI Authored PR Metadata (required for Codex/Linear PRs)

Linear Issue

  • Key: N/A
  • URL: N/A

Commit & Branch

  • Branch: fix/autosave-user-turns-only
  • Commit SHA: 6d46baecaad7ba82cd09cb40efb40321d6906657

Validation Run

  • pnpm --filter openhuman-app format:check — N/A: no frontend files changed
  • pnpm typecheck — N/A: no frontend files changed
  • Focused tests: agent::tests 100, turn_origin 3, agent::harness::session 205, memory_goals 7 — all green
  • Rust fmt/check (if changed): cargo fmt --all, cargo clippy -p openhuman -- -D warnings clean
  • Tauri fmt/check (if changed): N/A: no Tauri files changed

Summary by CodeRabbit

  • New Features

    • Added support for identifying direct chat messages as user-authored.
    • Direct chat messages are now preserved in conversation history.
    • Direct chat actions continue without unnecessary approval prompts.
  • Bug Fixes

    • Autosave now records only user-authored messages when enabled.
    • Internal agent prompts are no longer incorrectly saved as user memories.
    • Improved handling of asynchronous message persistence across chat channels.

@yh928
yh928 requested a review from a team August 1, 2026 00:40
@coderabbitai

coderabbitai Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 6aca5149-19ca-4ade-9f8e-c5800139862f

📥 Commits

Reviewing files that changed from the base of the PR and between 98aebed and f1d9e62.

📒 Files selected for processing (2)
  • src/openhuman/agent/tests.rs
  • src/openhuman/security/approval/gate.rs
🚧 Files skipped from review as they are similar to previous changes (2)
  • src/openhuman/agent/tests.rs
  • src/openhuman/security/approval/gate.rs

📝 Walkthrough

Walkthrough

The change gates conversation autosave on user-authored turn origins. It adds DirectChat, updates direct RPC and approval handling, and tests persistence for user origins plus rejection for automation and unscoped turns.

Changes

User-authored autosave

Layer / File(s) Summary
User-authored origin classification
src/openhuman/agent/turn_origin.rs
Adds DirectChat, is_user_authored, and current_is_user_authored. User-authored origins are WebChat, ExternalChannel, and DirectChat.
Direct-chat origin integration
src/openhuman/inference/local/ops.rs, src/openhuman/security/approval/gate.rs
Direct RPC execution uses DirectChat. The approval gate allows DirectChat without prompting, like Cli.
Autosave guard and regression coverage
src/openhuman/agent/harness/session/turn/core.rs, src/openhuman/agent/tests.rs
Autosave now requires a user-authored origin. Tests cover asynchronous persistence for user turns and non-persistence for trusted automation and unscoped turns.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant DirectRPC
  participant AgentTurnOrigin
  participant SessionAutosave
  participant ConversationMemory
  DirectRPC->>AgentTurnOrigin: execute turn as DirectChat
  AgentTurnOrigin->>SessionAutosave: classify current origin
  SessionAutosave->>ConversationMemory: save user_msg for user-authored turns
Loading

Possibly related PRs

Suggested labels: rust-core, agent, bug, memory

Poem

A rabbit checks each origin bright,
User words hop to memory right.
Trusted prompts remain outside,
Direct chats save with trusted stride.
Unscoped turns fail closed today—
Hare keeps stray prompts away.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name Status Explanation Resolution
Linked Issues check ❓ Inconclusive The session-level origin gate excludes automation and unscoped turns, preserves user turns, leaves existing rows unchanged, and adds regression tests; coverage is not verified. Provide CI evidence that changed-line coverage meets the required 80% threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the main change: storing only person-authored content as conversation memory.
Out of Scope Changes check ✅ Passed The DirectChat origin and approval-gate changes support user-turn autosave and preserve existing direct-chat trust behavior.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot added agent Built-in agents, prompts, orchestration, and agent runtime in src/openhuman/agent/. bug rust-core Core Rust runtime in src/: CLI, core_server, shared infrastructure. labels Aug 1, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/openhuman/agent/tests.rs`:
- Around line 738-744: Extend the negative autosave test wait in
src/openhuman/agent/tests.rs lines 738-744 to use the positive test’s
50-iteration, one-second polling window before checking mem.list; apply the same
wait change at lines 764-767. Update both sites consistently, or replace both
waits with a deterministic store-call signal.
- Around line 658-671: Extend the autosave regression coverage in the existing
test around the WebChat-scoped turn by adding an
AgentTurnOrigin::ExternalChannel turn through turn_origin::with_origin. Verify
the resulting persisted documents include a user_msg: entry, matching the
existing positive WebChat persistence assertion.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: fa187edd-e21a-4d0c-b775-f1071c794c07

📥 Commits

Reviewing files that changed from the base of the PR and between 43cc1b4 and 6d46bae.

📒 Files selected for processing (3)
  • src/openhuman/agent/harness/session/turn/core.rs
  • src/openhuman/agent/tests.rs
  • src/openhuman/agent/turn_origin.rs

Comment thread src/openhuman/agent/tests.rs
Comment thread src/openhuman/agent/tests.rs Outdated
@greptile-apps

greptile-apps Bot commented Aug 1, 2026

Copy link
Copy Markdown

Greptile Summary

This PR fixes a bug where internal agent prompts (e.g., memory_goals::enrich) were being stored as MemoryCategory::Conversation documents keyed user_msg:<uuid>, indistinguishable from actual user messages, because config-built internal agents inherit the auto_save flag.

  • Adds AgentTurnOrigin::DirectChat to decouple the "trusted caller" axis (approval gate) from the "person authored this text" axis (autosave), and migrates agent_chat RPC from Cli to DirectChat so direct-chat messages are saved.
  • Adds is_user_authored() to AgentTurnOrigin as an allowlist (WebChat | ExternalChannel | DirectChat), and gates autosave in turn/core.rs on both auto_save and current_is_user_authored().
  • New tests cover: external-channel turns store, direct-chat turns store, automation turns do not store, and unscoped turns do not store.

Confidence Score: 5/5

Safe to merge. The change only restricts the autosave write path — no reads, no data deletion, no migration — and the two production user-facing paths (WebChat, ExternalChannel) are correctly included in the allowlist.

The fix is a single boolean guard at the one chokepoint where all autosaves originate, backed by an allowlist that fails closed on unrecognised origins. The new DirectChat variant is correctly plumbed through both the autosave predicate and the approval gate without changing the gate's trust decision. Tests cover all four branches with a symmetric polling window that would catch a broken guard even if it fires asynchronously.

Files Needing Attention: No files require special attention.

Important Files Changed

Filename Overview
src/openhuman/agent/turn_origin.rs Adds DirectChat variant, is_user_authored() allowlist method, and current_is_user_authored() free function. Design is sound — allowlist ensures new origins fail closed.
src/openhuman/agent/harness/session/turn/core.rs Autosave gate tightened to require both auto_save and current_is_user_authored(); fix is a single-line addition at the right chokepoint.
src/openhuman/inference/local/ops.rs agent_chat RPC switched from Cli to DirectChat; approval-gate behavior is unchanged, but now correctly marks the turn as user-authored so direct-chat messages are saved.
src/openhuman/security/approval/gate.rs DirectChat added to the Cli match arm with no change in trust policy; origin.class() is now logged so the distinction is visible in traces.
src/openhuman/agent/tests.rs Four new tests covering positive (ExternalChannel, DirectChat) and negative (TrustedAutomation, unscoped) autosave cases with a symmetric polling window.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["agent.turn(user_message)"] --> B{"self.auto_save?"}
    B -- No --> C[Skip autosave]
    B -- Yes --> D{"current_is_user_authored()"}
    D -- false --> C
    D -- true --> E[tokio::spawn autosave]
    E --> F["memory.store user_msg:uuid"]
Loading

Reviews (3): Last reviewed commit: "fix(agent): keep autosave for the direct..." | Re-trigger Greptile

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6d46baecaa

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/openhuman/agent/turn_origin.rs Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/openhuman/agent/tests.rs`:
- Around line 704-710: Update the key collection flow around Memory::list in the
affected test helper to propagate listing failures instead of converting them
into an empty collection. Replace unwrap_or_default with unwrap, or change the
helper to return Result and propagate the error, while preserving the existing
key-mapping behavior on success.

In `@src/openhuman/approval/gate.rs`:
- Around line 819-830: Add a regression case to the existing approval tests that
invokes the gate through with_origin using AgentTurnOrigin::DirectChat and
asserts GateOutcome::Allow without parking. Keep the test setup and assertions
aligned with the existing Cli coverage to verify both origins follow the same
approval path.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 03baa7f7-138f-4607-a10a-8a288cfbf846

📥 Commits

Reviewing files that changed from the base of the PR and between 6d46bae and 6170a77.

📒 Files selected for processing (4)
  • src/openhuman/agent/tests.rs
  • src/openhuman/agent/turn_origin.rs
  • src/openhuman/approval/gate.rs
  • src/openhuman/inference/local/ops.rs

Comment thread src/openhuman/agent/tests.rs
Comment thread src/openhuman/security/approval/gate.rs
yh928 and others added 2 commits August 5, 2026 10:51
`auto_save` says the workspace keeps its chat in memory; it did not say whether
a turn is chat at all. An internal agent is built from the same config
(`Agent::from_config_for_agent`), so it inherits the flag — and its "user
message" is the prompt the host wrote for it.

Live, that put `memory_goals::enrich`'s prompt in the `global` namespace as a
`Conversation` document keyed `user_msg:<uuid>`:

    "Maintain the existing goals list. Call goals_list first, then make the
     MINIMAL set of changes (goals_add / goals_edit / goals_delete)…
     ## Context
     Recent conversation recap (segment seg-18c746e3…)…"

Prompt boilerplate then competes for slots in every later recall, which is what
made a namespace-wide search read like a transcript dump.

The distinction already existed: `AgentTurnOrigin`. `WebChat` and
`ExternalChannel` carry what a person sent; `TrustedAutomation` (cron,
subconscious, goal continuation, workflow), `Cli` — documented as "command-line
/ sub-agent / one-off internal" — and an unscoped `Unknown` carry host text.
`AgentTurnOrigin::is_user_authored` is an allowlist for the same reason the
permission gate uses one: a new origin is a turn nobody has classified, and
mistaking host text for a user message writes it where the user's own words
belong, indistinguishable afterwards.

Gated at the session, not at each caller, so a new internal agent cannot forget
to opt out — the kind of omission nothing surfaces until the store is inspected.

Existing rows are left alone; this is the write path only.

Tests: an automation turn and an unscoped turn store no `user_msg:*` document;
the existing round-trip test now scopes `WebChat`, as every production entry
point does. agent::tests 100, turn_origin 3, agent::harness::session 205,
memory_goals 7. clippy clean.

Closes tinyhumansai#5312

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SRSNnqQsokuGmkbpLoLCGy
… allowlist

Review follow-ups on the user-authored autosave gate.

`agent_chat` scoped `AgentTurnOrigin::Cli`, and its comment says why: to tell
the approval gate "trusted caller, do not fail closed". That variant also
covers sub-agent and internal invocations, so reusing it to answer "did a
person write this" dropped a real user message — the desktop Settings
agent-chat panel calls this RPC, and with `memory.auto_save` on its messages
stopped being stored.

The two questions need two variants. `DirectChat` is user-authored and shares
the gate's `Cli` arm, so the trust decision is unchanged and cannot drift.

Tests:
- `ExternalChannel` gets its own positive case; the allowlist had three
  members and only `WebChat` was covered by a stored-message assertion.
- `DirectChat` gets one, so the regression above stays fixed.
- Both negative tests now poll the same one-second window as the positive
  ones via `poll_for_stored_user_message`. The store is fire-and-forget, so a
  fixed 200 ms sleep would let a broken guard pass while failing live.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SRSNnqQsokuGmkbpLoLCGy
@yh928
yh928 force-pushed the fix/autosave-user-turns-only branch from 6170a77 to 98aebed Compare August 5, 2026 01:55
@coderabbitai

coderabbitai Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

@coderabbitai coderabbitai Bot added the memory Memory store, memory tree, recall, summarization, and embeddings in src/openhuman/memory/. label Aug 5, 2026
`poll_for_stored_user_message` turned a failed `Memory::list` into an empty key
list, so the tests that assert on which keys the autosave wrote could conclude
"no `user_msg:` key was stored" from a storage error rather than from the
behaviour under test — the automation and unscoped cases would pass without
reading storage at all. It now expects, and says why in the message.

Also adds the `DirectChat` approval case. `Cli` and `DirectChat` share one arm
because this gate decides on trust, not on whether the turn's text was
person-written — and an arm covered by only one of its origins is an arm that
can be split without anything failing. The test asserts `Allow` and that nothing
was parked, since allowing without a prompt means exactly that.

approval::gate 50, agent::tests 107 pass.

Reported by CodeRabbit on tinyhumansai#5313.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SRSNnqQsokuGmkbpLoLCGy

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yh928 has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent Built-in agents, prompts, orchestration, and agent runtime in src/openhuman/agent/. bug memory Memory store, memory tree, recall, summarization, and embeddings in src/openhuman/memory/. rust-core Core Rust runtime in src/: CLI, core_server, shared infrastructure.

Projects

Status: Todo

Development

Successfully merging this pull request may close these issues.

Internal agents store their own prompts as the user's conversation memories

1 participant