Skip to content

feat: use git-backed workspace diffs for memory consolidation#18982

Merged
jif-oai merged 15 commits intomainfrom
jif/memories-on-file-system
Apr 27, 2026
Merged

feat: use git-backed workspace diffs for memory consolidation#18982
jif-oai merged 15 commits intomainfrom
jif/memories-on-file-system

Conversation

@jif-oai
Copy link
Copy Markdown
Collaborator

@jif-oai jif-oai commented Apr 22, 2026

Why

This PR make the morpheus agent (memory phase 2) use a git diff to start it's consolidation. The workflow is the following:

  1. The agent acquire a lock
  2. If .codex/memories does not exist or is not a git root, initialize everything (and make a first empty commit)
  3. Update raw_memories.md and rollout_summaries/ as before. Basically we select max N phase 1 memories based on a given policy
  4. We use git (gix) to get a diff between the current state of .codex/memories and the last commit.
  5. Dump the diff in phase2_workspace_diff.md
  6. Spawn morpheus and point it to phase2_workspace_diff.md
  7. Wait for morpheus to be done
  8. Re-create a new .git and make one single commit on it. We do this because we don't want to preserve history through .git and this is cheap anyway
  9. We release the lock
    On top of this, we keep the retry policies etc etc

The goals of this new workflow are:

  • Better support of any memory extensions such as chronicle
  • Allow the user to manually edit memories and this will be considered by the phase 2 agent

As a follow-up we will need to add support for user's edition while morpheus is running

What Changed

  • Added memory workspace helpers that prepare the git baseline, compute the diff, write phase2_workspace_diff.md, and reset the baseline after successful consolidation.
  • Updated Phase 2 to sync current inputs into raw_memories.md and rollout_summaries/, prune old extension resources, skip clean workspaces, and run the consolidation subagent only when the workspace has changes.
  • Tightened Phase 2 job ownership around long-running consolidation with heartbeats and an ownership check before resetting the baseline.
  • Simplified the prompt and state APIs so DB watermarks are bookkeeping, while workspace dirtiness decides whether consolidation work exists.
  • Updated the memory pipeline README and tests for workspace diffs, extension-resource cleanup, pollution-driven forgetting, selection ranking, and baseline persistence.

Verification

  • Added/updated coverage in core/src/memories/tests.rs, core/src/memories/workspace_tests.rs, state/src/runtime/memories.rs, and core/tests/suite/memories.rs.

@jif-oai jif-oai requested a review from a team as a code owner April 22, 2026 13:03
@jif-oai jif-oai changed the title feat: git-based memory system feat: use git-backed workspace diffs for memory consolidation Apr 22, 2026
Copy link
Copy Markdown
Contributor

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bedf34af22

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread codex-rs/core/src/memories/mod.rs
Comment thread codex-rs/core/src/memories/phase2.rs
jif-oai and others added 4 commits April 22, 2026 14:25
@jif-oai
Copy link
Copy Markdown
Collaborator Author

jif-oai commented Apr 24, 2026

@codex review

Copy link
Copy Markdown
Contributor

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bd15970b5a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread codex-rs/core/src/memories/mod.rs
Comment thread codex-rs/core/src/memories/workspace.rs Outdated
Comment thread codex-rs/core/src/memories/workspace.rs
Comment thread codex-rs/core/src/memories/workspace.rs
@jif-oai
Copy link
Copy Markdown
Collaborator Author

jif-oai commented Apr 24, 2026

@codex review

Copy link
Copy Markdown
Contributor

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3f80399650

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread codex-rs/core/src/memories/mod.rs
Comment thread codex-rs/state/src/runtime/memories.rs
Comment thread codex-rs/core/src/memories/phase2.rs
Comment thread codex-rs/core/src/memories/workspace.rs
@jif-oai jif-oai merged commit 01ab25d into main Apr 27, 2026
24 of 25 checks passed
@jif-oai jif-oai deleted the jif/memories-on-file-system branch April 27, 2026 12:32
@github-actions github-actions Bot locked and limited conversation to collaborators Apr 27, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants