Simple Serialization and Deserialization of Memory classes to and from dictionaries w/ System Messages included

[lameTookan]

Feature request

I want simple methods to use on the various memory classes to enable easy serialization and de serialization of the current state of the convo, crucially including system messages as json dictionaries.

If this is currently already included, I cannot for the life of me find it anywhere in the documentation or the source code.

In particular there doesn't seem to be a way to add system messages directly to the memory.

Heres some pseudo code outlining what this would look like

serialized_memory = memory_instance.serialize_to_json()
new_memory = MemoryClass()
new_memory.load_from_json(serialized_memory)

In the new memory, the entire contents of the history(at least what has not been pruned) would be included. Including any system messages, which with the summary variants would include the convo summaries.

Motivation

The absence of easy serialization and deserialization methods makes it difficult to save and load conversations in JSON files or other formats. Working with dictionaries is straightforward, and they can be easily converted into various data storage formats.

Your contribution

Possibly. I just started messing around with this module today, and I am relatively new to Python. I will be doing my best to learn more about how this library works so I can add this feature.

With some more research and experimentation this could be something I could do.

[dosubot]

🤖

Thank you for your interest in contributing to LangChain! Your proposed feature of adding simple serialization and deserialization methods to the memory classes sounds like a valuable addition to the framework.

It's great to hear that you're willing to learn more about the library and work on implementing this feature. If you have any questions or need any guidance during the process, feel free to ask. We're here to support you.

Once you have a proposed solution or implementation, please create a pull request on the LangChain repository. The maintainers will review your contribution and provide feedback. We appreciate your willingness to contribute and look forward to seeing your work!

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[jeffheaton]

Agree, this seems like a very useful feature, ideally using the same interface across all memory types.

[D3nam]

I would like to contribute to this issue, and get more experience working on this project!

[Aditya-k-23]

Hey @baskaryan,

We are working on this issue and need clarification before we provide a complete resolution.
Under the BaseChatMessageHistory class, we see methods add_user_message and add_ai_message, which were added as a part of PR #9725 last month, we were wondering why a method like add_system_message was not added for system messages.

To make system messages a part of the chat message history as suggested by the user above, we were wondering if we should include such a method in our solution. Could you provide your thoughts on this?

[Aditya-k-23]

@jeffheaton @lameTookan
Can you please clarify which specific classes you are referring to when you say a MemoryClass?

[lameTookan]

@Aditya-k-23 Basically all of the classes described in this section of the documentation
https://python.langchain.com/docs/modules/memory/

[avaove]

@lameTookan Hello again, couple questions:

1. What is the use case for having system messages / other agent kwargs? (The reason we ask this, is because if the use case is solely so that we can see it in the serialized memory object, we can just append it during serialization instead of actually appending to the memory object itself).
2. Just wanted to confirm the need for serialization/deserialization for these two memory types: "Knowledge Graph" and "Vector Store" (the other classes are in memory and easier to implement)

[dosubot]

Hi, @lameTookan,

I'm helping the LangChain team manage their backlog and am marking this issue as stale.

It looks like you requested the addition of simple serialization and deserialization methods for memory classes, specifically for system messages as JSON dictionaries. You were considering contributing to the module after learning more about how it works. Other contributors have expressed interest in working on this issue and have raised questions about specific classes and use cases for system messages. The maintainers have provided guidance and support for potential contributions.

Could you please let us know if this issue is still relevant to the latest version of the LangChain repository? If it is, please feel free to let the LangChain team know by commenting on the issue. Otherwise, you are welcome to close the issue yourself, or it will be automatically closed in 7 days.

Thank you!