How to limit the conversation history in Haystack Agents?

[SuperMohit]

After enabling the tracer, I came to know that Agent is sending full converastional history along with Tool call results to subsequent calls, how to truncate the history or do summarization?

[sjrl]

Hey @SuperMohit thanks for the question! We are planning work on enabling Agent memory in this epic. No work has been started yet but you can follow it there.

As for doing something about it now, it's possible to customize the behavior of how messages are stored inside of State within Agent. You can find more info on State here. But basically you will want to specify a custom state_schema which then you will pass to Agent. So something like this

import os

os.environ["SERPERDEV_API_KEY"] = "YOUR_API_KEY"  # Replace with your actual API key
os.environ["OPENAI_API_KEY"] = "YOUR_API_KEY"  # Replace with your actual API key

from haystack.components.agents import Agent
from haystack.components.agents.state import merge_lists
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.generators.utils import print_streaming_chunk
from haystack.components.websearch import SerperDevWebSearch
from haystack.dataclasses import ChatMessage
from haystack.tools.component_tool import ComponentTool
from typing import Optional, List

# Create a web search tool using SerperDevWebSearch
web_tool = ComponentTool(component=SerperDevWebSearch(), name="web_tool")


# NOTE: For convenience, this is what merge_lists looks like in haystack/components/agents/state/state_utils.py
# def merge_lists(current: Union[List[T], T, None], new: Union[List[T], T]) -> List[T]:
#     current_list = [] if current is None else current if isinstance(current, list) else [current]
#     new_list = new if isinstance(new, list) else [new]
#     return current_list + new_list


def my_custom_message_handler(
    messages_currently_in_state: Optional[List[ChatMessage]], new_messages: List[ChatMessage]
) -> List[ChatMessage]:
    """
    Custom message handler that merges new messages with the existing ones in the state,
    while keeping a history of the last 3 messages.
    """
    history_length = 10
    messages_currently_in_state = messages_currently_in_state or []
    return messages_currently_in_state[-history_length:] + new_messages


# Create the agent with the web search tool
agent = Agent(
    chat_generator=OpenAIChatGenerator(model="gpt-4o-mini"),
    tools=[web_tool],
    # state_schema={"messages": {"type": List[ChatMessage], "handler": merge_lists}},  # NOTE: This is the default state schema
    state_schema={"messages": {"type": List[ChatMessage], "handler": my_custom_message_handler}},
    streaming_callback=print_streaming_chunk,
)

# Run the agent with a query
result = agent.run(
    messages=[ChatMessage.from_user("Find information about Haystack AI framework and the deepset cofounders.")]
)

# Print the final response
print(result["messages"])

[SuperMohit]

Thanks, this helps for now.

[Silence-Well]

Hi~ @sjrl
I would like to ask how the Multi-turn conversations of the current Agent can be implemented, and how the chat history should be managed. Currently, I can only manually assemble messages to record the chat history in order to achieve Multi-turn conversations.