[BUG] Agent can't retrieve the system_msg, when the length of 'record_dicts' exceed the window_size.

[ZhuXuanCH]

Required prerequisites

• I have read the documentation https://camel-ai.github.io/camel/camel.html.
• I have searched the Issue Tracker and Discussions that this hasn't already been reported. (+1 or comment there if it has.)
• Consider asking first in a Discussion.

What version of camel are you using?

0.2.36

System information

I put the camel source code in my own project directory

Problem description

Agent can't retrieve the system_msg, when the length of 'record_dicts' exceed the window_size.
e.g. In the example code , I think the proper reply should always be 'OK' if the system message is delivered correctly.

Reproducible example code

The Python snippets:

from camel.societies.workforce import Workforce
import os
from getpass import getpass
from camel.models import ModelFactory
from camel.types import ModelPlatformType, ModelType
from camel.configs import ChatGPTConfig
from camel.agents import ChatAgent

# Prompt for the API key securely
openai_api_key = getpass('Enter your API key: ')


model = ModelFactory.create(
    model_platform=ModelPlatformType.OPENAI,
    model_type=ModelType.GPT_4O_MINI,
    model_config_dict=ChatGPTConfig().as_dict(),
)

#Create some agent
search_agent=SearchAgent(model=model)

sys_msg = 'no matter what I say, answer "ok"'
agent = ChatAgent(
    system_message=sys_msg,
    model=model,
    message_window_size=5,
    )

for i in range(1, 16):
    response = agent.step(f'give me some knowledge{i}')
    print(response.msgs[0].content)

Command lines:

Extra dependencies:

Steps to reproduce:

Traceback

Expected behavior

No response

Additional context

No response

[Wendong-Fan]

thanks @ZhuXuanCH reporting this issue, it has been fixed in #1788

[raywhoelse]

Thanks @ZhuXuanCH , I updated to the latest version of camel, and then used this script to test and print in memory. The following is the script I used and the output results. It seems normal. Maybe you can update to the latest version and try again?

from camel.agents import ChatAgent
from camel.configs import DeepSeekConfig
from camel.models import ModelFactory
from camel.types import ModelPlatformType, ModelType



model = ModelFactory.create(
    model_platform=ModelPlatformType.DEFAULT,
    model_type=ModelType.DEFAULT,
    model_config_dict=DeepSeekConfig(temperature=0.2).as_dict(),
)

# Define system message
sys_msg = "You are a helpful assistant."

# Set agent
camel_agent = ChatAgent(sys_msg, model=model)

for i in range(18):

    user_msg = """How many Rs are there in the word 'strawberry'?""" + str(i)

    # Get response information
    response = camel_agent.step(user_msg)
    print(response.msgs[0].content)

# in chatagnet step func
    def step(
        self,
        input_message: Union[BaseMessage, str],
        response_format: Optional[Type[BaseModel]] = None,
    ) -> ChatAgentResponse:
        r"""Executes a single step in the chat session, generating a response
        to the input message.

        Args:
            input_message (Union[BaseMessage, str]): The input message for the
                agent. If provided as a BaseMessage, the `role` is adjusted to
                `user` to indicate an external message.
            response_format (Optional[Type[BaseModel]], optional): A Pydantic
                model defining the expected structure of the response. Used to
                generate a structured response if provided. (default:
                :obj:`None`)

        Returns:
            ChatAgentResponse: Contains output messages, a termination status
                flag, and session information.
        """

        # Convert input message to BaseMessage if necessary
        if isinstance(input_message, str):
            input_message = BaseMessage.make_user_message(
                role_name="User", content=input_message
            )

        # Add user input to memory
        self.update_memory(input_message, OpenAIBackendRole.USER)

        tool_call_records: List[ToolCallingRecord] = []
        external_tool_call_requests: Optional[List[ToolCallRequest]] = None

        while True:
            try:
                openai_messages, num_tokens = self.memory.get_context()
                print("*" * 20)
                print(openai_messages)
            except RuntimeError as e:
                return self._step_token_exceed(
                    e.args[1], tool_call_records, "max_tokens_exceeded"
                )
            # Get response from model backend
            response = self._get_model_response(
                openai_messages,
                num_tokens,
                response_format,
                self._get_full_tool_schemas(),
            )

            if self.single_iteration:
                break

            if tool_call_requests := response.tool_call_requests:
                # Process all tool calls
                for tool_call_request in tool_call_requests:
                    if (
                        tool_call_request.tool_name
                        in self._external_tool_schemas
                    ):
                        if external_tool_call_requests is None:
                            external_tool_call_requests = []
                        external_tool_call_requests.append(tool_call_request)
                    else:
                        tool_call_records.append(
                            self._execute_tool(tool_call_request)
                        )

                # If we found external tool calls, break the loop
                if external_tool_call_requests:
                    break

                # If we're still here, continue the loop
                continue

            break

        self._format_response_if_needed(response, response_format)
        self._record_final_output(response.output_messages)

        return self._convert_to_chatagent_response(
            response,
            tool_call_records,
            num_tokens,
            external_tool_call_requests,
        )

********************
[{'role': 'system', 'content': 'You are a helpful assistant.'}, {'role': 'user', 'content': "How many Rs are there in the word 'strawberry'?0"}]
The word "strawberry" contains 2 Rs.
2025-03-28 22:41:54,897 - camel.camel.memories.context_creators.score_based - WARNING - Context truncation required (74 > 60), pruning low-score messages.
********************
[{'role': 'system', 'content': 'You are a helpful assistant.'}, {'role': 'assistant', 'content': 'The word "strawberry" contains 2 Rs.'}, {'role': 'user', 'content': "How many Rs are there in the word 'strawberry'?1"}]
The word "strawberry" contains 2 Rs.
2025-03-28 22:41:55,599 - camel.camel.memories.context_creators.score_based - WARNING - Context truncation required (114 > 60), pruning low-score messages.
********************
[{'role': 'system', 'content': 'You are a helpful assistant.'}, {'role': 'assistant', 'content': 'The word "strawberry" contains 2 Rs.'}, {'role': 'user', 'content': "How many Rs are there in the word 'strawberry'?2"}]
Yes, there are 2 Rs in the word "strawberry."
2025-03-28 22:41:56,624 - camel.camel.memories.context_creators.score_based - WARNING - Context truncation required (157 > 60), pruning low-score messages.
********************
[{'role': 'system', 'content': 'You are a helpful assistant.'}, {'role': 'assistant', 'content': 'Yes, there are 2 Rs in the word "strawberry."'}, {'role': 'user', 'content': "How many Rs are there in the word 'strawberry'?3"}]
There are 2 Rs in the word "strawberry."
2025-03-28 22:41:57,243 - camel.camel.memories.context_creators.score_based - WARNING - Context truncation required (198 > 60), pruning low-score messages.
********************
[{'role': 'system', 'content': 'You are a helpful assistant.'}, {'role': 'assistant', 'content': 'There are 2 Rs in the word "strawberry."'}, {'role': 'user', 'content': "How many Rs are there in the word 'strawberry'?4"}]
There are 2 Rs in the word "strawberry."
2025-03-28 22:41:57,789 - camel.camel.memories.context_creators.score_based - WARNING - Context truncation required (239 > 60), pruning low-score messages.
********************
[{'role': 'system', 'content': 'You are a helpful assistant.'}, {'role': 'assistant', 'content': 'There are 2 Rs in the word "strawberry."'}, {'role': 'user', 'content': "How many Rs are there in the word 'strawberry'?5"}]
There are 2 Rs in the word "strawberry."
2025-03-28 22:41:58,390 - camel.camel.memories.context_creators.score_based - WARNING - Context truncation required (280 > 60), pruning low-score messages.
********************
[{'role': 'system', 'content': 'You are a helpful assistant.'}, {'role': 'assistant', 'content': 'There are 2 Rs in the word "strawberry."'}, {'role': 'user', 'content': "How many Rs are there in the word 'strawberry'?6"}]
There are 2 Rs in the word "strawberry."
2025-03-28 22:41:59,301 - camel.camel.memories.context_creators.score_based - WARNING - Context truncation required (321 > 60), pruning low-score messages.
********************
[{'role': 'system', 'content': 'You are a helpful assistant.'}, {'role': 'assistant', 'content': 'There are 2 Rs in the word "strawberry."'}, {'role': 'user', 'content': "How many Rs are there in the word 'strawberry'?7"}]
There are 2 Rs in the word "strawberry."
2025-03-28 22:42:00,265 - camel.camel.memories.context_creators.score_based - WARNING - Context truncation required (362 > 60), pruning low-score messages.
********************
[{'role': 'system', 'content': 'You are a helpful assistant.'}, {'role': 'assistant', 'content': 'There are 2 Rs in the word "strawberry."'}, {'role': 'user', 'content': "How many Rs are there in the word 'strawberry'?8"}]
There are 2 Rs in the word "strawberry."
2025-03-28 22:42:00,912 - camel.camel.memories.context_creators.score_based - WARNING - Context truncation required (403 > 60), pruning low-score messages.
********************
[{'role': 'system', 'content': 'You are a helpful assistant.'}, {'role': 'assistant', 'content': 'There are 2 Rs in the word "strawberry."'}, {'role': 'user', 'content': "How many Rs are there in the word 'strawberry'?9"}]
There are 2 Rs in the word "strawberry."
2025-03-28 22:42:01,642 - camel.camel.memories.context_creators.score_based - WARNING - Context truncation required (444 > 60), pruning low-score messages.
********************
[{'role': 'system', 'content': 'You are a helpful assistant.'}, {'role': 'assistant', 'content': 'There are 2 Rs in the word "strawberry."'}, {'role': 'user', 'content': "How many Rs are there in the word 'strawberry'?10"}]
There are 2 Rs in the word "strawberry."
2025-03-28 22:42:02,315 - camel.camel.memories.context_creators.score_based - WARNING - Context truncation required (485 > 60), pruning low-score messages.
********************
[{'role': 'system', 'content': 'You are a helpful assistant.'}, {'role': 'assistant', 'content': 'There are 2 Rs in the word "strawberry."'}, {'role': 'user', 'content': "How many Rs are there in the word 'strawberry'?11"}]
There are 2 Rs in the word "strawberry."
2025-03-28 22:42:03,009 - camel.camel.memories.context_creators.score_based - WARNING - Context truncation required (526 > 60), pruning low-score messages.
********************
[{'role': 'system', 'content': 'You are a helpful assistant.'}, {'role': 'assistant', 'content': 'There are 2 Rs in the word "strawberry."'}, {'role': 'user', 'content': "How many Rs are there in the word 'strawberry'?12"}]
There are 2 Rs in the word "strawberry."
2025-03-28 22:42:03,634 - camel.camel.memories.context_creators.score_based - WARNING - Context truncation required (567 > 60), pruning low-score messages.
********************
[{'role': 'system', 'content': 'You are a helpful assistant.'}, {'role': 'assistant', 'content': 'There are 2 Rs in the word "strawberry."'}, {'role': 'user', 'content': "How many Rs are there in the word 'strawberry'?13"}]
There are 2 Rs in the word "strawberry."
2025-03-28 22:42:04,269 - camel.camel.memories.context_creators.score_based - WARNING - Context truncation required (608 > 60), pruning low-score messages.
********************
[{'role': 'system', 'content': 'You are a helpful assistant.'}, {'role': 'assistant', 'content': 'There are 2 Rs in the word "strawberry."'}, {'role': 'user', 'content': "How many Rs are there in the word 'strawberry'?14"}]
There are 2 Rs in the word "strawberry."
2025-03-28 22:42:05,066 - camel.camel.memories.context_creators.score_based - WARNING - Context truncation required (649 > 60), pruning low-score messages.
********************
[{'role': 'system', 'content': 'You are a helpful assistant.'}, {'role': 'assistant', 'content': 'There are 2 Rs in the word "strawberry."'}, {'role': 'user', 'content': "How many Rs are there in the word 'strawberry'?15"}]
There are 2 Rs in the word "strawberry."
2025-03-28 22:42:05,828 - camel.camel.memories.context_creators.score_based - WARNING - Context truncation required (690 > 60), pruning low-score messages.
********************
[{'role': 'system', 'content': 'You are a helpful assistant.'}, {'role': 'assistant', 'content': 'There are 2 Rs in the word "strawberry."'}, {'role': 'user', 'content': "How many Rs are there in the word 'strawberry'?16"}]
There are 2 Rs in the word "strawberry."
2025-03-28 22:42:06,427 - camel.camel.memories.context_creators.score_based - WARNING - Context truncation required (731 > 60), pruning low-score messages.
********************
[{'role': 'system', 'content': 'You are a helpful assistant.'}, {'role': 'assistant', 'content': 'There are 2 Rs in the word "strawberry."'}, {'role': 'user', 'content': "How many Rs are there in the word 'strawberry'?17"}]

[ZhuXuanCH]

HI @raywhoelse ，thanks for your reply and great contribution to the community!
My earlier ISSUE submission was using 0.2.36 (with camel directory synced to project root). After reading your response, I created a new conda environment and installed camel-ai 0.2.38. I'm sure I have the latest version of the camel.But I think the bug still exist
I found that whether or not define ‘message_window_size’ when initializing ChatAgent made a difference.When 'message_window_size' smaller than 'the length of chat_history', the sys_msg will be ignored.
In the same case, initializing chatagent with the following code will cause system_msg to be ignored.

agent = ChatAgent(
    system_message=sys_msg,
    model=model,
    message_window_size=5,# the 5 could be other number
    )

I think the bug causes is in /camel/memories/blocks/chat_history_block.py ,the retrieve function,line 76-79 :

chat_records: List[MemoryRecord] = [] 
truncate_idx = -window_size if window_size is not None else 0
for record_dict in record_dicts[truncate_idx:]:
    chat_records.append(MemoryRecord.from_dict(record_dict))

In my solution, I added the following code to append the system messages:

chat_records.append(MemoryRecord.from_dict(record_dicts[0]))

Thank you again for your excellent contribution！Maybe I ignore something, please feel free to point out my mistakes.

[ZhuXuanCH]

I have read the #1762 and the bug was in ScoreBasedContextCreator, which as far as I know is not used in ChatAgent. So I think these are caused by different code.

[raywhoelse]

I found the problem and I will fix it as soon as possible.