Declaring and Open AI Functions Agent with memory and system message

[rlnasuti]

Hello!

I'm trying to instantiate an Open AI Functions Agent with both memory and a custom system message. I'm having a difficult time figuring out how to do it.

In one method, I can set the system message - like this:

    agent = OpenAIFunctionsAgent.from_llm_and_tools(
        llm=llm,
        tools=tools,
        agent_kwargs=agent_kwargs,
        system_message=OpenAIFunctionsAgent.create_prompt(SystemMessage(content="Talk like a chicken")),
        memory=memory,
        verbose=True
    )

This will give me an agent that calls tools and talks like a chicken, but has no memory.

Using another method:

    agent = initialize_agent(
        llm=llm,
        memory=memory,
        agent_kwargs=agent_kwargs,
        tools=[dummy_tool],
        agent=AgentType.OPENAI_FUNCTIONS,
        verbose=True,
    )

I can declare an agent with memory, but if I send in a system_message parameter as a kwarg, it doesn't make its way to the agent. I end up with "You are a helpful AI assistant".

How can I instantiate this agent with both memory and a custom system message?

[dosubot[bot]]

🤖

Hello,

Based on the information you provided and the context from the LangChain repository, it seems you want to instantiate an OpenAI Functions Agent with both memory and a custom system message.

To achieve this, you can include the system message in the agent_kwargs when initializing the agent. Here's how you can do it:

system_message = SystemMessage(content="You are Jelly.")

agent_kwargs = {
    "extra_prompt_messages": [MessagesPlaceholder(variable_name="memory")],
    "system_message": system_message,
}

open_ai_agent = initialize_agent(
    tools,
    llm,
    agent=AgentType.OPENAI_FUNCTIONS,
    verbose=True,
    agent_kwargs=agent_kwargs,
    memory=memory,
)

This solution was confirmed to work in LangChain version 0.0.234. However, it was noted that the system_message argument was discontinued in later versions, and the system_message should be passed as a string within agent_kwargs. You can refer to this issue for more details.

If you want to use the from_llm_and_tools method of the OpenAIMultiFunctionsAgent class, you can do it like this:

from langchain.chains import LLMMathChain
from langchain.llms import OpenAI
from langchain.utilities import SerpAPIWrapper, SQLDatabase
from langchain_experimental.sql import SQLDatabaseChain
from langchain.agents import Tool, AgentType
from langchain.agents.openai_functions_multi_agent.base import OpenAIMultiFunctionsAgent
from langchain.chat_models import ChatOpenAI
from langchain.prompts import MessagesPlaceholder
from langchain.memory import ConversationBufferMemory

# Assuming you have a language model and tools
llm = ChatOpenAI(temperature=0, model="gpt-3.5-turbo-0613")
search = SerpAPIWrapper()
llm_math_chain = LLMMathChain.from_llm(llm=llm, verbose=True)
db = SQLDatabase.from_uri("sqlite:///../../../../../notebooks/Chinook.db")
db_chain = SQLDatabaseChain.from_llm(llm, db, verbose=True)
tools = [
    Tool(
        name="Search",
        func=search.run,
        description="useful for when you need to answer questions about current events. You should ask targeted questions",
    ),
    Tool(
        name="Calculator",
        func=llm_math_chain.run,
        description="useful for when you need to answer questions about math",
    ),
    Tool(
        name="FooBar-DB",
        func=db_chain.run,
        description="useful for when you need to answer questions about FooBar. Input should be in the form of a question containing full context",
    ),
]

# Create a custom system message
custom_system_message = MessagesPlaceholder(variable_name="memory")

# Create the memory
memory = ConversationBufferMemory(memory_key="memory", return_messages=True)

# Create the agent
agent = OpenAIMultiFunctionsAgent.from_llm_and_tools(
    llm,
    tools,
    extra_prompt_messages=[custom_system_message],
)

In this example, llm is an instance of ChatOpenAI which is the language model to use. tools is a list of tools the agent has access to. memory is the memory instance that allows the agent to remember intermediate steps. extra_prompt_messages is the custom system message to use. If not provided, a default one will be used.

I hope this helps! If you have any other questions, feel free to ask.

Sources

• Pass custom System Message to OpenAI Functions Agent
• libs/langchain/langchain/agents/agent_toolkits/conversational_retrieval/openai_functions.py
• docs/extras/modules/agents/how_to/add_memory_openai_functions.ipynb
• libs/langchain/langchain/agents/openai_functions_agent/base.py
• libs/langchain/langchain/agents/openai_functions_multi_agent/base.py
• libs/langchain/langchain/agents/agent.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[haesleinhuepf]

This is not the answer to the question above. memory is not connected to the agent.

[rlnasuti]

Figured it out.

llm = ChatOpenAI()

memory = ConversationBufferMemory(
    llm=llm, 
    memory_key="memory", 
    return_messages=True)

tools = [dummy_tool]

_system_message = "Talk like Matt Mercer"

prompt = OpenAIFunctionsAgent.create_prompt(
    system_message=SystemMessage(content=_system_message),
    extra_prompt_messages=[MessagesPlaceholder(variable_name="memory")],
)

agent = OpenAIFunctionsAgent(llm=llm, tools=tools, prompt=prompt)

agent_executor = AgentExecutor(
    agent=agent,
    tools=tools,
    memory=memory,
    verbose=True,
    return_intermediate_steps=False,
)

[rlnasuti]

dosu-bot's solution is cleaner and it works

[dosubot[bot]]

🤖

Based on the information provided, it seems like you have a solid understanding of how to instantiate an OpenAIFunctionsAgent and an AgentExecutor in the LangChain framework. However, without the specific implementation details of the solution provided by dosu-bot, it's challenging to provide a direct comparison or suggest improvements.

That being said, I can provide some general advice on how you might improve your current implementation:

1. Tool Validation: Ensure that the dummy_tool you're using is compatible with the OpenAIFunctionsAgent. The agent's tools should be instances of the BaseTool class or its subclasses. If dummy_tool is not compatible, you might encounter issues during execution.

2. Prompt Creation: Your current implementation uses a hardcoded system message ("Talk like Matt Mercer") and the memory as extra prompt messages. Depending on your use case, you might want to make this more dynamic. For example, you could create a function that generates system messages based on certain conditions.

3. Error Handling: Consider how you want to handle errors during execution. The AgentExecutor class provides the handle_parsing_errors attribute, which determines how to handle errors raised by the agent's output parser. You might want to set this attribute based on your specific needs.

4. Execution Control: You're currently using the default values for max_iterations, max_execution_time, and early_stopping_method. Depending on your use case, you might want to adjust these values. For example, if you're dealing with complex tasks that require more iterations, you might want to increase max_iterations.

5. Verbose Logging: You've set verbose=True, which is great for debugging, but in a production environment, you might want to turn this off to improve performance.

Without more specific details about the dosu-bot's solution, it's hard to provide more targeted advice. If you could provide more details about the dosu-bot's solution, I'd be happy to help further.

Sources

• libs/langchain/langchain/agents/openai_functions_agent/base.py
• libs/langchain/langchain/agents/agent.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.