AgentType.OPENAI_FUNCTIONS did not work

[zingzheng]

System Info

langchain 0.0.249
Python 3.11.2

Who can help?

No response

Information

• The official example notebooks/scripts
• My own modified scripts

Related Components

• LLMs/Chat Models
• Embedding Models
• Prompts / Prompt Templates / Prompt Selectors
• Output Parsers
• Document Loaders
• Vector Stores / Retrievers
• Memory
• Agents / Agent Executors
• Tools / Toolkits
• Chains
• Callbacks/Tracing
• Async

Reproduction

I follow the egs of openai_functions_agent at

https://python.langchain.com/docs/modules/agents/
https://python.langchain.com/docs/modules/agents/agent_types/openai_functions_agent
https://python.langchain.com/docs/modules/agents/how_to/custom-functions-with-openai-functions-agent

from langchain.chat_models import ChatOpenAI
from langchain.schema import SystemMessage
from langchain.agents import OpenAIFunctionsAgent
from langchain.agents import tool
from langchain.agents import AgentExecutor

llm = ChatOpenAI(temperature=0)

@tool
def get_word_length(word: str) -> int:
    """Returns the length of a word."""
    import pdb   # never go in here
    pdb.set_trace()
    return len(word)

tools = [get_word_length]

system_message = SystemMessage(content="You are very powerful assistant, but bad at calculating lengths of words")
prompt = OpenAIFunctionsAgent.create_prompt(system_message=system_message)

agent = OpenAIFunctionsAgent(llm=llm, tools=tools, prompt=prompt)
agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)

agent_executor.run("how many letters in the word educa?")

But the openai_functions_agent do not invoke the tool as expected。

> Entering new AgentExecutor chain...
[{'role': 'system', 'content': 'You are very powerful assistant, but bad at calculating lengths of words'}, {'role': 'user', 'content': 'how many letters in the word educa?'}]
There are 5 letters in the word "educa".

> Finished chain.

I tried using other tools but they were not used . It seems that OPENAI_FUNCTIONS has bugs。

Expected behavior

AgentType.OPENAI_FUNCTIONS works as docs show

[dosubot]

Answer generated by a 🤖

Answer

Hello,

Thank you for bringing this to our attention. The OpenAIFunctionsAgent may not be invoking the tool as expected due to several reasons:

1. The tool is not included in the list of allowed tools. The get_allowed_tools method returns a list of tool names that the agent can use. If the tool is not in this list, it will not be invoked.

2. The tool is not correctly formatted for OpenAI's function API. The functions property returns a list of tools formatted for OpenAI's function API using the format_tool_to_openai_function function. If there is an issue with the formatting, the tool may not be invoked correctly.

3. The tool is not correctly parsed from the AI message. The _parse_ai_message function parses the AI message to extract the tool name and input. If there is an issue with the message format, the tool may not be invoked correctly.

To resolve the issue, ensure that the tool is included in the list of allowed tools, correctly formatted for OpenAI's function API, and correctly parsed from the AI message.

In the context shared, it's also important to note that the OpenAIMultiFunctionsAgent class is used, which supports multiple actions in a single invocation. This could be a potential source of issues if the tool is not designed to handle multiple actions.

If you have followed all these steps and the issue still persists, could you please provide more information about the error message you are receiving or any other unusual behavior you are noticing? This will help us to better understand the issue and provide a more accurate solution.

Best,
Dosu

Sources

• libs/langchain/langchain/agents/openai_functions_agent/base.py
• libs/langchain/langchain/agents/openai_functions_multi_agent/base.py

---

This response is meant to be useful, save you time, and share context. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

[dosubot]

Hi, @zingzheng! I'm Dosu, and I'm here to help the LangChain team manage our backlog. I wanted to let you know that we are marking this issue as stale.

Based on my understanding of the issue, the problem you reported is that the OpenAIFunctionsAgent is not invoking the specified tool as expected. I mentioned a few potential reasons for this, such as the tool not being included in the list of allowed tools, incorrect formatting for OpenAI's function API, or incorrect parsing from the AI message. I also mentioned that using the OpenAIMultiFunctionsAgent class could potentially cause issues if the tool is not designed to handle multiple actions.

Before we proceed, we would like to confirm if this issue is still relevant to the latest version of the LangChain repository. If it is, please let us know by commenting on this issue. Otherwise, feel free to close the issue yourself or it will be automatically closed in 7 days.

Thank you for your understanding and cooperation!