{tool_name} is not a valid tool, try another one. #1559
Comments
|
I got the same error while trying |
|
I'm having the same issue when using Anyone who has an idea on how to fix this? |
|
Same problem here. Trying to use gpt-3.5-turbo, it works with the regular model. Langchain v 0.0.123 Code: Error: |
|
I got it working by using As far as i know, that's not documented anywhere. |
|
I keep getting the same issue, even when using chat-zero-shot-react-description as agent instead of zero-shot-react-description. There should be an alternative way to handle this |
|
Mine also likes to say "Action: Use {tool_name}" instead of "Action {tool_name}", and it then gets stuck in an infinite loop trying until my context runs out |
|
same issue with gpt-4 and gpt-3.5-turbo both, even though the tool it selected is correct, the agent says it is not a valid tool. |
|
Just to add, the latest gpt4all model has the same problem with agent: |
|
I am using gpt-3.5-turbo, work totally fine when set the agent to be like this: agent = initialize_agent(tools,
llm,
agent="chat-zero-shot-react-description",
verbose=True)after that the agent can choose right tool to use in the specific cases |
|
That did it! Thanks! |
Yeah I've did this but it still sometimes wants to use "Use " instead of "" like maybe once every 5-10 tries.. |
|
I ran into this issue. When inspecting the langchain code, in my case it was being thrown by this line: https://github.com/hwchase17/langchain/blob/bee59b4689fe23dce1450bde1a5d96b0aa52ee61/langchain/agents/agent.py#L760 After digging deeper it seemed to be thrown if the custom tool had a different class name (excluding whitespace) to what it's whereas this would throw the error Try changing them to match and see if that helps! |
|
I think it's a matter of the gpt model performance. When the model doens't understand and complete exactly the same as what we want, e.g. upper case vs lower case, paraphrase of the name, the procedure just doesn't move on to the next step (since now it requires the action should be exactly the same as one of the tool names defined) |
|
Came across after getting errors implementing tokens used code Worked by selecting agent type of |
|
I have the same problem, but is not always. Sometimes the tools works fine and other times I got this error. How can it be solved? |
|
Has anyone here tried Bard or Llama instead? I am on the waiting list for GPT-4, which certainly will improve this in relation to understanding which tool to use, however, I can't wait. It is a tricky bug because it works nicely sometimes. |
same here, error details: |
yeah, sometimes works, sometimes fail, any update? |
|
The error is being caused by ChatOpenAI. Please make the following substitution: Replace:
With:
|
|
Hello all, |
Same here |
|
I am getting the same of @catskytw, any news on that? |
|
same here with |
|
So, I was using the Google Search tool with LangChain and was facing this same issue. And here's what I understood and did the following to fix the error: Before I explain everything here's my code: import os
import dotenv
from langchain.llms import Cohere
from langchain.agents import initialize_agent, load_tools, AgentType
from langchain.utilities import GoogleSearchAPIWrapper
from langchain.tools import Tool
from langchain.prompts import PromptTemplate
from langchain.chains import LLMChain
if __name__ == "__main__":
dotenv.load_dotenv()
llm = Cohere(model="command-xlarge-20221108", cohere_api_key=os.environ["COHERE_API_KEY"])
prompt = PromptTemplate(
input_variables=["query"],
template="Write a summary of the following text: {query}"
)
summarize_chain = LLMChain(llm=llm, prompt=prompt)
search = GoogleSearchAPIWrapper()
search_tool = Tool(
name="LangChainSear",
description="Search Google for recent results.",
func=search.run,
)
summary_tool = Tool(
name="Summarizer",
description="Useful for summarizing texts.",
func=summarize_chain.run
)
agent = initialize_agent(
tools=[search_tool, summary_tool],
llm=llm,
agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
verbose=True,
max_iterations=4
)
agent.run("What is LangChain? Then please summarize the results.")
If we maintain the above cases, then the framework seems to work and produces correct results. Let me know your thoughts. |
|
Same issue here when using |
Hi I found someone providing another work around by creating a blind director, but it's just reducing the fee of davinci slightly. Due to using azure openai which still doesn't provide fuction calling, I have no chance to change the agent type from ZERO_SHOT_REACT_DESCRIPTION fo Function Calling(Fuction calling should solve this issue as far as I know) Appreciate if anyone can give me some suggestions or tips. |
|
Me & @StereoPT have found a pretty good solution With GPT-35-Turbo LLM (we were using the Azure Open AI but strongly believe this also works with the regular one as should be the same model), we wanted to mess around with a CSV so we used this: Basically we were getting an error everytime gpt generated code because... it was wrapped in ` agent = create_pandas_dataframe_agent(
prefix="Remove any ` from the Action Input",
llm=self.model,
df=df,
verbose=True,
max_iterations=5,
)We never had any problems after this simple change, hope this helps! |
|
Thank you for your reply @ricardosantosduarte , but it didn't work for me. The only strategy that works for me is to use davinci as my OpenAI model, which is really expensive compared to GPT-3.5-turbo. |
|
Thanks @ricardosantosduarte , but it doesn't work on my side. |
|
watching this issue |
|
I have a similar issue, but using the HuggingFacePipeline decorator on the model tiiuae/falcon-40b. When running this: I get this: In other words, it thinks the "Action" is is the "Action Input", although the action should be "Action: python_repl_ast" Of course, some part of it might be the limitations of falcon-40b, but the model is quite capable (see huggingfaces leaderboard) and it writes the correct query. Edit: open_llama_13b faces a similar issues:Thought: I think there will be a lot of output, but I think it is best to use the timestamp to see if there is a difference. |
|
I was having the same issue as many above with the pandas dataframe agent. After playing around with different models/agent_types I found that the following configuration seems to work ok (I have no idea why): |
It's the AgentType.OPENAI_FUNCTIONS that made the difference |
|
watching this issue |
|
Watching this issue |
|
this is maddening |
|
I think I found a workaround for this issue which requires a bit of prompt engineering In your prompt, you should mention how the model would answer Question: Write a Python agent.run("""
For instance:
Question: Find out how much 2 plus 2 is.
Thought: I must use the Python shell to calculate 2 + 2
Action: Python REPL
Action Input:
2 + 2
Observation: 4
Thought: I now know the answer
Final Answer: 4
Example 2:
Question: You have a variable age in your scope. If it's greater or equal than 21, say OK. Else, say Nay.
Thought: I should write an if/else block in the Python shell.
Action: Python REPL
Action Input:
if age >= 21:
print("OK") # this line has four spaces at the beginning
else:
print("Nay") # this line has four spaces at the beginning
Observation: OK
Thought: I now know the answer
Final Answer: I have executed the task successfully.
Now begin for real!
Question: Write a Python script that prints "Hello, world!"
""") This worked for me Source: https://betterprogramming.pub/creating-my-first-ai-agent-with-vicuna-and-langchain-376ed77160e3 |
|
Another workaround is to change line 113 in
From
To |
|
I am facing an issue related to this #8407 Could someone help? |
|
Whole heartedly agree with the comment 'this is maddening'. Can't think of another time in my life when a bug was so randomly irreproducible. It works for hours and then shits the bed. While I certainly appreciate all the flavors of workarounds being tossed out here - is there anyone who has a theory as to what the actual issue is that is causing the agent to completely lose sight of a tool it references earlier in the observation?
|
|
Pure guess, but it almost feels like there's an (a)synchronous issue at play. Like maybe the library isn't quite finished being referenced/loaded before the agent begins to execute. |
Fantastic theory @TimLafferty . We will need someone smarter than me to validate that :) |
|
@mvonoven In your case, the llm understands what it needs to do, but doesn't understand how to format it and the format confused me too as a human 🤖. After "action:" langchain wants it to write a tool name, right? But imo, in the agent prompts, instead of writing action, it should be just written as "tool:" and "action input:" should just be "input:" A tool name by itself is not an action. I think this has not been changed because most people are using good models such as openai's gpt's so they haven't run into this issue as much. |
|
Some of the comments actually provided a hint for the answer.
On run time you should see the following: |
|
Using
import pandas as pd
from typing import Any, Dict, List, Optional, Union
from langchain.agents.mrkl.output_parser import MRKLOutputParser
from langchain.callbacks import get_openai_callback
from langchain.callbacks.base import BaseCallbackHandler
from langchain.input import print_text
from langchain.llms import OpenAI
from langchain.memory import (
ConversationKGMemory,
ConversationSummaryMemory,
ConversationBufferWindowMemory,
CombinedMemory)
from langchain.agents import (
create_pandas_dataframe_agent,
AgentExecutor,
AgentType
)
from langchain.schema import AgentAction, AgentFinish, LLMResult
class CSVCallbackHandler(BaseCallbackHandler):
"""Callback Handler that prints to std out."""
def __init__(self, color: Optional[str] = None) -> None:
"""Initialize callback handler."""
self.color = color
def on_llm_start(self, serialized: Dict[str, Any], prompts: List[str], **kwargs: Any) -> None:
"""Print out the prompts."""
print("on_llm_start")
def on_llm_end(self, response: LLMResult, **kwargs: Any) -> None:
"""Do nothing."""
print("on_llm_end")
print(response)
def on_llm_new_token(self, token: str, **kwargs: Any) -> None:
"""Do nothing."""
print("______on_llm_new_token______")
print(token)
def on_llm_error(self, error: Union[Exception, KeyboardInterrupt], **kwargs: Any) -> None:
"""Do nothing."""
pass
def on_chain_start(self, serialized: Dict[str, Any], inputs: Dict[str, Any], **kwargs: Any) -> None:
"""Print out that we are entering a chain."""
class_name = serialized.get("name", "")
print("on_chain_start")
def on_chain_end(self, outputs: Dict[str, Any], **kwargs: Any) -> None:
print(outputs)
def on_chain_error(self, error: Union[Exception, KeyboardInterrupt], **kwargs: Any) -> None:
pass
def on_tool_start(self, serialized: Dict[str, Any], input_str: str, **kwargs: Any,) -> None:
pass
def on_agent_action(self, action: AgentAction, color: Optional[str] = None, **kwargs: Any) -> Any:
print(action)
def on_tool_end(self, output: str, color: Optional[str] = None, observation_prefix: Optional[str] = "Observation: ", llm_prefix: Optional[str] = None, **kwargs: Any,) -> None:
if observation_prefix is not None:
print(f"\n{observation_prefix}", output)
print(output)
if llm_prefix is not None:
print_text(f"\n{llm_prefix}", output)
print(output)
def on_tool_error(self, error: Union[Exception, KeyboardInterrupt], **kwargs: Any) -> None:
pass
def on_text(self, text: str, color: Optional[str] = None, end: str = "", **kwargs: Any,) -> None:
pass
def on_agent_finish(self, finish: AgentFinish, color: Optional[str] = None, **kwargs: Any) -> None:
pass
DEBUG = True
csv_cb = CSVCallbackHandler()
llm_code = OpenAI(temperature=0, callbacks=[csv_cb], streaming=False) # gpt-3.5-turbo-16k-0613
chat_history_buffer = ConversationBufferWindowMemory(k=50, memory_key="chat_history_buffer", input_key="input")
chat_history_summary = ConversationSummaryMemory(llm=llm_code, memory_key="chat_history_summary", input_key="input")
chat_history_summary.chat_memory.add_ai_message(
"This is data is extracted from multiple csv files and data is about campaigns"
)
chat_history_KG = ConversationKGMemory(llm=llm_code, memory_key="chat_history_KG", input_key="input")
memory = CombinedMemory(memories=[chat_history_buffer, chat_history_summary, chat_history_KG])
MULTI_DF_PREFIX = """
You are working with {num_dfs} pandas dataframes in Python named df1, df2, etc which represent multiple excel and csv files.
Your task is to analyze the df data and answer questions related to it.
You should use the tools below, summary and conversation history to answer the question:
If you do not have an answer to the question do not make one up reply I apologize I dont know the answer to the question is not related to the data:
Summary of the whole conversation:
{chat_history_summary}
Last few messages between you and user:
{chat_history_buffer}
Entities that the conversation is about:
{chat_history_KG}
"""
csv_data_1 = "/docs/91271562-2c89-4708-a861-67ebab915b26/Deco2u-Campaigns-20-May-202020-Jun-2023.csv"
csv_data_2 = "/docs/91271562-2c89-4708-a861-67ebab915b26/Untitled-report-Jun-3-2020-to-Jul-3-2023.csv"
csv_files = [csv_data_1, csv_data_2]
df = []
for item in csv_files:
if not isinstance(item, str):
raise ValueError(f"Expected str, got {type(item)}")
df.append(pd.read_csv(item))
pd_agent: AgentExecutor = create_pandas_dataframe_agent(
llm_code,
df,
agent_type=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
verbose=DEBUG,
prefix=MULTI_DF_PREFIX,
agent_executor_kwargs={
"memory": memory,
"handle_parsing_errors": True,
"output_parser": MRKLOutputParser()
},
input_variables=['num_dfs', 'dfs_head', 'input',
'agent_scratchpad', 'chat_history_buffer',
'chat_history_summary', 'chat_history_KG']
)
if DEBUG:
print("^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^")
print(pd_agent.agent.llm_chain.prompt.template)
print("^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^")
with get_openai_callback() as cb:
while True:
query = input("Ask me anything...")
resp = pd_agent.run({'input': query,
'agent_scratchpad': "", 'chat_history_buffer': chat_history_buffer,
'chat_history_summary': chat_history_summary, 'chat_history_KG': chat_history_KG})
print(resp) |
|
not sure if this is the cause...currently custom tools cannot be loaded by their names using the "load_tools" function. If that function is ever called under the hood to load the custom tools, it won't work. |
|
I have had this problem with a tool I never subclassed and simply instantiated as |
|
Hi, @gcsun, I'm helping the LangChain team manage their backlog and am marking this issue as stale. From what I understand, the issue you reported involves the second agent causing all tools to become invalid, displaying an error message indicating that the tool is not valid. There have been discussions around potential workarounds, such as using a different agent type or changing the tool's name, as well as suggestions to modify the system prompt and potential issues related to custom tools and tool names. However, the issue remains unresolved. Could you please confirm if this issue is still relevant to the latest version of the LangChain repository? If it is, please let the LangChain team know by commenting on the issue. Otherwise, feel free to close the issue yourself, or the issue will be automatically closed in 7 days. Thank you for your understanding and cooperation. |
dosubot
bot
added
the
stale
dosubot
bot
removed
the
stale
Thanks sooooo much - that resolved it. |
|
initialize_agent is deprecated, any new? |
and others
I encapsulated an agent into a tool ，and load it into another agent 。
when the second agent run, all the tools become invalid.
After calling any tool, the output is “*** is not a valid tool, try another one.
why! agent cannot be a tool?
The text was updated successfully, but these errors were encountered: