Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Don't try tools that have already been invalidated in the current run #702

Closed
drshrey opened this issue Jan 23, 2023 · 2 comments
Closed

Don't try tools that have already been invalidated in the current run #702

drshrey opened this issue Jan 23, 2023 · 2 comments

Comments

@drshrey
Copy link

drshrey commented Jan 23, 2023

If you reproduce the agents example in https://langchain.readthedocs.io/en/latest/getting_started/getting_started.html but remove the "serpapi" parameter, the agent gets into an infinite loop with respect to trying actions with tools in which it has already observed the tool is not a valid tool.

A proposed fix is to maintain a set of all visited tools and do a check before having the agent settle on using that tool. I haven't seen the source code to validate this viability, but this outcome definitely seems not ideal.

Screenshot 2023-01-23 at 12 01 45 PM
@drshrey drshrey changed the title Don't try actions that have already been invalidated Don't try tools that have already been invalidated Jan 23, 2023
@drshrey drshrey changed the title Don't try tools that have already been invalidated Don't try tools that have already been invalidated in the same run Jan 23, 2023
@drshrey drshrey changed the title Don't try tools that have already been invalidated in the same run Don't try tools that have already been invalidated in the current run Jan 23, 2023
@jackapbutler
Copy link

Hey @hwchase17, I'm a research engineer working on language modelling and I'm interested in contributing to langchain. I'm happy investigate this and try to fix it? Do you have any advice or thoughts on the best way to proceed?

@kurehajime kurehajime mentioned this issue Feb 28, 2023
Merged
hwchase17 pushed a commit that referenced this issue Mar 1, 2023
@kurehajime
Fixed arguments passed to InvalidTool.run(). (#1340)
6c629b5 
[InvalidTool.run()](https://github.com/hwchase17/langchain/blob/72ef69d1ba33f052bf3948c1e1d7d6441b14af0a/langchain/agents/tools.py#L43)
returns "{arg}is not a valid tool, try another one.".
However, no function name is actually given in the argument.
This causes LLM to be stuck in a loop, unable to find the right tool.

This may resolve these Issues.
#998
#702
zachschillaci27 pushed a commit to zachschillaci27/langchain that referenced this issue Mar 8, 2023
@kurehajime @zachschillaci27
Fixed arguments passed to InvalidTool.run(). (langchain-ai#1340)
2e40419 
[InvalidTool.run()](https://github.com/hwchase17/langchain/blob/72ef69d1ba33f052bf3948c1e1d7d6441b14af0a/langchain/agents/tools.py#L43)
returns "{arg}is not a valid tool, try another one.".
However, no function name is actually given in the argument.
This causes LLM to be stuck in a loop, unable to find the right tool.

This may resolve these Issues.
langchain-ai#998
langchain-ai#702
mikeknoop pushed a commit to zapier/langchain-nla-util that referenced this issue Mar 9, 2023
@kurehajime
Fixed arguments passed to InvalidTool.run(). (#1340)
a3a8877 
[InvalidTool.run()](https://github.com/hwchase17/langchain/blob/aa7e6ae88e1aa6777bc018fcb7f185bb6f1b6f21/langchain/agents/tools.py#L43)
returns "{arg}is not a valid tool, try another one.".
However, no function name is actually given in the argument.
This causes LLM to be stuck in a loop, unable to find the right tool.

This may resolve these Issues.
langchain-ai/langchain#998
langchain-ai/langchain#702
@dosubot
Copy link

dosubot bot commented Aug 19, 2023

Hi, @drshrey. I'm helping the LangChain team manage their backlog and I wanted to let you know that we are marking this issue as stale.

From what I understand, the issue is that the agents example gets stuck in an infinite loop when running without the "serpapi" parameter. It seems that a proposed fix has been suggested, which involves maintaining a set of visited tools and checking before settling on using a tool. @jackapbutler, a research engineer, has expressed interest in investigating and fixing the issue.

Before we proceed, we would like to confirm if this issue is still relevant to the latest version of the LangChain repository. If it is, please let us know by commenting on the issue. Otherwise, feel free to close the issue yourself or it will be automatically closed in 7 days.

Thank you for your understanding and contribution to the LangChain project.

@dosubot dosubot bot added the stale Issue has not had recent activity or appears to be solved. Stale issues will be automatically closed label Aug 19, 2023
@dosubot dosubot bot closed this as not planned Won't fix, can't repro, duplicate, stale Sep 10, 2023
@dosubot dosubot bot removed the stale Issue has not had recent activity or appears to be solved. Stale issues will be automatically closed label Sep 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@drshrey @jackapbutler