Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Agent][Bug Fix][Enhancement] Making default implementation of _validate_tools less restrictive #3803

Conversation

philipk19238
Copy link

Overview

The default implementation of _validate_tools is too restrictive as it raises a ValueError when multi-input tools are passed in. This does not make sense as many agents rely on tools that require some sort of multi-input and there are many other issues referencing this problem.

The default implementation should be empty and Agent children classes should explicitly define their own validator if they need it.

References

Copy link

@mmunagala mmunagala left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It makes sense for the deriving class to overrride vs. checking validating the tool here.

@philipk19238 philipk19238 changed the title [Agent] Making default implementation of _validate_tools less restrictive [Agent][Bug Fix/Enhancement] Making default implementation of _validate_tools less restrictive Apr 30, 2023
@philipk19238 philipk19238 changed the title [Agent][Bug Fix/Enhancement] Making default implementation of _validate_tools less restrictive [Agent][Bug Fix][Enhancement] Making default implementation of _validate_tools less restrictive Apr 30, 2023
@pranjaldoshi96
Copy link
Contributor

This restriction must have been added as agent might not behave appropriately if multi-input tools are provided. One of the maintainer might know.

@vowelparrot
Copy link
Contributor

The current agents scored poorly on internal tests when using multi-input tools. The Chat Agent is the closest to ready for it

@philipk19238
Copy link
Author

philipk19238 commented May 1, 2023

I'm not sure why they have to quality control single vs multi-input. This decision should be made by the developer and if there are any issues they can go and adjust the prompts that are passed in. I've overwritten this code in our in-house repo and our models work perfectly.

If there are issues with certain Agents (ZeroShot etc etc) then this restriction should be coded in method of the children class - not inherited in the base Agent class.

Copy link
Contributor

@hwchase17 hwchase17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think removing it from the base class makes sense, but we should also add it to all existing children classes

@philipk19238
Copy link
Author

philipk19238 commented May 2, 2023

i think removing it from the base class makes sense, but we should also add it to all existing children classes

Are you suggesting to implement the check for multi input tools into existing children classes (vs inheritance)? I agree from a design standpoint but still doesn't solve the issue at hand as users still want to use multi input tools regardless of performance.

@hwchase17
Copy link
Contributor

we added an agent that can use tools with multiple inputs - hope this helps!

@hwchase17 hwchase17 closed this May 3, 2023
@Celthi
Copy link

Celthi commented May 5, 2023

we added an agent that can use tools with multiple inputs - hope this helps!

Which agent?

@RatanPrakash
Copy link

we added an agent that can use tools with multiple inputs - hope this helps!

Which Agent? kindly mention here. or just merge the PR to accept multi input tool as the current one is too restrictive.

@evantancy
Copy link

@Celthi @RatanPrakash it's the structured chat agent

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants