RFC: Add return_direct to Tools, Actions, and FunctionCalling Agents#532
RFC: Add return_direct to Tools, Actions, and FunctionCalling Agents#532
return_direct to Tools, Actions, and FunctionCalling Agents#532Conversation
|
@douglas-reid does this seem like a reasonable way to achieve this? It would result in:
|
|
@eob my first reaction is that this is fine, as it seems to address an immediate need. it feels like "llm smell" in the sense that a summarizing action seems like something that should be ideally smoothed out of workflows at a different level. and i worry that there are likely tools that don't always want to return |
douglas-reid
left a comment
douglas-reid
left a comment
There was a problem hiding this comment.
Some initial reactions.
| output: Optional[List[Block]] | ||
| """Any direct output produced by the Tool.""" | ||
|
|
||
| return_direct: bool = False |
There was a problem hiding this comment.
curious: should we label this more like is_final or tool_is_direct similar to track that state (and update the comment -- as i'm not expecting users to set return_direct on Actions) ?
aside: Would this be "simpler" if the Action returned by an Agent/AgentExecutor when Tool.return_direct == True was a FinishAction instead? Or even a new type of Action (ex: DirectReturnAction) ? Maybe that would be more complex (as the AgentExecutor would have to be involved) ??
| return_direct: bool = False | ||
| """Whether to return the tool's output directly. | ||
|
|
||
| Setting this to True means that after the tool is called, the AgentExecutor will stop looping. |
There was a problem hiding this comment.
| Setting this to True means that after the tool is called, the AgentExecutor will stop looping. | |
| Setting this to True means that after the Tool is run, the Tool's output will be returned without further Agent interactions. This will preempt the execution of other tools, as well as any subsequent calls to LLMs, etc., to interpret the Tool output. |
| self.run_action(agent=agent, action=action, context=context) | ||
|
|
||
| # If this action wishes to end the loop, comply with that wish | ||
| if action.return_direct: |
There was a problem hiding this comment.
aside (take it or leave it): my api spidey-sense is tingling seeing this code block. i think seeing this, i'd prefer something like action.is_final (or even if isinstance(action, DirectReturnAction)) for clarity (as Actions aren't really "returning" anything per se).
|
|
||
| # If this action wishes to end the loop, comply with that wish | ||
| if action.return_direct: | ||
| logging.info( |
There was a problem hiding this comment.
curious: do you think we need this log statement for full coverage? WDYT about just adding "is_final" or similar to the existing "next tool" log? that might lead to more readable code (no floating break) and fewer lines of logging code?
| # If this action wishes to end the loop, comply with that wish | ||
| if action.return_direct: | ||
| logging.info( | ||
| f"Exciting reasoning loop by request of: {action.tool}", |
There was a problem hiding this comment.
| f"Exciting reasoning loop by request of: {action.tool}", | |
| f"Exiting Agent execution by request of: {action.tool}", |
|
@douglas-reid thanks a ton -- will do some tinkering down these threads. A few thoughts I had that are in the same vein as yours:
|
2 participants
This PR adds a
return_directflag toToolthe style of LangChain that permits a tool to signal that its response should be delivered as the "final answer" to the user.This prevents a common error condition in which a reasoning agent runs a "terminal" tool, but then instead of returning its output, it either:
Here's an example of that in action with the Prompt Golf bot:
What happened here is:
GameTurnToolwhich generated a new image