Agent tool sources #6854
Agent tool sources #6854
Conversation
| @@ -133,20 +148,23 @@ def chat( | |||
| print(log_str) | |||
|
|
|||
| # Query with standalone question | |||
| response = self._query_engine.query(condensed_question) | |||
| query_response = self._query_engine.query(condensed_question) | |||
| tool_output = self._get_tool_output_from_response( | |||
There was a problem hiding this comment.
if the query_response has source_nodes, this gets lost right?
There was a problem hiding this comment.
nope, it ends up in the raw_output of the tool output -> see self._get_tool_output_from_response()
|
|
||
| logger = logging.getLogger(__name__) | ||
|
|
||
|
|
||
| class StreamingChatResponse: | ||
| @dataclass | ||
| class AgentChatResponse: |
There was a problem hiding this comment.
does this break backwards compat for existing chat engines/agents in parsing the response type?
There was a problem hiding this comment.
which backwards compat are you thinking of? The only breaking change I noticed was that agents don't stream a generator of generators anymore, just a single generator for the final response
There was a problem hiding this comment.
oh mostly that the existing chat engine response type was RESPONSE_TYPE, but now it's AgentChatResponse. So if the user was doing something like response.source_nodes it would break
There was a problem hiding this comment.
ah yea, that is a possibility 🤔
There was a problem hiding this comment.
could do a temporary property on AgentChatResponse called source_nodes (and try fetching raw_output.source_ndoes..., but seems hacky)
There was a problem hiding this comment.
I feel like breaking this is inevitable.
There was a problem hiding this comment.
and I suspect we are probably gonna break it a couple more times as we figure out the interface.
| @@ -24,7 +24,8 @@ To stream response: | |||
| ```python | |||
| chat_engine = index.as_chat_engine() | |||
| streaming_response = chat_engine.stream_chat("Tell me a joke.") | |||
| streaming_response.print_response_stream() | |||
There was a problem hiding this comment.
curious: why don't we preserve this API?
There was a problem hiding this comment.
we probably should!
llama_index/chat_engine/utils.py
Outdated
| from llama_index.memory import BaseMemory | ||
| from llama_index.types import TokenGen | ||
|
|
||
|
|
||
| def response_gen_with_chat_history( | ||
| message: str, memory: BaseMemory, response_gen: TokenGen | ||
| ) -> TokenGen: | ||
| ) -> Generator[ChatResponse, None, None]: |
There was a problem hiding this comment.
nit: Isn't this a ChatResponseGen ;)
Description
This exposes the names, inputs, and outputs of tools called in between each agent response.
BREAKING CHANGE ALERT: Agents no longer stream intermediate steps, in order to align with chat engine behaviour.
Type of Change
How Has This Been Tested?
Todo: