Skip to content

Extend AnswerBuilder to convert only last message for agentic pipelines #9330

Closed
@tstadel

Description

@tstadel

Is your feature request related to a problem? Please describe.
As most agents are expected to be run as a Chat application, returning exactly one answer per interaction/request/pipeline.run seems to be a defacto pattern.
The current Agent always returns all messages that have been accumulated during the agent invocation. To render a final answer (e.g. for end users), in most cases, we are only interested in the last message. Currently we need to use an OutputAdapter for this, which makes standard Agentic pipelines more complex than necessary.

Describe the solution you'd like
Add a flag to AnswerBuilder which allows to select only the last message to be converted as Answer.
Optional: Set any other messages to Answer's meta (e.g. for tool call inspections)

Describe alternatives you've considered
Adding the flag to Agent would come with the downside that any tool call / iteration messages will be lost.

Additional context
Add any other context or screenshots about the feature request here.

Activity

added
P1High priority, add to the next sprint
on May 2, 2025
sjrl

sjrl commented on May 5, 2025

@sjrl
Contributor

Maybe an alternative or in addition to this could be to add an additional output to the Agent called final_message of type ChatMessage which always contains the last message?

@bilgeyucel also mentioned for standalone uses of Agent that this would be helpful.

bilgeyucel

bilgeyucel commented on May 5, 2025

@bilgeyucel
Contributor

Agree with @sjrl here. Additionally, Haystack users don't use AnswerBuilder as they get a similar benefit through Pipeline's include_outputs_from parameter. Handling this in AnswerBuilder would force users to add AnswerBuilder to their pipeline

tstadel

tstadel commented on May 6, 2025

@tstadel
MemberAuthor

Interesting, I thought about this briefly too, but rejected it because

  • it doesn't allow one to add additional information from the rest of the messages, e.g. tool calls. For this you'd need all messages again. AnswerParser could take care of this, e.g. by filling Answer's meta fields with tool call info.
  • when using AnswerBuilder, returning a single message would require an OutputAdapter again, as AnswerBuilder expects list[Message] (not a single Message).
sjrl

sjrl commented on May 6, 2025

@sjrl
Contributor

Interesting, I thought about this briefly too, but rejected it because

  • it doesn't allow one to add additional information from the rest of the messages, e.g. tool calls. For this you'd need all messages again. AnswerParser could take care of this, e.g. by filling Answer's meta fields with tool call info.
  • when using AnswerBuilder, returning a single message would require an OutputAdapter again, as AnswerBuilder expects list[Message] (not a single Message).

That's fair! I think it might be worth doing all of the following:

  • Update AnswerBuilder to convert last message from Agent and put the rest into metadata for the reasons @tstadel outlined
  • Add a final_message of type ChatMessage (or final_answer of type string) output to Agent. This should be helpful for users who want to 1) use the component standalone 2) easily connect the final_message as input to another component other than AnswerBuilder in a Pipeline and 3) use Agent as a tool in ComponentTool where usually you'd want just the final answer as the tool output.
vblagoje

vblagoje commented on May 9, 2025

@vblagoje
Member

Hey @tstadel @sjrl, I sketched out what needs to be done in AnswerBuilder and it’s turning it into a complicated beast—right now AnswerBuilder verifies that every chat message contains valid text and throws a ValueError when it doesn’t and this is going to be problematic for agent messages that have tool calls and responses. We are mixing a lot of domain logic here, all the way from RAG, pattern matching, reference creation, and now on top of this we are pulling in agent logic into AnswerBuilder. We’d be better off spinning up a separate AgentAnswerBuilder that can look at the agent’s state and handle everything needed to produce the answer, however it needs to look. LMK

modified the milestone: 2.14.0 on May 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

P1High priority, add to the next sprint

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

    Development

    Participants

    @vblagoje@julian-risch@sjrl@bilgeyucel@tstadel

    Issue actions

      Extend AnswerBuilder to convert only last message for agentic pipelines · Issue #9330 · deepset-ai/haystack