Investigate and refine form-filling bot

[mrm1001]

There are many use cases where agents might need to extract some structured input from a user in the form of question-answer pairs.

User story

• I need to build an agent AccountRetriever that gathers user details (e.g. email and name) in order to fetch the account information of an end user.
• I need to build a fulfilment agent ProductFinder that needs to get product details for an order (e.g. clothes size, number of items).

This agent needs to be able to be used in a multi-agent architecture. For example, a user might want to build a travel bot with the following agents working together: AccountRetriever, FlightBookingAgent, CarRentalBookingAgent, etc. The actual multi-agent architecture is outside the scope of this issue.

Outcome

The outcome of this issue would be a colab or notion page with either pseudo-code or a diagram explaining how this problem could be solved from an end-user point of view, without going into implementation details.

[julian-risch]

Here is the notebook: https://colab.research.google.com/drive/1LY5-OcMLU4luHdNCBODpH8s53Sy0Z73-?usp=sharing

[mrm1001]

• Architecture:

  • There should be a validation step to support the following flow:
    • Please enter your name, city and email
    • My email is bla and my name is Maria.
    • Your email is not correct, please enter a new email. Your city is missing.
  • You decided to decompose it into 2 steps rather than doing the json extraction and figuring out the missing fields in one step. This way is more expensive since you need to do 2 LLM calls at every iteration. One alternative is doing both steps in one go. For example, you could use function calling. OpenAI uses function calling for structured data extraction (see build hour). I have done a demo here: https://colab.research.google.com/drive/1VoGlxmEQ8YSWSEEOujHzYf876YQCmhQP?usp=sharing. We don't need to decide which way is best, but we could maybe make 2 cookbooks instead of one with both approaches.
  • Follow-up question is: do we have a notebook with function calling in a pipeline with memory? Similar to the tutorial on function calling but with a pipeline?

• The routing

  • How about renaming llm_decision_router to routing_llm? It makes it more clear that it's not actually doing any branching out.

• user_message_template

  • In order to make the user’s life easier and avoid them doing too much prompt eng, it would be neat to be able to write the prompt as a function of the schema/parameters that need to be extracted from the user. When using function calling, you just pass the schema and there is little prompt eng involved.

• Bigger topics:

  • Once we have pipelineComponents, we could look into creating one that is a prompt_builder with a generator together. It’s a pattern that you see in almost every pipeline and it makes the pipeline quite bulky.
  • I find the ConditionalRouter a bit difficult to use, and I have examples of our users struggling too. To define the routes it’s difficult to define the rules, specially if the content is nested like replies[0].content. Maybe we could review this component in the future. Would it make sense to create a new component in the future which does the 2 things: calls an LLM and also defines routes based on the output? I would find it easier to define the two things together in one component, and it might make the route definition easier. Like a LLMRouter.

[julian-risch]

Thank you for your feedback!

do we have a notebook with function calling in a pipeline with memory? Similar to the tutorial on function calling but with a pipeline?

The notebook on ReAct agent using one tool should be using memory and function calling once it's ready.

How about renaming llm_decision_router to routing_llm? It makes it more clear that it's not actually doing any branching out.

Good idea. I updated the notebook to address this feedback.

user_message_template

I agree that it would be a nice simplification for the user to mention the form fields only once and infer the prompt message from the schema. I updated the notebook.

Would it make sense to create a new component in the future which does the 2 things: calls an LLM and also defines routes based on the output?

I can imagine a PipelineComponent doing all of that, yes. Would wrap PromptBuilder, Generator, and ConditionalRouter.

With DevRel taking over the work on the cookbook(s) I'll close this issue now. The tracking issue for cookbooks is here: https://github.com/deepset-ai/devrel-board/issues/435