How to integrate RAG pipeline with Websearch fallback?

[M-JULIANI]

Hello,

Trying to tie a RAG pipeline (with a pgVectorStore) to a fallback to websearch flow (from this tutorial) if the requested info is not found.

Current RAG Pipeline (functional)

       rag_pipeline = Pipeline()
        # Add components to your pipeline
        rag_pipeline.add_component("text_embedder", text_embedder)
        rag_pipeline.add_component("retriever", PgvectorEmbeddingRetriever(document_store=self.document_store, filters = {} if filters is None else filters, top_k=num_responses))
        rag_pipeline.add_component("prompt_builder", prompt_builder)
        rag_pipeline.add_component("llm", generator)
        rag_pipeline.add_component(instance=AnswerBuilder(), name="answer_builder")

        # Now, connect the components to each other
        rag_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
        rag_pipeline.connect("retriever", "prompt_builder.documents")
        rag_pipeline.connect("prompt_builder", "llm")
        rag_pipeline.connect("llm.replies", "answer_builder.replies")
        rag_pipeline.connect("llm.meta", "answer_builder.meta")
        rag_pipeline.connect("retriever", "answer_builder.documents")

Modified RAG Pipeline

    rag_pipeline = Pipeline()
        # Add components to your pipeline
        rag_pipeline.add_component("text_embedder", text_embedder)
        rag_pipeline.add_component("retriever", PgvectorEmbeddingRetriever(document_store=self.document_store, filters = {} if filters is None else filters, top_k=num_responses))
        rag_pipeline.add_component("prompt_builder", prompt_builder)
        rag_pipeline.add_component("llm", generator)
        rag_pipeline.add_component("router", router)
        rag_pipeline.add_component("websearch", websearch)
        rag_pipeline.add_component("prompt_builder_for_websearch", prompt_builder_for_websearch)
        rag_pipeline.add_component("llm_for_websearch", llm_for_websearch)
        rag_pipeline.add_component(instance=AnswerBuilder(), name="answer_builder")

        # Now, connect the components to each other
        rag_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
        rag_pipeline.connect("retriever", "prompt_builder.documents")
        rag_pipeline.connect("prompt_builder", "llm")
        
        rag_pipeline.connect("llm.replies", "router.replies")
        rag_pipeline.connect("router.go_to_websearch", "websearch.query")
        rag_pipeline.connect("router.go_to_websearch", "prompt_builder_for_websearch.query")
        rag_pipeline.connect("websearch.documents", "prompt_builder_for_websearch.documents")
        rag_pipeline.connect("prompt_builder_for_websearch", "llm_for_websearch")

        rag_pipeline.connect("llm_for_websearch.replies", "answer_builder.replies")
        rag_pipeline.connect("llm_for_websearch.meta", "answer_builder.meta")
        rag_pipeline.connect("retriever", "answer_builder.documents")

Testing it by asking something that is clearly not in the DB: 'What is the population of Quito?'

Response

{
    "llm": {
        "meta": [
            {
                "finish_reason": "stop",
                "index": 0,
                "model": "gpt-4-0125-preview",
                "usage": {
                    "completion_tokens": 14,
                    "prompt_tokens": 2403,
                    "total_tokens": 2417
                }
            }
        ]
    },
    "router": {
        "answer": "The provided information does not include any data about Quito's population."
    }
}

Diagnostic

Clearly the router isn't being connected properly to the pipeline

Ask

Any input on how to connect the pipeline correctly is appreciated

Marco

[M-JULIANI]

Ah, I think I solved it. However other inputs would be appreciated of course.

      rag_pipeline = Pipeline()

        # Add components to your pipeline
        rag_pipeline.add_component("text_embedder", text_embedder)
        rag_pipeline.add_component("retriever", PgvectorEmbeddingRetriever(document_store=self.document_store, filters={} if filters is None else filters, top_k=num_responses))
        rag_pipeline.add_component("prompt_builder", prompt_builder)
        rag_pipeline.add_component("llm", generator)
        rag_pipeline.add_component("router", router)  # Component to decide on fallback
        rag_pipeline.add_component("websearch", websearch)  # Fallback web search component
        rag_pipeline.add_component("prompt_builder_for_websearch", prompt_builder_for_websearch)  # Prompt builder for web search
        rag_pipeline.add_component("llm_for_websearch", llm_for_websearch)  # Language model for processing web search results
        rag_pipeline.add_component(instance=AnswerBuilder(), name="answer_builder")

        # Connect the components to each other
        rag_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
        rag_pipeline.connect("retriever", "prompt_builder.documents")
        rag_pipeline.connect("prompt_builder", "llm")

        # Connect router to handle decisions
        rag_pipeline.connect("llm.replies", "router.replies")
        rag_pipeline.connect("router.go_to_websearch", "websearch.query")
        rag_pipeline.connect("router.go_to_websearch", "prompt_builder_for_websearch.query")
        rag_pipeline.connect("websearch.documents", "prompt_builder_for_websearch.documents")
        rag_pipeline.connect("prompt_builder_for_websearch", "llm_for_websearch")

        # Ensure answers from both flows reach the answer builder
        rag_pipeline.connect("llm_for_websearch.replies", "answer_builder.replies")
        rag_pipeline.connect("llm_for_websearch.meta", "answer_builder.meta")
        rag_pipeline.connect("retriever", "answer_builder.documents")

[mrm1001]

Hi @M-JULIANI The last pipeline you published seems to be the same as the previous one. What was the problem at the end?

[M-JULIANI]

@mrm1001 hmm yeah that's weird, seems i must have copied the corrected code-block in somehow (?)

I was having trouble with the last bit, making sure the llm_for_websearch was connected properly to answer_builder.

Generally struggling to understand when things should connect to one another. At a high level it makes sense to me that certain components should be connected to one another, but when you start specifying the inputs and outputs of components, I feel less confident there.