haystack2x-demos/haystack_rag_serperdev_demo_new.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "authorship_tag": "ABX9TyMmb2aLOVYC49KL0SiETf9S",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/vblagoje/notebooks/blob/main/haystack2x-demos/haystack_rag_serperdev_demo_new.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Introduction\n",
        "\n",
        "This notebook provides a detailed guide on leveraging Haystack 2.x and [serper.dev](https://serper.dev/) for implementing an OpenAPI service-based Retriever-Augmented Generation (RAG) workflow. It outlines an approach for processing user queries by integrating web search results into the context of Large Language Models (LLMs), enhancing their response generation.\n",
        "\n",
        "The pipeline starts by fetching the OpenAPI specification for serper.dev and converting it into OpenAI function-calling definitions. serper.dev is then used as an OpenAPI service to retrieve relevant information from the web based on the user's input. The pipeline incorporates these search results, along with system and user prompts, into the LLM to enhance response generation.\n",
        "\n",
        "The notebook will guide you through installing the necessary libraries, selecting an LLM provider, and constructing a Haystack 2.0 pipeline that orchestrates function calling, service requests, and LLM response generation.\n",
        "\n",
        "Note: To run this pipeline, a [serper.dev](https://serper.dev/) account is required. Signing up is quick and provides you with 2,500 free queries—no credit card needed."
      ],
      "metadata": {
        "id": "J8TDTtQwiSn6"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 1. Setup\n",
        "\n",
        "Let's install necessary libraries and import key modules to build the foundation for the subsequent steps."
      ],
      "metadata": {
        "id": "6zUWNgF-kUeU"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install -q git+https://github.com/deepset-ai/haystack-experimental.git@openapi"
      ],
      "metadata": {
        "id": "bH8Lo7PSumLG",
        "outputId": "b6666b0f-c6bd-4ea1-87ab-ed150d3461d3",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "  Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n",
            "  Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n",
            "  Preparing metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m345.2/345.2 kB\u001b[0m \u001b[31m5.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m325.5/325.5 kB\u001b[0m \u001b[31m18.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m41.3/41.3 kB\u001b[0m \u001b[31m3.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m75.6/75.6 kB\u001b[0m \u001b[31m6.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m77.9/77.9 kB\u001b[0m \u001b[31m7.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m4.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h  Building wheel for haystack-experimental (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import getpass\n",
        "import os\n",
        "import json\n",
        "import requests\n",
        "\n",
        "from typing import Dict, Any, List\n",
        "\n",
        "from haystack import Pipeline\n",
        "from haystack.components.generators.utils import print_streaming_chunk\n",
        "from haystack.components.converters import OutputAdapter\n",
        "from haystack.components.generators.chat import OpenAIChatGenerator\n",
        "from haystack.dataclasses import ChatMessage, ByteStream\n",
        "from haystack.utils import Secret\n",
        "\n",
        "from haystack_experimental.components.tools.openapi import OpenAPITool, LLMProvider"
      ],
      "metadata": {
        "id": "RhxdVLCb_D8R"
      },
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 2.  Enter the API keys for LLM provider and serper.dev"
      ],
      "metadata": {
        "id": "V8GwXloVDtg9"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "llm_api_key = getpass.getpass(f\"Enter OpenAI api key for:\")\n",
        "serper_dev_key = getpass.getpass(\"Enter serperdev api key:\")"
      ],
      "metadata": {
        "id": "TrK-E_CF__MG",
        "outputId": "f0b7963d-998b-4e7f-ca3e-45d17a436e9f",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 3,
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Enter OpenAI api key for:··········\n",
            "Enter serperdev api key:··········\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 3. Build our RAG serper.dev pipeline\n",
        "\n"
      ],
      "metadata": {
        "id": "0N4yjfJHD8HU"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "pipe = Pipeline()\n",
        "pipe.add_component(\"serperdev\", OpenAPITool(generator_api=LLMProvider.OPENAI,\n",
        "                                            generator_api_params={\"model\":\"gpt-3.5-turbo\",\n",
        "                                                                  \"api_key\":Secret.from_token(llm_api_key)},\n",
        "                                            spec=\"https://bit.ly/serper_dev_spec_yaml\",\n",
        "                                            credentials=Secret.from_token(serper_dev_key)))\n",
        "\n",
        "pipe.add_component(\"final_prompt_adapter\", OutputAdapter(\"{{system_message + service_response}}\", List[ChatMessage]))\n",
        "pipe.add_component(\"llm\", OpenAIChatGenerator(api_key=Secret.from_token(llm_api_key),\n",
        "                                              generation_kwargs={\"max_tokens\": 1024},\n",
        "                                              streaming_callback=print_streaming_chunk))\n",
        "\n",
        "\n",
        "pipe.connect(\"serperdev\", \"final_prompt_adapter.service_response\")\n",
        "pipe.connect(\"final_prompt_adapter\", \"llm.messages\")"
      ],
      "metadata": {
        "id": "n7ErNg55_zJN",
        "outputId": "3f23681b-8597-47ba-e625-4fbfc216c789",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<haystack.core.pipeline.pipeline.Pipeline object at 0x79c23d72f880>\n",
              "🚅 Components\n",
              "  - serperdev: OpenAPITool\n",
              "  - final_prompt_adapter: OutputAdapter\n",
              "  - llm: OpenAIChatGenerator\n",
              "🛤️ Connections\n",
              "  - serperdev.service_response -> final_prompt_adapter.service_response (List[ChatMessage])\n",
              "  - final_prompt_adapter.output -> llm.messages (List[ChatMessage])"
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "As you can see in the pipeline graph above, for a given serper.dev query, our pipeline follows these steps:\n",
        "\n",
        "1. **Fetch OpenAPI Spec**: Retrieve the OpenAPI specification for serper.dev and convert it into OpenAI function-calling definitions.\n",
        "\n",
        "2. **Determine Parameters**: Use a function-calling model to identify the necessary parameters for the serper.dev service.\n",
        "\n",
        "3. **Query serper.dev**: Dispatch the request to serper.dev with the determined parameters and gather the responses.\n",
        "\n",
        "4. **Compile Results**: Organize and format the search results from serper.dev.\n",
        "\n",
        "5. **Generate Response**: Combine the system prompt, user query, and compiled search results, then pass them to LLM to generate the final response."
      ],
      "metadata": {
        "id": "1rawAmKSELBA"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "system_prompt = requests.get(\"https://bit.ly/serper_dev_system\").text"
      ],
      "metadata": {
        "id": "msBEddD4Azfh"
      },
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "query = \"Who won mens Australian Open in 2024?\"\n",
        "\n",
        "result = pipe.run(data={\"serperdev\": {\"messages\": [ChatMessage.from_user(query)]},\n",
        "                        \"final_prompt_adapter\": {\"system_message\": [ChatMessage.from_system(system_prompt)]}})\n"
      ],
      "metadata": {
        "id": "OVcVXOSRAfrm",
        "outputId": "86ffd109-17f3-4ade-cad1-24de07def9f7",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Jannik Sinner won the men's singles title at the 2024 Australian Open by defeating Daniil Medvedev in the final."
          ]
        }
      ]
    }
  ]
}