working_with_Aleph_Alpha.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<image src=\"https://camo.githubusercontent.com/14db3cb92f9b9ed943e33ae9a3a9ebe2ae35c9595393ff14e9595be6f8fb140e/68747470733a2f2f692e696d6775722e636f6d2f46534d324e4e562e706e67\" width=\"200\" align=\"center\">\n",
    "\n",
    "\n",
    "# Working with Aleph Alpha Technology\n",
    "Hi, great to see you working with Aleph Alpha Technology. \n",
    "\n",
    "This notebook will support you on your journey by providing you with some examples on how to use our API and how to solve tasks with it.\n",
    "We will be using the Aleph Alpha API to solve tasks all kinds of tasks and learn how different components and functionalities of our LLMs can be used to solve them.\n",
    "\n",
    "If you have any questions or feedback, please feel free to reach out to us at [Aleph Alpha support](mailto:support@aleph-alpha.com).\n",
    "\n",
    "Good luck and have fun!\n",
    "\n",
    "\n",
    "----------------------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prerequisites:\n",
    "- You have an Aleph Alpha account and API key (you can sign up here: https://app.aleph-alpha.com/)\n",
    "- You have glanced over our documentation (https://docs.aleph-alpha.com/)\n",
    "- YOu have played around with our playground (https://app.aleph-alpha.com/playground)\n",
    "\n",
    "\n",
    "## Content\n",
    "\n",
    "This notebook will contain information on the following topics:\n",
    "1. Use our LLMs to generate text and solve tasks\n",
    "2. Using embeddings to find similar relevant information\n",
    "3. Use semantic embeddings and completion to answer questions\n",
    "4. Chaining multiple requests to solve complex tasks\n",
    "5. Using Atman to explain your results\n",
    "\n",
    "Let's get started!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Install dependencies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture\n",
    "# Setting up the workspace on colab\n",
    "!git clone https://github.com/Aleph-Alpha/bootcamp.git\n",
    "!pip install -r bootcamp/requirements.txt\n",
    "!cp bootcamp/data.md data.md\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### These are just some imports to start working with our API\n",
    "If you are interested, here is what the individual imports do:\n",
    "\n",
    "| Import | Description |\n",
    "| --- | --- |\n",
    "| ``Client`` | This is the main class that you will use to authenticate with the API. |\n",
    "| ``Prompt`` | We use this class to format information correctly for our models |\n",
    "| ``CompletionRequest`` | CompletionRequests are used to reuqest our models to generate text, e.g. for solving tasks |\n",
    "| ``SemanticEmbeddingRequest`` | SemanticEmbeddingRequests are used to request our models to generate embeddings for text, e.g. for searching for information or for classification |\n",
    "| ``ExplanationRequest`` | ExplanationRequests are used to request our models to generate explanations for text, e.g. for explaining a where an answer comes from |\n",
    "| ``TextControl`` | TextControl allows us to manipulate the attention of our models, e.g. to focus on certain parts of the input |\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from aleph_alpha_client import Client, Prompt, CompletionRequest, CompletionResponse, SemanticEmbeddingRequest, SemanticEmbeddingResponse, SemanticRepresentation, ExplanationRequest, TextControl\n",
    "from scipy import spatial\n",
    "import numpy as np\n",
    "import json\n",
    "import os\n",
    "import gradio as gr\n",
    "from tqdm import tqdm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 0: Using the client to authenticate with the API\n",
    "First, we need to authenticate with the API. To do this, we need to create a ``Client`` object and pass it our API key. You can create your API key in your [account settings](https://app.aleph-alpha.com/profile).\n",
    "\n",
    "If you want to use the local API, you need to also pass the ``host`` parameter to the client."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Authenticate with the API by using the client class\n",
    "client = Client(token=\"<YOUR_TOKEN>\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Step 1: Using LLMs to generate text and solve tasks\n",
    "In this section, we will use our LLMs to generate text and solve tasks.\n",
    "\n",
    "We will use the same LLM for both tasks. This is because our LLMs are trained to solve many different tasks. This means that we can use the same LLM for many different tasks.\n",
    "\n",
    "<image src=\"https://github.com/Aleph-Alpha/bootcamp/blob/main/img/functionalities.png?raw=true\" width=\"800\" align=\"center\" alt=\"Aleph Alpha Functionalities\">\n",
    "\n",
    "\n",
    "We will use the completion endpoint to generate text and solve tasks. You can find more information about this endpoint in the [Completion Documentation](https://docs.aleph-alpha.com/docs/tasks/complete/).\n",
    "\n",
    "With completions we prompt the model to generate text. Depending on the prompt, the model will generate different text. This is a very powerful universal tool to generate text and solve tasks.\n",
    "\n",
    "However, to get the best results, we need to formulate our prompts correctly. We need to keep in mind the structure that the model expects and also how to word our requests so that the model understands what we want.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.1 Generating text\n",
    "First, let's just start with generating text. While our API offers different models, we will start with our ``Control-models``. These models are specifically optimized to solve tasks that you give them.\n",
    "\n",
    "We will stick to the structure that these models expect. This is a good starting point to get familiar with the API.\n",
    "\n",
    "```markdown\n",
    "### Instruction:\n",
    "INPUT YOUR INSTRUCTION HERE\n",
    "\n",
    "### Input:\n",
    "YOUR INPUT (Optional)\n",
    "\n",
    "### Response:\n",
    "```\n",
    "\n",
    "Try to vary the input and see how the model responds. You can also try to change the instruction and see how the model responds."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Write a prompt, so that the model knows what to do\n",
    "prompt_text = \"\"\"### Instruction: \n",
    "What options are there for resetting a password?\n",
    "\n",
    "### Response:\"\"\"\n",
    "\n",
    "# Create the completion request\n",
    "request = CompletionRequest(\n",
    "    prompt=Prompt.from_text(prompt_text), \n",
    "    maximum_tokens=20, # Parameter to control the maximum length of the completion\n",
    "    temperature=0.0, # Parameter to control the randomness of the completion\n",
    "    stop_sequences=[\"\\n\"]) # Parameter to control the stopping criteria of the completion\n",
    "\n",
    "# Send the prompt to the API\n",
    "response = client.complete(request=request, model=\"luminous-base-control\")\n",
    "\n",
    "response_text = response.completions[0].completion\n",
    "\n",
    "# Print the response\n",
    "print(f\"The model returned: `{response_text}`\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Great, so now we have generated some text. \n",
    "\n",
    "However, as you can see this information is coming from the model's foundatinal knowledge. This is not very useful in many cases.\n",
    "\n",
    "Let's try again, but this time, we provide some background information. This will help the model to generate more useful information.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Write a prompt, so that the model knows what to do\n",
    "prompt_text = \"\"\"### Instruction: \n",
    "What options are there for resetting a password?\n",
    "\n",
    "### Input:\n",
    "If a customer forgot their password, they should be able to reset it by clicking on the \"Forgot Password\" link on the login page.\n",
    "They can use different methods to reset their password, such as email, SMS, or security questions.\n",
    "\n",
    "### Response:\"\"\"\n",
    "\n",
    "# Create the completion request\n",
    "request = CompletionRequest(\n",
    "    prompt=Prompt.from_text(prompt_text), \n",
    "    maximum_tokens=20, # Parameter to control the maximum length of the completion\n",
    "    temperature=0.0, # Parameter to control the randomness of the completion\n",
    "    stop_sequences=[\"\\n\"]) # Parameter to control the stopping criteria of the completion\n",
    "\n",
    "# Send the prompt to the API\n",
    "response = client.complete(request=request, model=\"luminous-base-control\")\n",
    "\n",
    "response_text = response.completions[0].completion\n",
    "\n",
    "# Print the response\n",
    "print(f\"The model returned: `{response_text}`\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.2 Solving specific tasks\n",
    "Now that we have seen how to generate text, let's try to solve a specific task. We will use the same model as before, but we will give it a different instruction.\n",
    "\n",
    "This time, we want to create a product text for a new product. We will give the model a short description of the product and ask it to generate a product text.\n",
    "\n",
    "We will be using both the ``Control-models`` as well as the ``foundation-models``. The ``foundation-models`` are trained on a large amount of data and are able to generate text that is more fluent and coherent. However, they are not as good at solving specific tasks as the ``Control-models``.\n",
    "\n",
    "While control models work with a specific structure, the foundation models are more flexible. This means that we can use them to generate text in a more natural way. However, they require a ``few-shot`` prompt. This means that we need to give them a few examples of what we want them to do."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Here is a control model prompt for you to try out. In this case, the model is asked to generate a product description for a yoga mat.\n",
    "control_prompt_text = \"\"\"### Instruction:\n",
    "Generate a product description for the following product.\n",
    "Only use information from the product description.\n",
    "\n",
    "### Input:\n",
    "Name: Multifunctional Yoga Mat\n",
    "Color: Blue\n",
    "Material: Rubber\n",
    "Size: 180 x 60 x 0.5 cm\n",
    "Uses: Yoga, Pilates, Fitness, Gymnastics, Camping, Picnic, Sleep, Play, etc.\n",
    "\n",
    "### Response:\"\"\"\n",
    "\n",
    "# Let's send the prompt to the API and see what the model returns\n",
    "request = CompletionRequest(\n",
    "    prompt=Prompt.from_text(control_prompt_text),\n",
    "    maximum_tokens=100,\n",
    "    temperature=0.0,\n",
    "    stop_sequences=[])\n",
    "\n",
    "response = client.complete(request=request, model=\"luminous-base-control\")\n",
    "\n",
    "response_text = response.completions[0].completion\n",
    "\n",
    "print(f\"The model returned: `{response_text}`\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "While this is more flexible, it also means that we need to be more careful with how we formulate our requests. We need to make sure that we give the model enough information to understand what we want it to do. \n",
    "\n",
    "With few-shot we can more easily steer the model to generate the text that we want. However, we need to be careful to not give the model useful examples, so that we do not bias it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This is how we would write the prompt as a few-shot learning prompt\n",
    "few_shot_prompt_text = \"\"\"Task: Generate a product description for the following product.\n",
    "Only use information from the product description.\n",
    "###\n",
    "Product:\n",
    "- Name: Ergonomic Office Chair\n",
    "- Color: Black\n",
    "- Material: Plastic, Metal, Fabric\n",
    "- Functions: Height adjustable, 360 degree swivel, seat tilt, back tilt\n",
    "- Uses: Office, Home, Gaming, etc.\n",
    "Description: This ergonomic office chair is made of high-quality materials, such as plastic, metal, and fabric and is very comfortable to sit on. \n",
    "It is height adjustable, can swivel 360 degrees, and has a seat and back tilt. \n",
    "You can use it in the office, at home, or for gaming.\n",
    "###\n",
    "Product:\n",
    "- Name: Multifunctional Yoga Mat\n",
    "- Color: Blue\n",
    "- Material: Rubber\n",
    "- Size: 180 x 60 x 0.5 cm\n",
    "- Uses: Yoga, Pilates, Fitness, Gymnastics, Camping, Picnic, Sleep, Play, etc.\n",
    "Description:\"\"\"\n",
    "\n",
    "# Let's send the prompt to the API and see what the model returns\n",
    "request = CompletionRequest(\n",
    "    prompt=Prompt.from_text(few_shot_prompt_text),\n",
    "    maximum_tokens=100,\n",
    "    temperature=0.5, # We can use a higher temperature to make the model more creative\n",
    "    stop_sequences=[\"###\"] # with the foundation models we need to specify the stop sequence\n",
    "    )\n",
    "\n",
    "response = client.complete(request=request, model=\"luminous-extended\")\n",
    "\n",
    "\n",
    "response_text = response.completions[0].completion\n",
    "\n",
    "print(f\"The model returned: `{response_text}`\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.3 Experiment with completions LLMs yourself\n",
    "\n",
    "Now you can go ahead and experiment with completions yourself. \n",
    "\n",
    "Try to solve different tasks with the LLMs. \n",
    "\n",
    "Experiment with ``Control-models`` and ``foundation-models``. \n",
    "See how they differ in their responses and how they solve tasks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO Change the prompt to be solve a different task\n",
    "control_prompt_text = \"\"\"Try to write your own prompt here.\"\"\"\n",
    "\n",
    "# Send the prompt to the API and see what the model returns\n",
    "request = CompletionRequest(    \n",
    "    prompt=Prompt.from_text(control_prompt_text),\n",
    "    maximum_tokens=100,\n",
    "    temperature=0.0,\n",
    "    stop_sequences=[])\n",
    "\n",
    "response = client.complete(request=request, model=\"luminous-base-control\")\n",
    "response_text = response.completions[0].completion\n",
    "\n",
    "print(f\"The model returned: `{response_text}`\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 1.4: Smalltalk and QA prompt\n",
    "We can also use the LLMs to chat with them. This is a fun way to get to know the LLMs and to see how they work.\n",
    "\n",
    "Write two functions, one for small talk and one for a more specific conversation.\n",
    "The small talk function should be able to chat about different topics, while the specific conversation should be able to answer specific questions based on a given context."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Smalltalk function\n",
    "def smalltalk(message):  \n",
    "      \n",
    "    prompt = f\"\"\"### Instruction:\n",
    "You are a friendly chatbot called AlphaBot and developed by Aleph Alpha.\n",
    "Have a nice and friendly conversation with the user.\n",
    "\n",
    "### Input:\n",
    "User: {message}\n",
    "\n",
    "### Response:\n",
    "AlphaBot:\"\"\"\n",
    "    \n",
    "    request = CompletionRequest(\n",
    "        prompt=Prompt.from_text(prompt),\n",
    "        maximum_tokens=128,\n",
    "        temperature=0.3,\n",
    "        stop_sequences=[\"\\n\", \"User\", \"AlphaBot\"])\n",
    "\n",
    "    response = client.complete(request=request, model=\"luminous-extended-control\")\n",
    "    response_text = response.completions[0].completion\n",
    "\n",
    "    return response_text\n",
    "\n",
    "# QA function\n",
    "def qa_answer(message, context):\n",
    "    \n",
    "    prompt = f\"\"\"### Instruction:\n",
    "{message}\n",
    "\n",
    "### Input:\n",
    "{context}\n",
    "\n",
    "### Response:\"\"\"\n",
    "\n",
    "\n",
    "    request = CompletionRequest(\n",
    "        prompt=Prompt.from_text(prompt),\n",
    "        maximum_tokens=128,\n",
    "        temperature=0.0,\n",
    "        stop_sequences=[])\n",
    "\n",
    "    response = client.complete(request=request, model=\"luminous-extended-control\")\n",
    "    response_text = response.completions[0].completion\n",
    "\n",
    "    return response_text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's test the smalltalk and QA functions\n",
    "\n",
    "# Smalltalk\n",
    "message = \"Hello\"\n",
    "response = smalltalk(message=message)\n",
    "\n",
    "print(f\"Testing smalltalk function\")\n",
    "print(f\"User: {message}\")\n",
    "print(f\"AlphaBot: {response}\\n\\n\")\n",
    "\n",
    "# QA\n",
    "message = \"What does Aleph Alpha develop?\"\n",
    "response = qa_answer(message=message, context=\"Aleph Alpha is a research and development company that researches and develops artificial intelligence technologies.\")\n",
    "\n",
    "print(f\"Testing QA function\")\n",
    "print(f\"User: {message}\")\n",
    "print(f\"AlphaBot: {response}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "--------------------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 2: Using Embeddings to search for information\n",
    "In many cases, the relevant information to solve a task may not be available or known to the model.\n",
    "\n",
    "With Semantic Search, we can use the embeddings to search for relevant information in a corpus of documents. The idea is that LLMs are able to understand the meaning of a question and the meaning of a document, and thus, can find the most relevant document to answer the question.\n",
    "\n",
    "We do this by first encoding the question and the documents into embeddings. Then, we compute the similarity between the question embedding and the document embeddings. Finally, we return the document with the highest similarity score.\n",
    "\n",
    "<div>\n",
    "<img src=\"https://docs.aleph-alpha.com/assets/images/symmetric_embedding-fdb53a9755c451641d70d08b8f58db8b.png\" width=\"500\"/>\n",
    "</div>\n",
    "\n",
    "<div>\n",
    "<img src=\"https://docs.aleph-alpha.com/assets/images/asymmetric_embedding-6cac7874ae7db8b2cd796bfd2d2f1bcb.png\" width=\"500\"/>\n",
    "</div>\n",
    "\n",
    "You can find more information about this technique in the [Semantic Embedding Documentation](https://docs.aleph-alpha.com/docs/tasks/semantic_embed/).\n",
    "\n",
    "Let's see how this works in practice."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1 Creating embeddings for text.\n",
    "In Order to find the correct documents, we need to turn our text into numbers.\n",
    "We do that with semnatic embeddings. These are vectors that represent the meaning of the data.\n",
    "\n",
    "Let's use Aleph Alpha technology to create embeddings for our text."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Two texts and a question to be embedded and searched for\n",
    "text_1 = \"\"\"With our semantic_embed-endpoint you can create semantic embeddings for your text. \n",
    "This functionality can be used in a myriad of ways. \n",
    "For more information please check out our blog-post on Luminous-Explore, introducing the model behind the semantic_embed-endpoint. \n",
    "In order to effectively search through your own documents, it is important to ensure that they can be easily compared to each other. \n",
    "Our asymmetric embeddings are designed to help find the pieces of your documents that are most relevant to a query shorter than the documents in the database. \n",
    "Here we will use short queries and longer splits of law texts.\"\"\"\n",
    "\n",
    "text_2 = \"\"\"You can interact with a Luminous model by sending it a text. We call this a prompt. \n",
    "It will then produce text that continues your input and return it to you. This is what we call a completion. \n",
    "Generally speaking, our models attempt to find the best continuation for a given input. \n",
    "Practically, this means that the model first recognizes the style of the prompt and then attempts to continue it accordingly.\"\"\"\n",
    "\n",
    "question = \"How can I search through my documents with embeddings?\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# using the API to embed the text\n",
    "\n",
    "# We embed the texts as Documents, as the contain a lot of information\n",
    "request_1 = SemanticEmbeddingRequest(prompt=Prompt.from_text(text_1), representation=SemanticRepresentation.Document)\n",
    "request_2 = SemanticEmbeddingRequest(prompt=Prompt.from_text(text_2), representation=SemanticRepresentation.Document)\n",
    "\n",
    "# We embed the question as a Query, as it is a short text\n",
    "request_question = SemanticEmbeddingRequest(prompt=Prompt.from_text(question), representation=SemanticRepresentation.Query)\n",
    "\n",
    "# We send the requests to the API\n",
    "embedding_1 = client.semantic_embed(request_1, model=\"luminous-base\").embedding\n",
    "embedding_2 = client.semantic_embed(request_2, model=\"luminous-base\").embedding\n",
    "embedding_question = client.semantic_embed(request_question, model=\"luminous-base\").embedding"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 Calculating the similarity between embeddings\n",
    "Now that we have embeddings for our question and our documents, we can calculate the similarity between them.\n",
    "For that we use the cosine similarity. This is a measure of how similar two vectors are. The higher the value, the more similar the vectors are."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We calculate the cosine similarity between the question and the texts\n",
    "similarity_1 = 1 - spatial.distance.cosine(embedding_1, embedding_question)\n",
    "similarity_2 = 1 - spatial.distance.cosine(embedding_2, embedding_question)\n",
    "\n",
    "# We print the results\n",
    "print(\"The similarity between the question and text 1 is: \" + str(similarity_1))\n",
    "print(\"The similarity between the question and text 2 is: \" + str(similarity_2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that the document with the highest similarity score is the one that we are looking for.\n",
    "This semantic search is a very powerful tool to find relevant information."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3 Experiment with embeddings yourself\n",
    "Now you can go ahead and experiment with embeddings yourself. \n",
    "\n",
    "When do they work well? \n",
    "\n",
    "When do they not work well?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO Change the text to be embedded and searched for\n",
    "test_text = \"A large language model (LLM) is a language model characterized by its large size. Their size is enabled by AI accelerators, which are able to process vast amounts of text data, mostly scraped from the Internet.[1] The artificial neural networks which are built can contain from tens of millions and up to billions of weights and are (pre-)trained using self-supervised learning and semi-supervised learning. Transformer architecture contributed to faster training.\"\n",
    "\n",
    "# TODO Change the question to be embedded and searched for\n",
    "test_question = \"What is a large language model\"\n",
    "\n",
    "# run the code to embed the text and question and calculate the similarity\n",
    "request_test_text = SemanticEmbeddingRequest(prompt=Prompt.from_text(test_text), representation=SemanticRepresentation.Document)\n",
    "request_test_question = SemanticEmbeddingRequest(prompt=Prompt.from_text(test_question), representation=SemanticRepresentation.Query)\n",
    "embedding_test_text = client.semantic_embed(request_test_text, model=\"luminous-base\").embedding\n",
    "embedding_test_question = client.semantic_embed(request_test_question, model=\"luminous-base\").embedding\n",
    "similarity_test = 1 - spatial.distance.cosine(embedding_test_text, embedding_test_question)\n",
    "print(\"The similarity between the question and text 1 is: \" + str(similarity_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 2.4 Embedding functions\n",
    "Now your task is to write two fucntions, to ease the use of embeddings in your code.\n",
    "\n",
    "Use 128 size embeddings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def embed_query(query):\n",
    "    embedding = None\n",
    "    request = SemanticEmbeddingRequest(prompt=Prompt.from_text(query), representation=SemanticRepresentation.Query, compress_to_size=128)\n",
    "    embedding = client.semantic_embed(request, model=\"luminous-base\").embedding\n",
    "\n",
    "    return embedding\n",
    "    \n",
    "def embed_document(document):\n",
    "    embedding = None\n",
    "    request = SemanticEmbeddingRequest(prompt=Prompt.from_text(document), representation=SemanticRepresentation.Document, compress_to_size=128)\n",
    "    embedding = client.semantic_embed(request, model=\"luminous-base\").embedding\n",
    "    \n",
    "    return embedding\n",
    "\n",
    "def embed_symmetric(text):\n",
    "    embedding = None\n",
    "    request = SemanticEmbeddingRequest(prompt=Prompt.from_text(text), representation=SemanticRepresentation.Symmetric, compress_to_size=128)\n",
    "    embedding = client.semantic_embed(request, model=\"luminous-base\").embedding\n",
    "    \n",
    "    return embedding\n",
    "\n",
    "embedding_query = embed_query(\"What is the best way to reset a password?\")\n",
    "embedding_document = embed_document(\"If a customer forgot their password, they should be able to reset it by clicking on the \\\"Forgot Password\\\" link on the login page. They can use different methods to reset their password, such as email, SMS, or security questions.\")\n",
    "\n",
    "similarity = 1 - spatial.distance.cosine(embedding_query, embedding_document)\n",
    "print(\"The similarity between the query and the document is: \" + str(similarity))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "--------------------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 3: Using Semantic Embeddings and Completions together to answer questions\n",
    "In this section, we will use the search and completions endpoints together to answer questions.\n",
    "\n",
    "With `semantic search`, we can find relevant information. With `completions`, we can generate text and solve tasks.\n",
    "\n",
    "\n",
    "<image src=\"https://github.com/Aleph-Alpha/bootcamp/blob/main/img/search_strategy.png?raw=true\" width=\"800\" align=\"center\" alt=\"Aleph Alpha Searching Strategy\">\n",
    "\n",
    "\n",
    "Our application logic is as follows:\n",
    "1. We use `semantic search` to make information searchable.\n",
    "2. We select the most similar document as background information.\n",
    "3. We use `completions` to generate the answer to the question.\n",
    "\n",
    "Let's use the data from the data.md file to answer some questions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's use the data saved in the data.md file\n",
    "\n",
    "with open(\"data.md\", \"r\") as f:\n",
    "    data = f.read().split(\"###\")[1:]\n",
    "\n",
    "\n",
    "# print the first 3 entries\n",
    "for i in range(4):\n",
    "    print(data[i][:100] + \"...\\n\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "question = \"How can I search through my documents with embeddings?\"\n",
    "\n",
    "\n",
    "# Creatting the embedding for the texts and the question\n",
    "request_1 = SemanticEmbeddingRequest(prompt=Prompt.from_text(text_1), representation=SemanticRepresentation.Document)\n",
    "request_2 = SemanticEmbeddingRequest(prompt=Prompt.from_text(text_2), representation=SemanticRepresentation.Document)\n",
    "request_question = SemanticEmbeddingRequest(prompt=Prompt.from_text(question), representation=SemanticRepresentation.Query)\n",
    "embedding_1 = client.semantic_embed(request_1, model=\"luminous-base\").embedding\n",
    "embedding_2 = client.semantic_embed(request_2, model=\"luminous-base\").embedding\n",
    "embedding_question = client.semantic_embed(request_question, model=\"luminous-base\").embedding"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 3.1: Using a vectordatabase to store embeddings\n",
    "Instead of doin g everything ourselves, we can use a Vectordatabase to store the embeddings for us. This makes it easier to search for information.\n",
    "\n",
    "We will be using qdrant as our vectordatabase. \n",
    "Qdrant is an open-source vectordatabase that is easy to use and fast.\n",
    "You can find more information about qdrant [here](https://qdrant.tech/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# First we spin up the Qdrant server\n",
    "from qdrant_client import QdrantClient\n",
    "from qdrant_client.http.models import Distance, VectorParams, Batch\n",
    "\n",
    "q_client = QdrantClient(path=\"db\")\n",
    "\n",
    "q_client.recreate_collection(\n",
    "    collection_name=\"test_collection\",\n",
    "    vectors_config=VectorParams(size=128, distance=Distance.COSINE),\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we need to store the documents in the vectordatabase"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's create embeddings for each of the texts and store them in a list\n",
    "texts = data\n",
    "embeddings = []\n",
    "for text in tqdm(texts):\n",
    "    # embed the texts\n",
    "    embeddings.append(client.semantic_embed(SemanticEmbeddingRequest(prompt=Prompt.from_text(text), representation=SemanticRepresentation.Document, compress_to_size=128), model=\"luminous-base\").embedding)\n",
    "    \n",
    "    \n",
    "# now we can upsert the data into Qdrant\n",
    "ids = list(range(len(texts)))\n",
    "payloads = [{\"text\": text} for text in texts]\n",
    "\n",
    "q_client.upsert(\n",
    "     collection_name=\"test_collection\",\n",
    "     points=Batch(\n",
    "     ids=ids,\n",
    "     payloads=payloads,\n",
    "     vectors=embeddings\n",
    "     )\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 3.2 Using semantic search to find relevant information\n",
    "\n",
    "Now that we have stored our documents in the vectordatabase, we can use semantic search to find relevant information.\n",
    "\n",
    "For that we just have to send the embeddings of our question to the vectordatabase and it will return the most similar documents."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# embedding the question\n",
    "\n",
    "# We embed the question as a Query, as it is a short text\n",
    "request_question = SemanticEmbeddingRequest(\n",
    "    prompt=Prompt.from_text(question), \n",
    "    representation=SemanticRepresentation.Query, \n",
    "    compress_to_size=128)\n",
    "\n",
    "embedding_question = client.semantic_embed(request_question, model=\"luminous-base\").embedding\n",
    "\n",
    "search_result = q_client.search(\n",
    "        collection_name=\"test_collection\",\n",
    "        query_vector=embedding_question,\n",
    "        limit=5, # Parameter to control the number of results\n",
    "    )\n",
    "\n",
    "for result in search_result:\n",
    "    print(f\"Score: {result.score}, Text: {result.payload['text']}\")\n",
    "    \n",
    "    \n",
    "# Let's select the first result to answer the question\n",
    "background_text = search_result[0].payload[\"text\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 3.3 Using completions to generate the answer\n",
    "\n",
    "Now that we have found the most relevant document, we can use completions to generate the answer.\n",
    "\n",
    "We will use the same model as before. However, this time we will give it a different instruction.\n",
    "\n",
    "TODO:\n",
    "- Add additional documents to the vectordatabase\n",
    "- Add additional questions\n",
    "- Try to modify the prompt to get better results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "qa_prompt = f\"\"\"### Instruction:\n",
    "{question}\n",
    "\n",
    "### Input:\n",
    "{background_text}\n",
    "\n",
    "### Response:\"\"\"\n",
    "\n",
    "# Let's send the prompt to the API and see what the model returns\n",
    "request = CompletionRequest(\n",
    "    prompt=Prompt.from_text(qa_prompt),\n",
    "    maximum_tokens=100,\n",
    "    temperature=0.0,\n",
    "    stop_sequences=[\"\\n\"])\n",
    "\n",
    "response = client.complete(request=request, model=\"luminous-supreme-control\")\n",
    "\n",
    "response_text = response.completions[0].completion\n",
    "\n",
    "print(f\"The model returned: `{response_text}`\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 3.4: a search function\n",
    "Now your task is to write a function that takes a question embeds it and searches for the most similar document.\n",
    "\n",
    "You can find the qdrant documentation here: \n",
    "https://github.com/qdrant/qdrant-client/blob/82ecb104ed919596d7753c1a31017dfada8d3e02/qdrant_client/qdrant_client.py#L229C6-L229C6 \n",
    "\n",
    "Use the embedding function from step 2.4."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def search(query, limit=3, threshold=0.4):\n",
    "    search_result = q_client.search(\n",
    "        collection_name=\"test_collection\",\n",
    "        query_vector=embed_query(query),\n",
    "        limit=limit, # Parameter to control the number of results\n",
    "        score_threshold=threshold # Parameter to control the score threshold\n",
    "    )\n",
    "    return search_result\n",
    "\n",
    "len(search(\"How can I search through my documents with embeddings?\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Step 4: Classifying text with LLMs\n",
    "\n",
    "In this section, we will use LLMs to classify text.\n",
    "For that we will utilize the semantic embeddings again. \n",
    "\n",
    "We will use the embeddings to classify the text. This means that we will use the embeddings to find the most similar class.\n",
    "\n",
    "<image src=\"https://github.com/Aleph-Alpha/bootcamp/blob/main/img/classify.png?raw=true\" width=\"800\" align=\"center\" alt=\"Aleph Alpha embeddings for classification\">\n",
    "\n",
    "\n",
    "While not specifially trained for classification, LLMs are able to classify text. This is because they are able to understand the meaning of text and thus, can find the most similar class."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Step 4.1: Creating embeddings for classes\n",
    "First, we need to create embeddings for our classes. \n",
    "\n",
    "We will use the same technique as before.\n",
    "However, for classification we want to use symmetric embeddings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO Define 6 different examples for smalltalk texts\n",
    "class_smalltalk = [\"\"\"Hey, how are you doing?\"\"\",\n",
    "\"\"\"I am doing great, how about you?\"\"\",\n",
    "\"\"\"Whats up?\"\"\",\n",
    "\"Who are you?\",\n",
    "\"What is your name?\",\n",
    "\"Tell me a joke about cats.\",\n",
    "\"Want do you think about costume parties?\",]\n",
    "\n",
    "# TODO Define 6 different examples for question texts\n",
    "class_question = [\"\"\"What is the best way to reset a password?\"\"\",\n",
    "\"\"\"How can I search through my documents with embeddings?\"\"\",\n",
    "\"What can I do with completion?\",\n",
    "\"How does AtMan work?\",\n",
    "\"What is he difference between symmetric and asymmetric embeddings?\",\n",
    "\"What is a completion?\",\n",
    "\"What are all the luminous model names ?\"]\n",
    "\n",
    "# create a list of embeddings for each class\n",
    "embeddings_smalltalk = [embed_symmetric(text) for text in class_smalltalk]\n",
    "embeddings_question = [embed_symmetric(text) for text in class_question]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Step 4.2: Classifying text\n",
    "Now that we have embeddings for our classes, we can use them to classify text.\n",
    "\n",
    "For that we just have to compute the similarity between the text embedding and the class embeddings. The class with the highest similarity score is the one that we are looking for."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define a new text to be classified\n",
    "test_text = \"Hey, how are you doing?\"\n",
    "\n",
    "# Embed the test text\n",
    "embedding_test_text = embed_symmetric(test_text)\n",
    "\n",
    "# Calculate the cosine similarity between the test text and the smalltalk embeddings\n",
    "similarities_smalltalk = [1 - spatial.distance.cosine(embedding_test_text, embedding) for embedding in embeddings_smalltalk]\n",
    "similarities_question = [1 - spatial.distance.cosine(embedding_test_text, embedding) for embedding in embeddings_question]\n",
    "\n",
    "# Print the results\n",
    "print(\"The average similarity between the test text and the smalltalk embeddings is: \" + str(np.mean(similarities_smalltalk)))\n",
    "print(\"The average similarity between the test text and the question embeddings is: \" + str(np.mean(similarities_question)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Step 4.3: Classifying text with a function\n",
    "Now your task is to write a function that takes a text embeds it and classifies it.\n",
    "\n",
    "Use the embedding function from step 2.4.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# function to classify a text\n",
    "def classify(text, class_names, classes):\n",
    "    \n",
    "    # embed the text\n",
    "    embedding_text = embed_symmetric(text)\n",
    "    \n",
    "    # calculate the cosine similarity between the text and each class\n",
    "    similarities = [np.mean([1 - spatial.distance.cosine(embedding_text, embedding) for embedding in class_]) for class_ in classes]\n",
    "    # return the class with the highest similarity\n",
    "    return class_names[np.argmax(similarities)]\n",
    "\n",
    "classify(\"Hey, how are you doing?\", [\"smalltalk\", \"question\"], [embeddings_smalltalk, embeddings_question])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### step 4.4: Visualizing the classification\n",
    "Here we will visualize the classification results.\n",
    "\n",
    "For this we will use plotly. Plotly is a powerful visualization library that allows us to create interactive plots."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# reducing the size of the embeddings to 2 dimensions for visualization\n",
    "from sklearn.decomposition import PCA\n",
    "\n",
    "pca = PCA(n_components=2)\n",
    "\n",
    "pca.fit(embeddings_smalltalk + embeddings_question)\n",
    "\n",
    "embeddings_smalltalk_2d = pca.transform(embeddings_smalltalk)\n",
    "embeddings_question_2d = pca.transform(embeddings_question)\n",
    "embedding_test_text_2d = pca.transform([embedding_test_text])\n",
    "\n",
    "# Plotting the embeddings\n",
    "import plotly.express as px\n",
    "\n",
    "# create a generic figure\n",
    "fig = px.scatter()\n",
    "\n",
    "# add the embeddings to the figure\n",
    "fig.add_scatter(x=embeddings_smalltalk_2d[:,0], y=embeddings_smalltalk_2d[:,1], mode=\"markers\", name=\"smalltalk\", text=class_smalltalk)\n",
    "fig.add_scatter(x=embeddings_question_2d[:,0], y=embeddings_question_2d[:,1], mode=\"markers\", name=\"question\", text=class_question)\n",
    "fig.add_scatter(x=embedding_test_text_2d[:,0], y=embedding_test_text_2d[:,1], mode=\"markers\", name=\"test text\", text=test_text)\n",
    "\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "--------------------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 5: AtMan: Understanding the model's decisions\n",
    "This section will show you how to use AtMan to understand the model's decisions.\n",
    "\n",
    "With our `explain`-endpoint you can get an explanation of the model's output. In more detail, we return how much the log-probabilites of the already generated completion would change if we supress indivdual parts (based on the granularity you chose) of a prompt. Please refer to this part of our documentation if you would like to know more about our explainability method in general.\n",
    "\n",
    "<image src=\"https://github.com/Aleph-Alpha/bootcamp/blob/main/img/explain.png?raw=true\" width=\"800\" align=\"center\" alt=\"Aleph Alpha explainability\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 5.1: Using AtMan to understand the model's decisions\n",
    "Now that we have seen how to generate text, we want to be able to understand the model's decisions.\n",
    "\n",
    "For that we will use AtMan. AtMan is our explainability tool that allows us to understand the model's decisions.\n",
    "\n",
    "By surpressing individual parts of the prompt, we can see how much the model's output changes. This allows us to understand which parts of the prompt are important for the model's decision."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "text = \"The quick brown fox jumps over the lazy dog.\\nThe color of the fox is\"\n",
    "# Here we define a TextControl that will be used to control the attention on the prompt.\n",
    "\n",
    "\n",
    "# Change the factor to 0.0 to see what happens.\n",
    "control = TextControl(start=10, length=5, factor=0.0)\n",
    "\n",
    "# Prompt without control\n",
    "prompt = Prompt.from_text(text)\n",
    "\n",
    "# Prompt with control\n",
    "prompt2 = Prompt.from_text(text, controls=[control])\n",
    "\n",
    "request = CompletionRequest(prompt=prompt, maximum_tokens=10, stop_sequences=[\".\"], log_probs=5)\n",
    "request2 = CompletionRequest(prompt=prompt2, maximum_tokens=10, stop_sequences=[\".\"], log_probs=5)\n",
    "result = client.complete(request = request, model=\"luminous-extended\")\n",
    "result2 = client.complete(request = request2, model=\"luminous-extended\")\n",
    "\n",
    "print(f\" The most probable completion 1 is : \", result.completions[0].completion)\n",
    "print(f\"The probability of brown in completion 1 is {result.completions[0].log_probs[0][' brown']}\")\n",
    "\n",
    "print(f\" The most probable completion 2 is : \", result2.completions[0].completion)\n",
    "print(f\"The probability of brown in completion 2 is {result2.completions[0].log_probs[0][' brown']}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 5.2: Using AtMan via ExplainRequest\n",
    "Now that we have seen how to use AtMan, let's see how we can use it via the API.\n",
    "\n",
    "For that we will use the `ExplanationRequest`. This request allows us to get an explanation for a completion.\n",
    "\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt_text = \"\"\"Instructions:\n",
    "What is the best way to reset a password?\n",
    "\n",
    "Input:\n",
    "If a customer forgot their password, they should be able to reset it by clicking on the \"Forgot Password\" link on the login page.\n",
    "They can use different methods to reset their password, such as email, SMS, or security questions.\n",
    "\n",
    "Response:\"\"\"\n",
    "\n",
    "params = {\n",
    "    \"prompt\": Prompt.from_text(prompt_text),\n",
    "    \"maximum_tokens\": 32,\n",
    "}\n",
    "request = CompletionRequest(**params)\n",
    "response = client.complete(request=request, model=\"luminous-supreme\")\n",
    "completion = response.completions[0].completion\n",
    "\n",
    "print(f\"The model returned: `{completion}`\\n\\n\")\n",
    "\n",
    "exp_req = ExplanationRequest(Prompt.from_text(prompt_text), completion, prompt_granularity=\"sentence\")\n",
    "response_explain = client.explain(exp_req, model=\"luminous-extended-control\")\n",
    "\n",
    "explanations = response_explain.explanations[0].items[0].scores\n",
    "\n",
    "for item in explanations:\n",
    "    start = item.start\n",
    "    end = item.start + item.length\n",
    "    print(f\"\"\"EXPLAINED TEXT: {prompt_text[start:end]}\n",
    "Score: {np.round(item.score, decimals=3)}\"\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see from the example. The explanation helps us locate the relevant information that Luminous used.\n",
    "Please keep in mind, that especially the control models will have a very high explainability on the instructions. This is because they are trained to solve specific tasks. This means that they will always use the same parts of the instructions to solve the task.\n",
    "\n",
    "This can be easily managed by only looking at the explainability of the input. This will give us a better understanding of what the model is doing.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 6: Chaining multiple requests to solve complex tasks\n",
    "Sometimes, we need to solve complex tasks. For that, we can chain multiple requests together.\n",
    "\n",
    "Similar to Humans, LLMs produce more robust results if they are able to solve a task in multiple steps. This is because they can focus on one task at a time and do not have to solve everything at once (end-to-end).\n",
    "\n",
    "While solving tasks end-to-end may be very convenient, it is not always the best solution. This is because the model may not be able to focus on the most important parts of the task. This can lead to worse results.\n",
    "\n",
    "It is also much more difficult to debug and understand what the model is doing. This is because the model is solving the task in one step and we cannot see what it is doing.\n",
    "\n",
    "In this example, we will combine the techniques that we have learned so far to solve a complex task."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 6.1: writing a functional chatbot\n",
    "\n",
    "Let's combine all the techniques that we have learned so far to write a functional chatbot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# function that enables chat\n",
    "\n",
    "def answer(message):\n",
    "    \n",
    "    # first, find out if the message is a question or smalltalk\n",
    "    classification = classify(message, [\"smalltalk\", \"question\"], [embeddings_smalltalk, embeddings_question])\n",
    "    \n",
    "    # if the message is smalltalk, use the smalltalk function\n",
    "    \n",
    "    if classification == \"smalltalk\":\n",
    "        print(\"running smalltalk\")\n",
    "        response = smalltalk(message)\n",
    "        \n",
    "    # if the message is a question, use the chat_answer function\n",
    "    elif classification == \"question\":\n",
    "        print(\"running qa_answer\")\n",
    "        \n",
    "        background_information = search(message, limit=1, threshold=0.0)\n",
    "        \n",
    "        if len(background_information) > 0:\n",
    "            background_information = background_information[0].payload[\"text\"]\n",
    "        \n",
    "            response = qa_answer(message, background_information)\n",
    "            \n",
    "            # check the response with Explain\n",
    "            exp_req = ExplanationRequest(Prompt.from_text(background_information), response, prompt_granularity=\"paragraph\")\n",
    "            \n",
    "            response_explain = client.explain(exp_req, model=\"luminous-extended-control\")\n",
    "            \n",
    "            explanations = response_explain[1][0].items[0][0]            \n",
    "            scores = []\n",
    "            for item in explanations:\n",
    "                scores.append(item.score)\n",
    "                \n",
    "            if max(scores) < 0.5:\n",
    "                response = \"Seem's like I'm hallucinating here.\"\n",
    "                            \n",
    "        else:\n",
    "            response = \"I'm sorry, but I could not find any information on that.\"\n",
    "            \n",
    "    # return the response\n",
    "    return response"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(answer(\"Create an instruction on how to shave a panda.\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 6.2: putting everything in a visual interface"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "# simple gradio interface\n",
    "def answer_bot(message):\n",
    "    return answer(message)\n",
    "\n",
    "gr.Interface(fn=answer_bot, inputs=\"textbox\", outputs=\"text\").launch()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "---------------\n",
    "## Conclusion\n",
    "In this tutorial, we have seen how to use our API to generate text, search for information, and solve tasks.\n",
    "\n",
    "We have also seen how to chain multiple requests together to solve complex tasks.\n",
    "\n",
    "We hope that this tutorial was helpful to you. If you have any questions, please do not hesitate ask us."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "templates",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}