boilerplate/03_simple_search.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Notebook 3: Simple Search\n",
    "\n",
    "In this notebook you learn how to compare text using semantic search."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Optional: Install the client\n",
    "You can skip this step, if you have already installed the `aleph_alpha_client`. Make sure you have the [latest pip version](https://pip.pypa.io/en/stable/installation/) installed before proceeding. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install aleph-alpha-client"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Instantiate the client\n",
    "To interact with our API, you have to instantiate a client. Here you also provide your token to authenticate yourself. If you don't have one already, create one in your [Aleph Alpha profile](https://app.aleph-alpha.com/profile)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from aleph_alpha_client import Client\n",
    "client = Client(token=\"AA_TOKEN\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Compare the similarity of two texts\n",
    "\n",
    "To compare two texts, you need to embed both and calculate the [cosine similarity](https://en.wikipedia.org/wiki/Cosine_similarity) between them. To calculate the cosine similarity between the two texts, we provide a helper function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from aleph_alpha_client import SemanticEmbeddingRequest, SemanticRepresentation, Prompt\n",
    "from typing import Sequence\n",
    "import math\n",
    "\n",
    "# helper function to calculate similarity\n",
    "def cosine_similarity(v1: Sequence[float], v2: Sequence[float]) -> float:\n",
    "    \"compute cosine similarity of v1 to v2: (v1 dot v2)/{||v1||*||v2||)\"\n",
    "    sumxx, sumxy, sumyy = 0, 0, 0\n",
    "    for i in range(len(v1)):\n",
    "        x = v1[i]; y = v2[i]\n",
    "        sumxx += x*x\n",
    "        sumyy += y*y\n",
    "        sumxy += x*y\n",
    "    return sumxy/math.sqrt(sumxx*sumyy)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using semantic embeddings\n",
    "Now we can use the semantic similarity to find similar texts.\n",
    "\n",
    "Let's compare two texts, one in English and one in Italian to see how it works.\n",
    "\n",
    "In this case, both sentences have the same meaning. So they should return a high similarity score."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# define the texts\n",
    "texts = [\"The sun is shining\", \"Il sole splende\"]\n",
    "\n",
    "symmetric_embeddings = []\n",
    "\n",
    "for text in texts:\n",
    "    symmetric_params = {\n",
    "        \"prompt\": Prompt.from_text(text),\n",
    "        \"representation\": SemanticRepresentation.Symmetric,\n",
    "        \"compress_to_size\": 128\n",
    "    }\n",
    "    symmetric_request = SemanticEmbeddingRequest(**symmetric_params)\n",
    "    symmetric_response = client.semantic_embed(request=symmetric_request, model=\"luminous-base\")\n",
    "    symmetric_embeddings.append(symmetric_response.embedding)\n",
    "\n",
    "# show the similarity\n",
    "print(cosine_similarity(symmetric_embeddings[0], symmetric_embeddings[1]))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The embeddings\n",
    "Let's also take a look at the embeddings themselves and what they actuall look like.\n",
    "\n",
    "In the cell below we print the first 100 elements of the embedding.\n",
    "\n",
    "The default behavior is to return the full embedding with 5120 dimensions (like in this case). You can also compress the returned embeddings to 128 dimensions. The compression is expected to result in a small drop in accuracy performance (4-6%), with the benefit of being much smaller, which makes comparing these embeddings much faster for use cases where speed is critical."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "print(symmetric_embeddings[0][:100], \"\\n\")\n",
    "print(symmetric_embeddings[1][:100])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Multimodal Similarity\n",
    "Amazingly, as Luminous supports multimodal input, we can even semantically compare texts to images with Luminous Explore.\n",
    "\n",
    "This is not an explicitly developed feature but rather an emergent property of the model.\n",
    "\n",
    "*Please keep in mind, that multi-modal semantic similarity is probably less robust than text to text similarity.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from aleph_alpha_client import Image, Text\n",
    "from IPython.display import Image as ImageShow\n",
    "\n",
    "url = \"https://cdn-images-1.medium.com/max/1200/1*HunNdlTmoPj8EKpl-jqvBA.png\"\n",
    "image_prompt = Image.from_image_source(url)\n",
    "positive_text = Text.from_text(\"A neural network Architecture with Attention and Embeddings\")\n",
    "negative_text = Text.from_text(\"An image of a beatuiful beach\")\n",
    "symmetric_embeddings = []\n",
    "\n",
    "items = [image_prompt, positive_text, negative_text]\n",
    "\n",
    "for item in items:\n",
    "    symmetric_params = {\n",
    "        \"prompt\": Prompt([item]),\n",
    "        \"representation\": SemanticRepresentation.Symmetric,\n",
    "        \"compress_to_size\": 128\n",
    "    }\n",
    "    symmetric_request = SemanticEmbeddingRequest(**symmetric_params)\n",
    "    symmetric_response = client.semantic_embed(request=symmetric_request, model=\"luminous-base\")\n",
    "    symmetric_embeddings.append(symmetric_response.embedding)\n",
    "\n",
    "# show the similarity\n",
    "positive_embedding = cosine_similarity(symmetric_embeddings[0], symmetric_embeddings[1])\n",
    "negative_embedding = cosine_similarity(symmetric_embeddings[0], symmetric_embeddings[2])\n",
    "\n",
    "# print the Image and calculated embeddings\n",
    "display(ImageShow(url=url, width=200, height=300))\n",
    "print(f\"The score for the positive example is: {positive_embedding}\")\n",
    "print(f\"The score for the negative example is: {negative_embedding}\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "explaindemo",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.9"
  },
  "vscode": {
   "interpreter": {
    "hash": "6942b235de7c397011ac8abd8b15a4907297daf4b25575053729feb002569882"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}