Embeddings/Visualizing embeddings with ATLAS.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualizing Open AI Embeddings in Atlas"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Atlas enables you to:\n",
    "\n",
    "- Store, update and organize multi-million point datasets of unstructured text, images and embeddings.\n",
    "- Visually interact with embeddings of your data from a web browser.\n",
    "- Operate over unstructured data and embeddings with topic modeling, semantic duplicate clustering and semantic search.\n",
    "- Generate high dimensional and two-dimensional embeddings of your data.\n",
    "\n",
    "Use Atlas to:\n",
    "- Visualize, interact, collaborate and share large datasets of text and embeddings.\n",
    "- Collaboratively curate your unstructured datasets (clean, tag and label)\n",
    "- Build high-availability apps powered by semantic search\n",
    "- Understand and debug the latent space of your AI model trains\n",
    "- Read about how Atlas works or get started below!\n",
    "\n",
    "https://docs.nomic.ai/index.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "#!pip install nomic"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import datetime\n",
    "import numpy as np\n",
    "import nomic\n",
    "import pandas as pd\n",
    "\n",
    "from nomic import atlas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Today is 09-Nov-2023 10:42:26\n"
     ]
    }
   ],
   "source": [
    "print(f\"Today is {datetime.datetime.today().strftime('%d-%b-%Y %H:%M:%S')}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "nomic.login('7xDPkYXSYDc1_ErdTPIcoAR9RNd8YDlkS3nVNXcVoIMZ6') # This is a public demo account"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2023-11-09 10:42:27--  https://huggingface.co/spaces/ThirdEyeData/Semantic-Search/resolve/main/fine_food_reviews_with_embeddings_1k.csv\n",
      "Resolving huggingface.co (huggingface.co)... 52.85.242.8, 52.85.242.35, 52.85.242.84, ...\n",
      "Connecting to huggingface.co (huggingface.co)|52.85.242.8|:443... connected.\n",
      "HTTP request sent, awaiting response... 302 Found\n",
      "Location: https://cdn-lfs.huggingface.co/repos/56/ee/56eec609e6e674d4ddc29404618b1577b6380cfddc0941873c0c018dd15096e1/0acc913f3deda7b91fcfb73e86a8780d490a54e33f2d2b9b6343078c45f0501b?response-content-disposition=attachment%3B+filename*%3DUTF-8%27%27fine_food_reviews_with_embeddings_1k.csv%3B+filename%3D%22fine_food_reviews_with_embeddings_1k.csv%22%3B&response-content-type=text%2Fcsv&Expires=1699785748&Policy=eyJTdGF0ZW1lbnQiOlt7IkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTY5OTc4NTc0OH19LCJSZXNvdXJjZSI6Imh0dHBzOi8vY2RuLWxmcy5odWdnaW5nZmFjZS5jby9yZXBvcy81Ni9lZS81NmVlYzYwOWU2ZTY3NGQ0ZGRjMjk0MDQ2MThiMTU3N2I2MzgwY2ZkZGMwOTQxODczYzBjMDE4ZGQxNTA5NmUxLzBhY2M5MTNmM2RlZGE3YjkxZmNmYjczZTg2YTg3ODBkNDkwYTU0ZTMzZjJkMmI5YjYzNDMwNzhjNDVmMDUwMWI%7EcmVzcG9uc2UtY29udGVudC1kaXNwb3NpdGlvbj0qJnJlc3BvbnNlLWNvbnRlbnQtdHlwZT0qIn1dfQ__&Signature=lpPqXH7WAqUYfbO-8E9G0jqJhSxSUH7rER71Y4bdoLC6qmBjp7K7GrWRZvuGidhQuupuT1AbrtsO09TZIM3hK50y8o8Zt1-IUVZTKzJWxnaRsHc1XIQFdPBvTyA-%7Ek8eYzz8OmGW5wmb3dbIfpeuyZMK-RZv7LOmfJ9OZSYPFlCJRHeTycgL1IBi%7EcJxPTSjB9gmXDN3Wkta4FMHmIDvYkcwVLX40pkkSaRr92V3pOifgxtfI1HQu3IN-kI09YOdt2X5xzKxPau3kvMb15Su8q5USgolwCsRNeQ5sBSo3lh1fgnZPTO-lKVRG4gM-ysD6NTe5Htfp2D0WfWThyIMLQ__&Key-Pair-Id=KVTP0A1DKRTAX [following]\n",
      "--2023-11-09 10:42:28--  https://cdn-lfs.huggingface.co/repos/56/ee/56eec609e6e674d4ddc29404618b1577b6380cfddc0941873c0c018dd15096e1/0acc913f3deda7b91fcfb73e86a8780d490a54e33f2d2b9b6343078c45f0501b?response-content-disposition=attachment%3B+filename*%3DUTF-8%27%27fine_food_reviews_with_embeddings_1k.csv%3B+filename%3D%22fine_food_reviews_with_embeddings_1k.csv%22%3B&response-content-type=text%2Fcsv&Expires=1699785748&Policy=eyJTdGF0ZW1lbnQiOlt7IkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTY5OTc4NTc0OH19LCJSZXNvdXJjZSI6Imh0dHBzOi8vY2RuLWxmcy5odWdnaW5nZmFjZS5jby9yZXBvcy81Ni9lZS81NmVlYzYwOWU2ZTY3NGQ0ZGRjMjk0MDQ2MThiMTU3N2I2MzgwY2ZkZGMwOTQxODczYzBjMDE4ZGQxNTA5NmUxLzBhY2M5MTNmM2RlZGE3YjkxZmNmYjczZTg2YTg3ODBkNDkwYTU0ZTMzZjJkMmI5YjYzNDMwNzhjNDVmMDUwMWI%7EcmVzcG9uc2UtY29udGVudC1kaXNwb3NpdGlvbj0qJnJlc3BvbnNlLWNvbnRlbnQtdHlwZT0qIn1dfQ__&Signature=lpPqXH7WAqUYfbO-8E9G0jqJhSxSUH7rER71Y4bdoLC6qmBjp7K7GrWRZvuGidhQuupuT1AbrtsO09TZIM3hK50y8o8Zt1-IUVZTKzJWxnaRsHc1XIQFdPBvTyA-%7Ek8eYzz8OmGW5wmb3dbIfpeuyZMK-RZv7LOmfJ9OZSYPFlCJRHeTycgL1IBi%7EcJxPTSjB9gmXDN3Wkta4FMHmIDvYkcwVLX40pkkSaRr92V3pOifgxtfI1HQu3IN-kI09YOdt2X5xzKxPau3kvMb15Su8q5USgolwCsRNeQ5sBSo3lh1fgnZPTO-lKVRG4gM-ysD6NTe5Htfp2D0WfWThyIMLQ__&Key-Pair-Id=KVTP0A1DKRTAX\n",
      "Resolving cdn-lfs.huggingface.co (cdn-lfs.huggingface.co)... 18.66.122.69, 18.66.122.62, 18.66.122.21, ...\n",
      "Connecting to cdn-lfs.huggingface.co (cdn-lfs.huggingface.co)|18.66.122.69|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 35254390 (34M) [text/csv]\n",
      "Saving to: ‘fine_food_reviews_with_embeddings_1k.csv.1’\n",
      "\n",
      "fine_food_reviews_w 100%[===================>]  33.62M  8.30MB/s    in 6.1s    \n",
      "\n",
      "2023-11-09 10:42:35 (5.51 MB/s) - ‘fine_food_reviews_with_embeddings_1k.csv.1’ saved [35254390/35254390]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!wget https://huggingface.co/spaces/ThirdEyeData/Semantic-Search/resolve/main/fine_food_reviews_with_embeddings_1k.csv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>ProductId</th>\n",
       "      <th>UserId</th>\n",
       "      <th>Score</th>\n",
       "      <th>Summary</th>\n",
       "      <th>Text</th>\n",
       "      <th>combined</th>\n",
       "      <th>n_tokens</th>\n",
       "      <th>embedding</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>B003XPF9BO</td>\n",
       "      <td>A3R7JR3FMEBXQB</td>\n",
       "      <td>5</td>\n",
       "      <td>where does one  start...and stop... with a tre...</td>\n",
       "      <td>Wanted to save some to bring to my Chicago fam...</td>\n",
       "      <td>Title: where does one  start...and stop... wit...</td>\n",
       "      <td>52</td>\n",
       "      <td>[0.007018072064965963, -0.02731654793024063, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>297</td>\n",
       "      <td>B003VXHGPK</td>\n",
       "      <td>A21VWSCGW7UUAR</td>\n",
       "      <td>4</td>\n",
       "      <td>Good, but not Wolfgang Puck good</td>\n",
       "      <td>Honestly, I have to admit that I expected a li...</td>\n",
       "      <td>Title: Good, but not Wolfgang Puck good; Conte...</td>\n",
       "      <td>178</td>\n",
       "      <td>[-0.003140551969408989, -0.009995664469897747,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>296</td>\n",
       "      <td>B008JKTTUA</td>\n",
       "      <td>A34XBAIFT02B60</td>\n",
       "      <td>1</td>\n",
       "      <td>Should advertise coconut as an ingredient more...</td>\n",
       "      <td>First, these should be called Mac - Coconut ba...</td>\n",
       "      <td>Title: Should advertise coconut as an ingredie...</td>\n",
       "      <td>78</td>\n",
       "      <td>[-0.01757248118519783, -8.266511576948687e-05,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>295</td>\n",
       "      <td>B000LKTTTW</td>\n",
       "      <td>A14MQ40CCU8B13</td>\n",
       "      <td>5</td>\n",
       "      <td>Best tomato soup</td>\n",
       "      <td>I have a hard time finding packaged food of an...</td>\n",
       "      <td>Title: Best tomato soup; Content: I have a har...</td>\n",
       "      <td>111</td>\n",
       "      <td>[-0.0013932279543951154, -0.011112828738987446...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>294</td>\n",
       "      <td>B001D09KAM</td>\n",
       "      <td>A34XBAIFT02B60</td>\n",
       "      <td>1</td>\n",
       "      <td>Should advertise coconut as an ingredient more...</td>\n",
       "      <td>First, these should be called Mac - Coconut ba...</td>\n",
       "      <td>Title: Should advertise coconut as an ingredie...</td>\n",
       "      <td>78</td>\n",
       "      <td>[-0.01757248118519783, -8.266511576948687e-05,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>995</th>\n",
       "      <td>623</td>\n",
       "      <td>B0000CFXYA</td>\n",
       "      <td>A3GS4GWPIBV0NT</td>\n",
       "      <td>1</td>\n",
       "      <td>Strange inflammation response</td>\n",
       "      <td>Truthfully wasn't crazy about the taste of the...</td>\n",
       "      <td>Title: Strange inflammation response; Content:...</td>\n",
       "      <td>110</td>\n",
       "      <td>[0.00011091353371739388, -0.00466986745595932,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>996</th>\n",
       "      <td>624</td>\n",
       "      <td>B0001BH5YM</td>\n",
       "      <td>A1BZ3HMAKK0NC</td>\n",
       "      <td>5</td>\n",
       "      <td>My favorite and only  MUSTARD</td>\n",
       "      <td>You've just got to experience this mustard... ...</td>\n",
       "      <td>Title: My favorite and only  MUSTARD; Content:...</td>\n",
       "      <td>80</td>\n",
       "      <td>[-0.020869314670562744, -0.013138455338776112,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>997</th>\n",
       "      <td>625</td>\n",
       "      <td>B0009ET7TC</td>\n",
       "      <td>A2FSDQY5AI6TNX</td>\n",
       "      <td>5</td>\n",
       "      <td>My furbabies LOVE these!</td>\n",
       "      <td>Shake the container and they come running. Eve...</td>\n",
       "      <td>Title: My furbabies LOVE these!; Content: Shak...</td>\n",
       "      <td>47</td>\n",
       "      <td>[-0.009749102406203747, -0.0068712360225617886...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>998</th>\n",
       "      <td>619</td>\n",
       "      <td>B007PA32L2</td>\n",
       "      <td>A15FF2P7RPKH6G</td>\n",
       "      <td>5</td>\n",
       "      <td>got this for the daughter</td>\n",
       "      <td>all i have heard since she got a kuerig is why...</td>\n",
       "      <td>Title: got this for the daughter; Content: all...</td>\n",
       "      <td>50</td>\n",
       "      <td>[-0.00521062919870019, 0.0009606690146028996, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>999</th>\n",
       "      <td>999</td>\n",
       "      <td>B001EQ5GEO</td>\n",
       "      <td>A3VYU0VO6DYV6I</td>\n",
       "      <td>5</td>\n",
       "      <td>I love Maui Coffee!</td>\n",
       "      <td>My first experience with Maui Coffee was bring...</td>\n",
       "      <td>Title: I love Maui Coffee!; Content: My first ...</td>\n",
       "      <td>118</td>\n",
       "      <td>[-0.006057822611182928, -0.015015840530395508,...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1000 rows × 9 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     Unnamed: 0   ProductId          UserId  Score   \n",
       "0             0  B003XPF9BO  A3R7JR3FMEBXQB      5  \\\n",
       "1           297  B003VXHGPK  A21VWSCGW7UUAR      4   \n",
       "2           296  B008JKTTUA  A34XBAIFT02B60      1   \n",
       "3           295  B000LKTTTW  A14MQ40CCU8B13      5   \n",
       "4           294  B001D09KAM  A34XBAIFT02B60      1   \n",
       "..          ...         ...             ...    ...   \n",
       "995         623  B0000CFXYA  A3GS4GWPIBV0NT      1   \n",
       "996         624  B0001BH5YM   A1BZ3HMAKK0NC      5   \n",
       "997         625  B0009ET7TC  A2FSDQY5AI6TNX      5   \n",
       "998         619  B007PA32L2  A15FF2P7RPKH6G      5   \n",
       "999         999  B001EQ5GEO  A3VYU0VO6DYV6I      5   \n",
       "\n",
       "                                               Summary   \n",
       "0    where does one  start...and stop... with a tre...  \\\n",
       "1                     Good, but not Wolfgang Puck good   \n",
       "2    Should advertise coconut as an ingredient more...   \n",
       "3                                     Best tomato soup   \n",
       "4    Should advertise coconut as an ingredient more...   \n",
       "..                                                 ...   \n",
       "995                      Strange inflammation response   \n",
       "996                      My favorite and only  MUSTARD   \n",
       "997                           My furbabies LOVE these!   \n",
       "998                          got this for the daughter   \n",
       "999                                I love Maui Coffee!   \n",
       "\n",
       "                                                  Text   \n",
       "0    Wanted to save some to bring to my Chicago fam...  \\\n",
       "1    Honestly, I have to admit that I expected a li...   \n",
       "2    First, these should be called Mac - Coconut ba...   \n",
       "3    I have a hard time finding packaged food of an...   \n",
       "4    First, these should be called Mac - Coconut ba...   \n",
       "..                                                 ...   \n",
       "995  Truthfully wasn't crazy about the taste of the...   \n",
       "996  You've just got to experience this mustard... ...   \n",
       "997  Shake the container and they come running. Eve...   \n",
       "998  all i have heard since she got a kuerig is why...   \n",
       "999  My first experience with Maui Coffee was bring...   \n",
       "\n",
       "                                              combined  n_tokens   \n",
       "0    Title: where does one  start...and stop... wit...        52  \\\n",
       "1    Title: Good, but not Wolfgang Puck good; Conte...       178   \n",
       "2    Title: Should advertise coconut as an ingredie...        78   \n",
       "3    Title: Best tomato soup; Content: I have a har...       111   \n",
       "4    Title: Should advertise coconut as an ingredie...        78   \n",
       "..                                                 ...       ...   \n",
       "995  Title: Strange inflammation response; Content:...       110   \n",
       "996  Title: My favorite and only  MUSTARD; Content:...        80   \n",
       "997  Title: My furbabies LOVE these!; Content: Shak...        47   \n",
       "998  Title: got this for the daughter; Content: all...        50   \n",
       "999  Title: I love Maui Coffee!; Content: My first ...       118   \n",
       "\n",
       "                                             embedding  \n",
       "0    [0.007018072064965963, -0.02731654793024063, 0...  \n",
       "1    [-0.003140551969408989, -0.009995664469897747,...  \n",
       "2    [-0.01757248118519783, -8.266511576948687e-05,...  \n",
       "3    [-0.0013932279543951154, -0.011112828738987446...  \n",
       "4    [-0.01757248118519783, -8.266511576948687e-05,...  \n",
       "..                                                 ...  \n",
       "995  [0.00011091353371739388, -0.00466986745595932,...  \n",
       "996  [-0.020869314670562744, -0.013138455338776112,...  \n",
       "997  [-0.009749102406203747, -0.0068712360225617886...  \n",
       "998  [-0.00521062919870019, 0.0009606690146028996, ...  \n",
       "999  [-0.006057822611182928, -0.015015840530395508,...  \n",
       "\n",
       "[1000 rows x 9 columns]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "csvfile = \"fine_food_reviews_with_embeddings_1k.csv\"\n",
    "\n",
    "df = pd.read_csv(csvfile)\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>ProductId</th>\n",
       "      <th>UserId</th>\n",
       "      <th>Score</th>\n",
       "      <th>Summary</th>\n",
       "      <th>Text</th>\n",
       "      <th>combined</th>\n",
       "      <th>n_tokens</th>\n",
       "      <th>embedding</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>B003XPF9BO</td>\n",
       "      <td>A3R7JR3FMEBXQB</td>\n",
       "      <td>5</td>\n",
       "      <td>where does one  start...and stop... with a tre...</td>\n",
       "      <td>Wanted to save some to bring to my Chicago fam...</td>\n",
       "      <td>Title: where does one  start...and stop... wit...</td>\n",
       "      <td>52</td>\n",
       "      <td>[0.007018072064965963, -0.02731654793024063, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>297</td>\n",
       "      <td>B003VXHGPK</td>\n",
       "      <td>A21VWSCGW7UUAR</td>\n",
       "      <td>4</td>\n",
       "      <td>Good, but not Wolfgang Puck good</td>\n",
       "      <td>Honestly, I have to admit that I expected a li...</td>\n",
       "      <td>Title: Good, but not Wolfgang Puck good; Conte...</td>\n",
       "      <td>178</td>\n",
       "      <td>[-0.003140551969408989, -0.009995664469897747,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>296</td>\n",
       "      <td>B008JKTTUA</td>\n",
       "      <td>A34XBAIFT02B60</td>\n",
       "      <td>1</td>\n",
       "      <td>Should advertise coconut as an ingredient more...</td>\n",
       "      <td>First, these should be called Mac - Coconut ba...</td>\n",
       "      <td>Title: Should advertise coconut as an ingredie...</td>\n",
       "      <td>78</td>\n",
       "      <td>[-0.01757248118519783, -8.266511576948687e-05,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>295</td>\n",
       "      <td>B000LKTTTW</td>\n",
       "      <td>A14MQ40CCU8B13</td>\n",
       "      <td>5</td>\n",
       "      <td>Best tomato soup</td>\n",
       "      <td>I have a hard time finding packaged food of an...</td>\n",
       "      <td>Title: Best tomato soup; Content: I have a har...</td>\n",
       "      <td>111</td>\n",
       "      <td>[-0.0013932279543951154, -0.011112828738987446...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>294</td>\n",
       "      <td>B001D09KAM</td>\n",
       "      <td>A34XBAIFT02B60</td>\n",
       "      <td>1</td>\n",
       "      <td>Should advertise coconut as an ingredient more...</td>\n",
       "      <td>First, these should be called Mac - Coconut ba...</td>\n",
       "      <td>Title: Should advertise coconut as an ingredie...</td>\n",
       "      <td>78</td>\n",
       "      <td>[-0.01757248118519783, -8.266511576948687e-05,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>995</th>\n",
       "      <td>623</td>\n",
       "      <td>B0000CFXYA</td>\n",
       "      <td>A3GS4GWPIBV0NT</td>\n",
       "      <td>1</td>\n",
       "      <td>Strange inflammation response</td>\n",
       "      <td>Truthfully wasn't crazy about the taste of the...</td>\n",
       "      <td>Title: Strange inflammation response; Content:...</td>\n",
       "      <td>110</td>\n",
       "      <td>[0.00011091353371739388, -0.00466986745595932,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>996</th>\n",
       "      <td>624</td>\n",
       "      <td>B0001BH5YM</td>\n",
       "      <td>A1BZ3HMAKK0NC</td>\n",
       "      <td>5</td>\n",
       "      <td>My favorite and only  MUSTARD</td>\n",
       "      <td>You've just got to experience this mustard... ...</td>\n",
       "      <td>Title: My favorite and only  MUSTARD; Content:...</td>\n",
       "      <td>80</td>\n",
       "      <td>[-0.020869314670562744, -0.013138455338776112,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>997</th>\n",
       "      <td>625</td>\n",
       "      <td>B0009ET7TC</td>\n",
       "      <td>A2FSDQY5AI6TNX</td>\n",
       "      <td>5</td>\n",
       "      <td>My furbabies LOVE these!</td>\n",
       "      <td>Shake the container and they come running. Eve...</td>\n",
       "      <td>Title: My furbabies LOVE these!; Content: Shak...</td>\n",
       "      <td>47</td>\n",
       "      <td>[-0.009749102406203747, -0.0068712360225617886...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>998</th>\n",
       "      <td>619</td>\n",
       "      <td>B007PA32L2</td>\n",
       "      <td>A15FF2P7RPKH6G</td>\n",
       "      <td>5</td>\n",
       "      <td>got this for the daughter</td>\n",
       "      <td>all i have heard since she got a kuerig is why...</td>\n",
       "      <td>Title: got this for the daughter; Content: all...</td>\n",
       "      <td>50</td>\n",
       "      <td>[-0.00521062919870019, 0.0009606690146028996, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>999</th>\n",
       "      <td>999</td>\n",
       "      <td>B001EQ5GEO</td>\n",
       "      <td>A3VYU0VO6DYV6I</td>\n",
       "      <td>5</td>\n",
       "      <td>I love Maui Coffee!</td>\n",
       "      <td>My first experience with Maui Coffee was bring...</td>\n",
       "      <td>Title: I love Maui Coffee!; Content: My first ...</td>\n",
       "      <td>118</td>\n",
       "      <td>[-0.006057822611182928, -0.015015840530395508,...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1000 rows × 9 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     Unnamed: 0   ProductId          UserId  Score   \n",
       "0             0  B003XPF9BO  A3R7JR3FMEBXQB      5  \\\n",
       "1           297  B003VXHGPK  A21VWSCGW7UUAR      4   \n",
       "2           296  B008JKTTUA  A34XBAIFT02B60      1   \n",
       "3           295  B000LKTTTW  A14MQ40CCU8B13      5   \n",
       "4           294  B001D09KAM  A34XBAIFT02B60      1   \n",
       "..          ...         ...             ...    ...   \n",
       "995         623  B0000CFXYA  A3GS4GWPIBV0NT      1   \n",
       "996         624  B0001BH5YM   A1BZ3HMAKK0NC      5   \n",
       "997         625  B0009ET7TC  A2FSDQY5AI6TNX      5   \n",
       "998         619  B007PA32L2  A15FF2P7RPKH6G      5   \n",
       "999         999  B001EQ5GEO  A3VYU0VO6DYV6I      5   \n",
       "\n",
       "                                               Summary   \n",
       "0    where does one  start...and stop... with a tre...  \\\n",
       "1                     Good, but not Wolfgang Puck good   \n",
       "2    Should advertise coconut as an ingredient more...   \n",
       "3                                     Best tomato soup   \n",
       "4    Should advertise coconut as an ingredient more...   \n",
       "..                                                 ...   \n",
       "995                      Strange inflammation response   \n",
       "996                      My favorite and only  MUSTARD   \n",
       "997                           My furbabies LOVE these!   \n",
       "998                          got this for the daughter   \n",
       "999                                I love Maui Coffee!   \n",
       "\n",
       "                                                  Text   \n",
       "0    Wanted to save some to bring to my Chicago fam...  \\\n",
       "1    Honestly, I have to admit that I expected a li...   \n",
       "2    First, these should be called Mac - Coconut ba...   \n",
       "3    I have a hard time finding packaged food of an...   \n",
       "4    First, these should be called Mac - Coconut ba...   \n",
       "..                                                 ...   \n",
       "995  Truthfully wasn't crazy about the taste of the...   \n",
       "996  You've just got to experience this mustard... ...   \n",
       "997  Shake the container and they come running. Eve...   \n",
       "998  all i have heard since she got a kuerig is why...   \n",
       "999  My first experience with Maui Coffee was bring...   \n",
       "\n",
       "                                              combined  n_tokens   \n",
       "0    Title: where does one  start...and stop... wit...        52  \\\n",
       "1    Title: Good, but not Wolfgang Puck good; Conte...       178   \n",
       "2    Title: Should advertise coconut as an ingredie...        78   \n",
       "3    Title: Best tomato soup; Content: I have a har...       111   \n",
       "4    Title: Should advertise coconut as an ingredie...        78   \n",
       "..                                                 ...       ...   \n",
       "995  Title: Strange inflammation response; Content:...       110   \n",
       "996  Title: My favorite and only  MUSTARD; Content:...        80   \n",
       "997  Title: My furbabies LOVE these!; Content: Shak...        47   \n",
       "998  Title: got this for the daughter; Content: all...        50   \n",
       "999  Title: I love Maui Coffee!; Content: My first ...       118   \n",
       "\n",
       "                                             embedding  \n",
       "0    [0.007018072064965963, -0.02731654793024063, 0...  \n",
       "1    [-0.003140551969408989, -0.009995664469897747,...  \n",
       "2    [-0.01757248118519783, -8.266511576948687e-05,...  \n",
       "3    [-0.0013932279543951154, -0.011112828738987446...  \n",
       "4    [-0.01757248118519783, -8.266511576948687e-05,...  \n",
       "..                                                 ...  \n",
       "995  [0.00011091353371739388, -0.00466986745595932,...  \n",
       "996  [-0.020869314670562744, -0.013138455338776112,...  \n",
       "997  [-0.009749102406203747, -0.0068712360225617886...  \n",
       "998  [-0.00521062919870019, 0.0009606690146028996, ...  \n",
       "999  [-0.006057822611182928, -0.015015840530395508,...  \n",
       "\n",
       "[1000 rows x 9 columns]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 7.01807206e-03, -2.73165479e-02,  1.05734831e-02, ...,\n",
       "        -7.01120170e-03, -2.18614824e-02, -3.75671238e-02],\n",
       "       [-3.14055197e-03, -9.99566447e-03, -3.48033849e-03, ...,\n",
       "        -9.74494778e-03, -2.39829952e-03, -9.20392852e-03],\n",
       "       [-1.75724812e-02, -8.26651158e-05, -1.15222773e-02, ...,\n",
       "        -1.39020244e-02, -3.90170924e-02, -2.35151257e-02],\n",
       "       ...,\n",
       "       [-9.74910241e-03, -6.87123602e-03, -5.70622832e-03, ...,\n",
       "        -3.00459806e-02, -8.14515445e-03, -1.95114054e-02],\n",
       "       [-5.21062920e-03,  9.60669015e-04,  2.82862745e-02, ...,\n",
       "        -5.38039953e-03, -1.33138765e-02, -2.71892995e-02],\n",
       "       [-6.05782261e-03, -1.50158405e-02, -2.07575737e-03, ...,\n",
       "        -2.90671214e-02, -1.41164539e-02, -2.28756946e-02]])"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "embeddings = np.array(df.embedding.apply(eval).to_list())\n",
    "embeddings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>ProductId</th>\n",
       "      <th>UserId</th>\n",
       "      <th>Score</th>\n",
       "      <th>Summary</th>\n",
       "      <th>Text</th>\n",
       "      <th>combined</th>\n",
       "      <th>n_tokens</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>B003XPF9BO</td>\n",
       "      <td>A3R7JR3FMEBXQB</td>\n",
       "      <td>5</td>\n",
       "      <td>where does one  start...and stop... with a tre...</td>\n",
       "      <td>Wanted to save some to bring to my Chicago fam...</td>\n",
       "      <td>Title: where does one  start...and stop... wit...</td>\n",
       "      <td>52</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>297</td>\n",
       "      <td>B003VXHGPK</td>\n",
       "      <td>A21VWSCGW7UUAR</td>\n",
       "      <td>4</td>\n",
       "      <td>Good, but not Wolfgang Puck good</td>\n",
       "      <td>Honestly, I have to admit that I expected a li...</td>\n",
       "      <td>Title: Good, but not Wolfgang Puck good; Conte...</td>\n",
       "      <td>178</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>296</td>\n",
       "      <td>B008JKTTUA</td>\n",
       "      <td>A34XBAIFT02B60</td>\n",
       "      <td>1</td>\n",
       "      <td>Should advertise coconut as an ingredient more...</td>\n",
       "      <td>First, these should be called Mac - Coconut ba...</td>\n",
       "      <td>Title: Should advertise coconut as an ingredie...</td>\n",
       "      <td>78</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>295</td>\n",
       "      <td>B000LKTTTW</td>\n",
       "      <td>A14MQ40CCU8B13</td>\n",
       "      <td>5</td>\n",
       "      <td>Best tomato soup</td>\n",
       "      <td>I have a hard time finding packaged food of an...</td>\n",
       "      <td>Title: Best tomato soup; Content: I have a har...</td>\n",
       "      <td>111</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>294</td>\n",
       "      <td>B001D09KAM</td>\n",
       "      <td>A34XBAIFT02B60</td>\n",
       "      <td>1</td>\n",
       "      <td>Should advertise coconut as an ingredient more...</td>\n",
       "      <td>First, these should be called Mac - Coconut ba...</td>\n",
       "      <td>Title: Should advertise coconut as an ingredie...</td>\n",
       "      <td>78</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    id   ProductId          UserId  Score   \n",
       "0    0  B003XPF9BO  A3R7JR3FMEBXQB      5  \\\n",
       "1  297  B003VXHGPK  A21VWSCGW7UUAR      4   \n",
       "2  296  B008JKTTUA  A34XBAIFT02B60      1   \n",
       "3  295  B000LKTTTW  A14MQ40CCU8B13      5   \n",
       "4  294  B001D09KAM  A34XBAIFT02B60      1   \n",
       "\n",
       "                                             Summary   \n",
       "0  where does one  start...and stop... with a tre...  \\\n",
       "1                   Good, but not Wolfgang Puck good   \n",
       "2  Should advertise coconut as an ingredient more...   \n",
       "3                                   Best tomato soup   \n",
       "4  Should advertise coconut as an ingredient more...   \n",
       "\n",
       "                                                Text   \n",
       "0  Wanted to save some to bring to my Chicago fam...  \\\n",
       "1  Honestly, I have to admit that I expected a li...   \n",
       "2  First, these should be called Mac - Coconut ba...   \n",
       "3  I have a hard time finding packaged food of an...   \n",
       "4  First, these should be called Mac - Coconut ba...   \n",
       "\n",
       "                                            combined  n_tokens  \n",
       "0  Title: where does one  start...and stop... wit...        52  \n",
       "1  Title: Good, but not Wolfgang Puck good; Conte...       178  \n",
       "2  Title: Should advertise coconut as an ingredie...        78  \n",
       "3  Title: Best tomato soup; Content: I have a har...       111  \n",
       "4  Title: Should advertise coconut as an ingredie...        78  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = df.drop('embedding', axis=1)\n",
    "df = df.rename(columns={'Unnamed: 0': 'id'})\n",
    "df['id'] = df['id'].astype(str)\n",
    "\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1000, 8)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m2023-11-09 10:42:43.197\u001b[0m | \u001b[1mINFO    \u001b[0m | \u001b[36mnomic.project\u001b[0m:\u001b[36m_create_project\u001b[0m:\u001b[36m790\u001b[0m - \u001b[1mCreating project `obsolete-occasion` in organization `Atlas Demo`\u001b[0m\n",
      "\u001b[32m2023-11-09 10:42:49.226\u001b[0m | \u001b[1mINFO    \u001b[0m | \u001b[36mnomic.atlas\u001b[0m:\u001b[36mmap_embeddings\u001b[0m:\u001b[36m107\u001b[0m - \u001b[1mUploading embeddings to Atlas.\u001b[0m\n",
      "1it [00:11, 11.88s/it]\n",
      "\u001b[32m2023-11-09 10:43:01.607\u001b[0m | \u001b[1mINFO    \u001b[0m | \u001b[36mnomic.project\u001b[0m:\u001b[36m_add_data\u001b[0m:\u001b[36m1422\u001b[0m - \u001b[1mUpload succeeded.\u001b[0m\n",
      "\u001b[32m2023-11-09 10:43:01.608\u001b[0m | \u001b[1mINFO    \u001b[0m | \u001b[36mnomic.atlas\u001b[0m:\u001b[36mmap_embeddings\u001b[0m:\u001b[36m126\u001b[0m - \u001b[1mEmbedding upload succeeded.\u001b[0m\n",
      "\u001b[32m2023-11-09 10:43:03.141\u001b[0m | \u001b[1mINFO    \u001b[0m | \u001b[36mnomic.project\u001b[0m:\u001b[36mcreate_index\u001b[0m:\u001b[36m1132\u001b[0m - \u001b[1mCreated map `obsolete-occasion` in project `obsolete-occasion`: https://atlas.nomic.ai/map/47c5d6b0-ed07-4708-9a36-c8b214a61b76/3662a65f-ff10-4096-9e08-0438505464e8\u001b[0m\n",
      "\u001b[32m2023-11-09 10:43:03.142\u001b[0m | \u001b[1mINFO    \u001b[0m | \u001b[36mnomic.atlas\u001b[0m:\u001b[36mmap_embeddings\u001b[0m:\u001b[36m139\u001b[0m - \u001b[1mobsolete-occasion: https://atlas.nomic.ai/map/47c5d6b0-ed07-4708-9a36-c8b214a61b76/3662a65f-ff10-4096-9e08-0438505464e8\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "project = atlas.map_embeddings(embeddings=embeddings,\n",
    "                               data=df.to_dict('records'),\n",
    "                               id_field='id',\n",
    "                               colorable_fields=['Score'])\n",
    "map = project.maps[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "Atlas Projection obsolete-occasion. Status Projecting. <a target=\"_blank\" href=\"https://atlas.nomic.ai/map/47c5d6b0-ed07-4708-9a36-c8b214a61b76/3662a65f-ff10-4096-9e08-0438505464e8\">view online</a>"
      ],
      "text/plain": [
       "obsolete-occasion: https://atlas.nomic.ai/map/47c5d6b0-ed07-4708-9a36-c8b214a61b76/3662a65f-ff10-4096-9e08-0438505464e8"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "map"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"capture.png\">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.10 - SDK v2",
   "language": "python",
   "name": "python310-sdkv2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.10"
  },
  "vscode": {
   "interpreter": {
    "hash": "365536dcbde60510dc9073d6b991cd35db2d9bac356a11f5b64279a5e6708b97"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}