Proof Read several Milvus Lite integrations and convert to notebook #1328
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,191 @@ | ||
| { | ||
|
There was a problem hiding this comment. |
||
| "cells": [ | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 4, | ||
| "id": "initial_id", | ||
| "metadata": { | ||
| "collapsed": true, | ||
| "ExecuteTime": { | ||
| "end_time": "2024-05-17T04:22:47.480654Z", | ||
| "start_time": "2024-05-17T04:22:44.365493Z" | ||
| } | ||
| }, | ||
| "outputs": [ | ||
| { | ||
| "name": "stdout", | ||
| "output_type": "stream", | ||
| "text": [ | ||
| "Defaulting to user installation because normal site-packages is not writeable\r\n", | ||
| "Requirement already satisfied: voyageai in /Users/jackson/Library/Python/3.9/lib/python/site-packages (0.2.2)\r\n", | ||
| "Requirement already satisfied: pymilvus in /Users/jackson/Library/Python/3.9/lib/python/site-packages (2.4.3)\r\n", | ||
| "Requirement already satisfied: aiohttp<4.0,>=3.5 in /Library/Python/3.9/site-packages (from voyageai) (3.9.5)\r\n", | ||
| "Requirement already satisfied: aiolimiter<2.0.0,>=1.1.0 in /Users/jackson/Library/Python/3.9/lib/python/site-packages (from voyageai) (1.1.0)\r\n", | ||
| "Requirement already satisfied: numpy>=1.11 in /Library/Python/3.9/site-packages (from voyageai) (1.26.4)\r\n", | ||
| "Requirement already satisfied: requests<3.0,>=2.20 in /Library/Python/3.9/site-packages (from voyageai) (2.31.0)\r\n", | ||
| "Requirement already satisfied: tenacity>=8.0.1 in /Users/jackson/Library/Python/3.9/lib/python/site-packages (from voyageai) (8.2.3)\r\n", | ||
| "Requirement already satisfied: setuptools>=67 in /Users/jackson/Library/Python/3.9/lib/python/site-packages (from pymilvus) (69.5.1)\r\n", | ||
| "Requirement already satisfied: grpcio<=1.63.0,>=1.49.1 in /Users/jackson/Library/Python/3.9/lib/python/site-packages (from pymilvus) (1.57.0)\r\n", | ||
| "Requirement already satisfied: protobuf>=3.20.0 in /Users/jackson/Library/Python/3.9/lib/python/site-packages (from pymilvus) (3.20.2)\r\n", | ||
| "Requirement already satisfied: environs<=9.5.0 in /Users/jackson/Library/Python/3.9/lib/python/site-packages (from pymilvus) (9.5.0)\r\n", | ||
| "Requirement already satisfied: ujson>=2.0.0 in /Users/jackson/Library/Python/3.9/lib/python/site-packages (from pymilvus) (5.9.0)\r\n", | ||
| "Requirement already satisfied: pandas>=1.2.4 in /Users/jackson/Library/Python/3.9/lib/python/site-packages (from pymilvus) (2.2.2)\r\n", | ||
| "Requirement already satisfied: milvus-lite<2.5.0,>=2.4.0 in /Users/jackson/Library/Python/3.9/lib/python/site-packages (from pymilvus) (2.4.3)\r\n", | ||
| "Requirement already satisfied: aiosignal>=1.1.2 in /Library/Python/3.9/site-packages (from aiohttp<4.0,>=3.5->voyageai) (1.3.1)\r\n", | ||
| "Requirement already satisfied: attrs>=17.3.0 in /Library/Python/3.9/site-packages (from aiohttp<4.0,>=3.5->voyageai) (23.2.0)\r\n", | ||
| "Requirement already satisfied: frozenlist>=1.1.1 in /Library/Python/3.9/site-packages (from aiohttp<4.0,>=3.5->voyageai) (1.4.1)\r\n", | ||
| "Requirement already satisfied: multidict<7.0,>=4.5 in /Library/Python/3.9/site-packages (from aiohttp<4.0,>=3.5->voyageai) (6.0.5)\r\n", | ||
| "Requirement already satisfied: yarl<2.0,>=1.0 in /Library/Python/3.9/site-packages (from aiohttp<4.0,>=3.5->voyageai) (1.9.4)\r\n", | ||
| "Requirement already satisfied: async-timeout<5.0,>=4.0 in /Library/Python/3.9/site-packages (from aiohttp<4.0,>=3.5->voyageai) (4.0.3)\r\n", | ||
| "Requirement already satisfied: marshmallow>=3.0.0 in /Users/jackson/Library/Python/3.9/lib/python/site-packages (from environs<=9.5.0->pymilvus) (3.21.1)\r\n", | ||
| "Requirement already satisfied: python-dotenv in /Users/jackson/Library/Python/3.9/lib/python/site-packages (from environs<=9.5.0->pymilvus) (1.0.1)\r\n", | ||
| "Requirement already satisfied: python-dateutil>=2.8.2 in /Library/Python/3.9/site-packages (from pandas>=1.2.4->pymilvus) (2.9.0.post0)\r\n", | ||
| "Requirement already satisfied: pytz>=2020.1 in /Users/jackson/Library/Python/3.9/lib/python/site-packages (from pandas>=1.2.4->pymilvus) (2024.1)\r\n", | ||
| "Requirement already satisfied: tzdata>=2022.7 in /Users/jackson/Library/Python/3.9/lib/python/site-packages (from pandas>=1.2.4->pymilvus) (2024.1)\r\n", | ||
| "Requirement already satisfied: charset-normalizer<4,>=2 in /Library/Python/3.9/site-packages (from requests<3.0,>=2.20->voyageai) (3.3.2)\r\n", | ||
| "Requirement already satisfied: idna<4,>=2.5 in /Library/Python/3.9/site-packages (from requests<3.0,>=2.20->voyageai) (3.7)\r\n", | ||
| "Requirement already satisfied: urllib3<3,>=1.21.1 in /Users/jackson/Library/Python/3.9/lib/python/site-packages (from requests<3.0,>=2.20->voyageai) (1.26.18)\r\n", | ||
| "Requirement already satisfied: certifi>=2017.4.17 in /Library/Python/3.9/site-packages (from requests<3.0,>=2.20->voyageai) (2024.2.2)\r\n", | ||
| "Requirement already satisfied: packaging>=17.0 in /Users/jackson/Library/Python/3.9/lib/python/site-packages (from marshmallow>=3.0.0->environs<=9.5.0->pymilvus) (23.2)\r\n", | ||
| "Requirement already satisfied: six>=1.5 in /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/site-packages (from python-dateutil>=2.8.2->pandas>=1.2.4->pymilvus) (1.15.0)\r\n" | ||
| ] | ||
| } | ||
| ], | ||
| "source": [ | ||
| "!pip install --upgrade voyageai pymilvus" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 26, | ||
| "outputs": [ | ||
| { | ||
| "name": "stdout", | ||
| "output_type": "stream", | ||
| "text": [ | ||
| "3\n" | ||
| ] | ||
| } | ||
| ], | ||
| "source": [ | ||
| "import voyageai\n", | ||
| "from pymilvus import MilvusClient\n", | ||
| "\n", | ||
| "MODEL_NAME = \"voyage-law-2\" # Which model to use, please check https://docs.voyageai.com/docs/embeddings for available models\n", | ||
| "DIMENSION = 1024 # Dimension of vector embedding \n", | ||
| "\n", | ||
| "# Connect to VoyageAI with API Key.\n", | ||
| "voyage_client = voyageai.Client(api_key=\"YOUR_VOYAGEAI_API_KEY\")\n", | ||
| "\n", | ||
| "docs = [\n", | ||
| " \"Artificial intelligence was founded as an academic discipline in 1956.\",\n", | ||
| " \"Alan Turing was the first person to conduct substantial research in AI.\",\n", | ||
| " \"Born in Maida Vale, London, Turing was raised in southern England.\",\n", | ||
| "]\n", | ||
| "\n", | ||
| "vectors = voyage_client.embed(\n", | ||
| " texts=docs,\n", | ||
| " model=MODEL_NAME,\n", | ||
| " truncation=False\n", | ||
| " ).embeddings\n", | ||
| "\n", | ||
| "# Prepare data to be stored in Milvus vector database.\n", | ||
| "# We can store the id, vector representation, raw text and labels such as \"subject\" in this case in Milvus.\n", | ||
| "data = [ {\"id\": i, \"vector\": vectors[i], \"text\": docs[i], \"subject\": \"history\"} for i in range(len(docs)) ]\n", | ||
| "\n", | ||
| "\n", | ||
| "# Connect to Milvus, all data is stored in a local file named \"milvus_voyage_demo.db\"\n", | ||
| "# in current directory. You can also connect to a remote Milvus server following this\n", | ||
| "# instruction: https://milvus.io/docs/install_standalone-docker.md.\n", | ||
| "milvus_client = MilvusClient(\"milvus_voyage_demo.db\")\n", | ||
| "COLLECTION_NAME = \"demo_collection\" # Milvus collection name\n", | ||
| "# Create a collection to store the vectors and text. \n", | ||
| "milvus_client.create_collection(\n", | ||
| " collection_name=COLLECTION_NAME,\n", | ||
| " dimension=DIMENSION\n", | ||
| ")\n", | ||
| "\n", | ||
| "# Insert all data into Milvus vector database.\n", | ||
| "res = milvus_client.insert(\n", | ||
| " collection_name=\"demo_collection\",\n", | ||
| " data=data\n", | ||
| ")\n", | ||
| "\n", | ||
| "print(res[\"insert_count\"])" | ||
| ], | ||
| "metadata": { | ||
| "collapsed": false, | ||
| "ExecuteTime": { | ||
| "end_time": "2024-05-17T04:49:05.807960Z", | ||
| "start_time": "2024-05-17T04:49:04.527034Z" | ||
| } | ||
| }, | ||
| "id": "5e5bffb900bf075e" | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 36, | ||
| "outputs": [ | ||
| { | ||
| "name": "stdout", | ||
| "output_type": "stream", | ||
| "text": [ | ||
| "Query: When was artificial intelligence founded?\n", | ||
| "[{'id': 0, 'distance': -0.7196217775344849, 'entity': {'text': 'Artificial intelligence was founded as an academic discipline in 1956.', 'subject': 'history'}}]\n" | ||
| ] | ||
| } | ||
| ], | ||
| "source": [ | ||
| "queries = [\"When was artificial intelligence founded?\"]\n", | ||
| "\n", | ||
| "query_vectors = voyage_client.embed(\n", | ||
| " texts=queries,\n", | ||
| " model=MODEL_NAME,\n", | ||
| " truncation=False\n", | ||
| " ).embeddings\n", | ||
| "\n", | ||
| "res = milvus_client.search(\n", | ||
| " collection_name=COLLECTION_NAME, # target collection\n", | ||
| " data=query_vectors, # query vectors\n", | ||
| " limit=2, # number of returned entities\n", | ||
| " output_fields=[\"text\", \"subject\"], # specifies fields to be returned\n", | ||
| ")\n", | ||
| "\n", | ||
| "for q in queries:\n", | ||
| " print('Query:', q)\n", | ||
| " for result in res:\n", | ||
| " print(result)\n", | ||
| " print(\"\\n\")" | ||
| ], | ||
| "metadata": { | ||
| "collapsed": false, | ||
| "ExecuteTime": { | ||
| "end_time": "2024-05-17T04:56:10.205658Z", | ||
| "start_time": "2024-05-17T04:56:09.702360Z" | ||
| } | ||
| }, | ||
| "id": "40d0cd0218d44802" | ||
| } | ||
| ], | ||
| "metadata": { | ||
| "kernelspec": { | ||
| "display_name": "Python 3", | ||
| "language": "python", | ||
| "name": "python3" | ||
| }, | ||
| "language_info": { | ||
| "codemirror_mode": { | ||
| "name": "ipython", | ||
| "version": 2 | ||
| }, | ||
| "file_extension": ".py", | ||
| "mimetype": "text/x-python", | ||
| "name": "python", | ||
| "nbconvert_exporter": "python", | ||
| "pygments_lexer": "ipython2", | ||
| "version": "2.7.6" | ||
| } | ||
| }, | ||
| "nbformat": 4, | ||
| "nbformat_minor": 5 | ||
| } | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why changing these two lines?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is the commit by a previous person