RAG Copilot Extension LP

assets/contributors.csv

@@ -1,81 +1,82 @@
-author,company,github,linkedin,twitter,website
-Jason Andrews,Arm,jasonrandrews,jason-andrews-7b05a8,,
-Pareena Verma,Arm,pareenaverma,pareena-verma-7853607,,
-Ronan Synnott,Arm,,ronansynnott,,
-Florent Lebeau,Arm,,,,
-Brenda Strech,Remote.It,bstrech,bstrech,@remote_it,www.remote.it
-Liliya Wu,Arm,Liliyaw,liliya-wu-8b6227216,,
-Julio Suarez,Arm,jsrz,juliosuarez,,
-Gabriel Peterson,Arm,gabrieldpeterson,gabrieldpeterson,@gabedpeterson,https://corteximplant.com/@gabe
-Christopher Seidl,Arm,,,,
-Michael Hall,Arm,,,,
-Kasper Mecklenburg,Arm,,,,
-Mathias Brossard,Arm,,,,
-Julie Gaskin,Arm,,,,
-Pranay Bakre,Arm,,,,
-Elham Harirpoush,Arm,,,,
-Frédéric -lefred- Descamps,OCI,,,,lefred.be
-Fr�d�ric -lefred- Descamps,OCI,,,,lefred.be
-Kristof Beyls,Arm,,,,
-David Spickett,Arm,,,,
-Uma Ramalingam,Arm,uma-ramalingam,,,
-Konstantinos Margaritis,VectorCamp,markos,konstantinosmargaritis,@freevec1,https://vectorcamp.gr/
-Diego Russo,Arm,diegorusso,diegor,diegor,https://www.diegor.it
-Jonathan Davies,Arm,,,,
-Zhengjun Xing,Arm,,,,
-Leandro Nunes,Arm,,,,
-Dawid Borycki,,dawidborycki,,,
-Ying Yu,Arm,,,,
-Bolt Liu,Arm,,,,
-Roberto Lopez Mendez,Arm,,,,
-Arnaud de Grandmaison,Arm,Arnaud-de-Grandmaison-ARM,arnauddegrandmaison,,
-Jose-Emilio Munoz-Lopez,Arm,,,,
-James Whitaker,Arm,,,,
-Johanna Skinnider,Arm,,,,
-Varun Chari,Arm,,,,
-Adnan AlSinan,Arm,,,,
-Graham Woodward,Arm,,,,
-Basma El Gaabouri,Arm,,,,
-Gayathri Narayana Yegna Narayanan,Arm,,,,
-Alexandros Lamprineas,Arm,,,,
-Annie Tallund,Arm,annietllnd,annietallund,,
-Cyril Rohr,RunsOn,crohr,cyrilrohr,,
-Rin Dobrescu,Arm,,,,
-Przemyslaw Wirkus,Arm,PrzemekWirkus,przemyslaw-wirkus-78b73352,,
-Nader Zouaoui,Day Devs,nader-zouaoui,nader-zouaoui,@zouaoui_nader,https://daydevs.com/
-Alaaeddine Chakroun,Day Devs,Alaaeddine-Chakroun,alaaeddine-chakroun,,https://daydevs.com/
-Koki Mitsunami,Arm,,kmitsunami,,
-Chen Zhang,Zilliz,,,,
-Tianyu Li,Arm,,,,
-Georgios Mermigkis,VectorCamp,gMerm,georgios-mermigkis,,https://vectorcamp.gr/ 
-Ben Clark,Arm,,,,
-Han Yin,Arm,hanyin-arm,nacosiren,,
-Willen Yang,Arm,,,,
-Daniel Gubay,,,,,
-Paul Howard,,,,,
-Iago Calvo Lista,Arm,,,,
-Stephen Theobald,Arm,,,,
-ThirdAI,,,,,
-Preema Merlin Dsouza,,,,,
-Dominica Abena O. Amanfo,,,,,
-Arm,,,,,
-Albin Bernhardsson,,,,,
-Przemyslaw Wirkus,,,,,
-Zach Lasiuk,,,,,
-Daniel Nguyen,,,,,
-Joe Stech,Arm,,,,
-visualSilicon,,,,,
-Konstantinos Margaritis,VectorCamp,,,,
-Kieran Hejmadi,,,,,
-Alex Su,,,,,
-Chaodong Gong,,,,,
-Owen Wu,Arm,,,,
-Koki Mitsunami,,,,,
-Nikhil Gupta,,,,,
-Nobel Chowdary Mandepudi,Arm,,,,
-Ravi Malhotra,Arm,,,,
-Masoud Koleini,,,,,
-Na Li,Arm,,,,
-Tom Pilar,,,,,
-Cyril Rohr,,,,,
-Odin Shen,Arm,odincodeshen,odin-shen-lmshen,,
+author,company,github,linkedin,twitter,website
+Jason Andrews,Arm,jasonrandrews,jason-andrews-7b05a8,,
+Pareena Verma,Arm,pareenaverma,pareena-verma-7853607,,
+Ronan Synnott,Arm,,ronansynnott,,
+Florent Lebeau,Arm,,,,
+Brenda Strech,Remote.It,bstrech,bstrech,@remote_it,www.remote.it
+Liliya Wu,Arm,Liliyaw,liliya-wu-8b6227216,,
+Julio Suarez,Arm,jsrz,juliosuarez,,
+Gabriel Peterson,Arm,gabrieldpeterson,gabrieldpeterson,@gabedpeterson,https://corteximplant.com/@gabe
+Christopher Seidl,Arm,,,,
+Michael Hall,Arm,,,,
+Kasper Mecklenburg,Arm,,,,
+Mathias Brossard,Arm,,,,
+Julie Gaskin,Arm,,,,
+Pranay Bakre,Arm,,,,
+Elham Harirpoush,Arm,,,,
+Frédéric -lefred- Descamps,OCI,,,,lefred.be
+Fr�d�ric -lefred- Descamps,OCI,,,,lefred.be
+Kristof Beyls,Arm,,,,
+David Spickett,Arm,,,,
+Uma Ramalingam,Arm,uma-ramalingam,,,
+Konstantinos Margaritis,VectorCamp,markos,konstantinosmargaritis,@freevec1,https://vectorcamp.gr/
+Diego Russo,Arm,diegorusso,diegor,diegor,https://www.diegor.it
+Jonathan Davies,Arm,,,,
+Zhengjun Xing,Arm,,,,
+Leandro Nunes,Arm,,,,
+Dawid Borycki,,dawidborycki,,,
+Ying Yu,Arm,,,,
+Bolt Liu,Arm,,,,
+Roberto Lopez Mendez,Arm,,,,
+Arnaud de Grandmaison,Arm,Arnaud-de-Grandmaison-ARM,arnauddegrandmaison,,
+Jose-Emilio Munoz-Lopez,Arm,,,,
+James Whitaker,Arm,,,,
+Johanna Skinnider,Arm,,,,
+Varun Chari,Arm,,,,
+Adnan AlSinan,Arm,,,,
+Graham Woodward,Arm,,,,
+Basma El Gaabouri,Arm,,,,
+Gayathri Narayana Yegna Narayanan,Arm,,,,
+Alexandros Lamprineas,Arm,,,,
+Annie Tallund,Arm,annietllnd,annietallund,,
+Cyril Rohr,RunsOn,crohr,cyrilrohr,,
+Rin Dobrescu,Arm,,,,
+Przemyslaw Wirkus,Arm,PrzemekWirkus,przemyslaw-wirkus-78b73352,,
+Nader Zouaoui,Day Devs,nader-zouaoui,nader-zouaoui,@zouaoui_nader,https://daydevs.com/
+Alaaeddine Chakroun,Day Devs,Alaaeddine-Chakroun,alaaeddine-chakroun,,https://daydevs.com/
+Koki Mitsunami,Arm,,kmitsunami,,
+Chen Zhang,Zilliz,,,,
+Tianyu Li,Arm,,,,
+Georgios Mermigkis,VectorCamp,gMerm,georgios-mermigkis,,https://vectorcamp.gr/ 
+Ben Clark,Arm,,,,
+Han Yin,Arm,hanyin-arm,nacosiren,,
+Willen Yang,Arm,,,,
+Daniel Gubay,,,,,
+Paul Howard,,,,,
+Iago Calvo Lista,Arm,,,,
+Stephen Theobald,Arm,,,,
+ThirdAI,,,,,
+Preema Merlin Dsouza,,,,,
+Dominica Abena O. Amanfo,,,,,
+Arm,,,,,
+Albin Bernhardsson,,,,,
+Przemyslaw Wirkus,,,,,
+Zach Lasiuk,,,,,
+Daniel Nguyen,,,,,
+Joe Stech,Arm,,,,
+visualSilicon,,,,,
+Konstantinos Margaritis,VectorCamp,,,,
+Kieran Hejmadi,,,,,
+Alex Su,,,,,
+Chaodong Gong,,,,,
+Owen Wu,Arm,,,,
+Koki Mitsunami,,,,,
+Nikhil Gupta,,,,,
+Nobel Chowdary Mandepudi,Arm,,,,
+Ravi Malhotra,Arm,,,,
+Masoud Koleini,,,,,
+Na Li,Arm,,,,
+Tom Pilar,,,,,
+Cyril Rohr,,,,,
+Odin Shen,Arm,odincodeshen,odin-shen-lmshen,,
+Avin Zarlez,Arm,AvinZarlez,avinzarlez,,https://www.avinzarlez.com/

content/learning-paths/servers-and-cloud-computing/copilot-extension/1-rag.md

@@ -0,0 +1,21 @@
+---
+title: RAG Overview
+weight: 2
+
+### FIXED, DO NOT MODIFY
+layout: learningpathall
+---
+
+## What is a RAG system?
+
+RAG stands for "Retrieval Augmented Generation". It describes an AI framework that combines information retrieval with text generation to improve the quality and accuracy of AI-generated content.
+
+The basic flow of a RAG system looks like this:
+
+1. Retrieval: The system searches a knowledge base, usually using some combination of vector and/or text search.
+2. Augmentation: The retrieved information is then provided as context to a generative AI model to provide additional context for the user's query.
+3. The AI model uses both the retrieved knowledge and its internal understanding to generate a more useful response to the user.
+
+The benefits of a RAG system revolve around improved factual accuracy of responses. It also allows a system to understand more up-to-date information, since you can add additional knowledge to the knowledge base much more easily than you could retrain the model.
+
+Most importantly, RAG lets you provide reference links to the user, showing the user where the system is getting its information.

content/learning-paths/servers-and-cloud-computing/copilot-extension/2-vector.md

@@ -0,0 +1,160 @@
+---
+title: Vector Database
+weight: 3
+
+### FIXED, DO NOT MODIFY
+layout: learningpathall
+---
+
+## What is a Vector Database?
+
+A vector database is a specialized database designed to store and query vector representations of data. They are a crucial component of many AI applications. But what exactly are they, and how do they work?
+
+Traditional databases store data in tables or objects with defined attributes. However, they struggle to recognize similarities between data points that aren't explicitly defined.
+
+Vector databases, on the other hand, are designed to store a large numbers of vectors (arrays of numbers), and provide algorithms to be able to search through those stored vectors. That makes it much easier to identify similarities by comparing the vector locations in N dimensional space. This is typically done using distance metrics like cosine similarity or Euclidean distance.
+
+How can we convert complex ideas, like the semantic meaning of a series of words, into a series of of number based vectors? We do so using a process called embedding.
+
+### Embeddings
+
+Embeddings are vectors generated through an AI model. We can convert collections of "tokens" (word fragments) into a point in N dimensional space.
+
+Then for any given vector (like the embedding of a question asked by a user) we can query our vector database to find embedded data that is most similar.
+
+For our use case, we want to know which Arm learning path is most relevant to a question a user asks.
+
+First, ahead of time, we have to convert the raw data (Arm learning path content) into more consumable "chunks". In our case, small `yaml` files. Then we run those chunks through our LLM model and embed the content into our FAISS vector database.
+
+### FAISS
+
+FAISS (Facebook AI Similarity Search) is a library developed by Facebook AI Research that is designed to efficiently search for similar vectors in large datasets. FAISS is highly optimized for both memory usage and speed, making it the fastest similarity search algorithm available.
+
+One of the key reasons FAISS is so fast is its implementation of efficient Approximate Nearest Neighbor (ANN) search algorithms. ANN algorithms allow FAISS to quickly find vectors that are close to a given query vector without having to compare it to every single vector in the database. This significantly reduces the search time, especially in large datasets.
+
+Additionally, FAISS performs all searches in-memory, which means that it can leverage the full speed of the system's RAM. This in-memory search capability ensures that the search operations are extremely fast, as they avoid the latency associated with disk I/O operations.
+
+In our application, we can take the input from the user and embed it using the same model we used for our database. We then use FAISS nearest neighbor search to compare the user input to the nearest vectors in the database. We then look at the original chunk files for those closest vectors. Using the data from those `chunk.yaml` files, we can retrieve the Arm resource(s) most relevant for that user's question.
+
+The retrieved resources are then used to augment the context for the LLM, which generates a final response that is both contextually relevant and contains accurate information.
+
+### In Memory Deployment
+
+To ensure that our application scales efficiently, we will copy the FAISS database into every deployment instance. By deploying a static in-memory vector store in each instance, we eliminate the need for a centralized database, which can become a bottleneck as the number of requests increases.
+
+When each instance has its own copy of the FAISS database, it can perform vector searches locally, leveraging the full speed of the system's RAM. This approach ensures that the search operations are extremely fast and reduces the latency associated with network calls to a centralized database.
+
+Moreover, this method enhances the reliability and fault tolerance of our application. If one instance fails, others can continue to operate independently without being affected by the failure. This decentralized approach also simplifies the deployment process, as each instance is self-contained and does not rely on external resources for vector searches.
+
+By copying the FAISS database into every deployment, we achieve a scalable, high-performance solution that can handle a large number of requests efficiently.
+
+## Collecting Data into Chunks
+
+Arm has provided a [companion GitHub repo](https://github.com/ArmDeveloperEcosystem/python-rag-extension/) for this Learning Path that serves as a Python-based Copilot RAG Extension example. In this repo, we have provided scripts to convert an Arm learning path into a series of `chunk.yaml` files for use in our RAG application.
+
+### Clone the GitHub repository
+
+To clone the repo, run
+
+```bash
+git clone https://github.com/ArmDeveloperEcosystem/python-rag-extension.git
+```
+
+### Chunk Creation Script Set up
+
+Navigate to the `vectorstore` folder in the [python-rag-extension github repo](https://github.com/ArmDeveloperEcosystem/python-rag-extension/) you just cloned.
+
+```bash
+cd python-rag-extension/vectorstore
+```
+
+It is recommended to use a virtual environment to manage dependencies.
+
+Ensure you have `conda` set up in your development environment. If you aren't sure how, you can follow this [Installation Guide](https://docs.anaconda.com/miniconda/install/).
+
+To create a new conda environment, use the following command:
+
+```sh
+conda create --name vectorstore python=3.11
+```
+
+Once set up is complete, activate the new environment:
+
+```sh
+conda activate vectorstore
+```
+
+Install the required packages:
+
+```sh
+conda install --file vectorstore-requirements.txt
+```
+
+### Generate Chunk Files
+
+To generate chunks, use the following command:
+
+```sh
+python chunk_a_learning_path.py --url <LEARNING_PATH_URL>
+```
+
+Replace `<LEARNING_PATH_URL>` with the URL of the learning path you want to process. If no URL is provided, the script will default to a [known learning path URL](https://learn.arm.com/learning-paths/cross-platform/kleidiai-explainer).
+
+The script will process the specified learning path and save the chunks as YAML files in a `./chunks/` directory.
+
+## Combine Chunks into FAISS index
+
+Once you have a `./chunks/` directory full of yaml files, we now need to use FAISS to create our vector database.
+
+### OpenAI Key and Endpoint
+
+Ensure your local environment has your `AZURE_OPENAI_KEY` and `AZURE_OPENAI_ENDPOINT` set.
+
+#### If needed, generate Azure OpenAI keys and deployment 
+
+1. **Create an OpenAI Resource**:
+    - Go to the [Azure Portal](https://portal.azure.com/).
+    - Click on "Create a resource".
+    - Search for "OpenAI" and select "Azure OpenAI Service".
+    - Click "Create".
+
+1. **Configure the OpenAI Resource**:
+    - Fill in the required details such as Subscription, Resource Group, Region, and Name.
+    - Click "Review + create" and then "Create" to deploy the resource.
+
+1. **Generate API Key and Endpoint**:
+    - Once the resource is created, navigate to the resource page.
+    - Under the "Resource Management->Keys and Endpoint" section, you will find the key and endpoint values.
+    - Copy these values and set them in your local environment.
+
+    ```sh
+    export AZURE_OPENAI_KEY="<your_openai_key>"
+    export AZURE_OPENAI_ENDPOINT="https://<your_openai_endpoint>.openai.azure.com/"
+    ```
+
+    You now have the necessary keys to use Azure OpenAI in your application.
+
+1. **Deploy text-embedding-ada-002 model**
+    - Go inside Azure AI Foundry for your new deployment
+    - Under "Deployments", ensure you have a deployment for "text-embedding-ada-002"
+
+### Generate Vector Database Files
+
+Run the python script to create the FAISS index `.bin` and `.json` files.
+
+**NOTE:** This assumes the chunk files are located in a `chunks` subfolder, as they should automatically be.
+
+```bash
+python local_vectorstore_creation.py
+```
+
+Copy the generated `bin` and `json` files to the root directory of your Flask application.
+
+They should be in the `vectorstore/chunks` folder. Since you are likely still in the `vectorstore` folder, run this command to copy:
+
+```bash
+cp chunks/faiss_index.bin ../
+cp chunks/metadata.json ../
+```
+
+Your vector database is now ready for your flask application.

content/learning-paths/servers-and-cloud-computing/copilot-extension/3-github-app.md

@@ -0,0 +1,43 @@
+---
+title: Create GitHub Application
+weight: 4
+
+### FIXED, DO NOT MODIFY
+layout: learningpathall
+---
+
+Now we need to create a Copilot extension on GitHub to connect to our deployed application.
+
+## Create a GitHub app
+
+> For the most up to date instructions, follow the [official documentation for creating a GitHub App for Copilot Extension](https://docs.github.com/en/copilot/building-copilot-extensions/creating-a-copilot-extension/creating-a-github-app-for-your-copilot-extension#creating-a-github-app).
+
+On any page of [GitHub](https://github.com/), click your profile picture and go to Settings. Scroll down to developer settings, and go to [create a GitHub App](https://github.com/settings/apps).
+
+![Create GitHub Application screen](images/githubapp-create.png)
+
+Your GitHub App must have:
+    - A name
+    - A homepage URL
+    - Make sure Webhook -> Active is deselected
+
+![GitHub App name and URL](images/githubapp-name.png)
+![Webhook deselected](images/githubapp-deselected.png)
+
+The rest can be the default values.
+
+Scroll to the bottom and click "Create GitHub App"
+
+## Get Client ID and Secret
+
+After you create your app, open it up. You will see listed your Client ID under General -> About.
+
+![Client ID and Secret](images/githubapp-clientid.png)
+
+Under that is **Client Secrets**, click "Generate a new client secret" and save the value. Make sure you copy it before it goes away, you will need it for the next step as part of the flask application.
+
+## Install Application
+
+Click **Install App** in the sidebar, then install your app onto your account.
+
+![Install](images/githubapp-install.png)