From a1c934a16498fa3a94cc49c83642356e5c738604 Mon Sep 17 00:00:00 2001 From: mfranzon Date: Wed, 10 Jul 2024 15:45:13 +0200 Subject: [PATCH 01/30] add rag-ollama application example --- content/guides/use-case/rag-ollama/_index.md | 16 ++ .../use-case/rag-ollama/containerize.md | 84 ++++++++++ content/guides/use-case/rag-ollama/develop.md | 157 ++++++++++++++++++ 3 files changed, 257 insertions(+) create mode 100644 content/guides/use-case/rag-ollama/_index.md create mode 100644 content/guides/use-case/rag-ollama/containerize.md create mode 100644 content/guides/use-case/rag-ollama/develop.md diff --git a/content/guides/use-case/rag-ollama/_index.md b/content/guides/use-case/rag-ollama/_index.md new file mode 100644 index 000000000000..3bd3d8ffec08 --- /dev/null +++ b/content/guides/use-case/rag-ollama/_index.md @@ -0,0 +1,16 @@ +--- +description: Containerize RAG application using Ollama and Docker +keywords: python, generative ai, genai, llm, ollama, rag, qdrant +title: Build a RAG application using Ollama and Docker +toc_min: 1 +toc_max: 2 +--- + +The Retrival Augmented Generation (RAG) guide teaches you how to containerize an existing RAG application using Docker. The example application is a RAG that acts like a sommelier, giving you the best pairings between wines and food. In this guide, you’ll learn how to: + +* Containerize and run a RAG application +* Set up a local environment to run the complete RAG stack locally for development + +Start by containerizing an existing RAG application. + +{{< button text="Containerize a RAG app" url="containerize.md" >}} diff --git a/content/guides/use-case/rag-ollama/containerize.md b/content/guides/use-case/rag-ollama/containerize.md new file mode 100644 index 000000000000..4d06cd88f1c7 --- /dev/null +++ b/content/guides/use-case/rag-ollama/containerize.md @@ -0,0 +1,84 @@ +--- +title: Containerize a RAG application +keywords: python, generative ai, genai, llm, ollama, containerize, intitialize, qdrant +description: Learn how to containerize a RAG application. +--- + + +## Overview + +This section walks you through containerizing a RAG application using Docker. + +> **Note** +> +> You can see more samples of containerized GenAI applications in the [GenAI Stack](https://github.com/docker/genai-stack) demo applications. + +## Get the sample application + +The sample application used in this guide is an example of RAG application, made by three main components, which are the building blocks for every RAG application. A Large Language Model hosted somewhere, in this case it is hosted in a container and served via [Ollama](https://ollama.ai/). A vector database, [Qdrant](https://qdrant.tech/), to store the embeddings of local data, and a web application, using [Streamlit](https://streamlit.io/) to offer the best user experience to the user. + +Clone the sample application. Open a terminal, change directory to a directory that you want to work in, and run the following command to clone the repository: + +```console +$ git clone https://github.com/mfranzon/winy.git +``` + +You should now have the following files in your `winy` directory. + +```text +├── winy/ +│ ├── .gitignore +│ ├── app/ +│ │ ├── main.py +│ │ ├── Dockerfile +| | └── requirements.txt +│ ├── tools/ +│ │ ├── create_db.py +│ │ ├── create_embeddings.py +│ │ ├── requirements.txt +│ │ ├── test.py +| | └── download_model.sh +│ ├── docker-compose.yaml +│ ├── wine_database.db +│ ├── LICENSE +│ └── README.md +``` + +## Run the application + +Inside the `winy` directory, run the following command in a +terminal. + +```console +$ docker compose up --build +``` + +Docker builds and runs your application. Depending on your network connection, it may take several minutes to download all the dependencies. You'll see a message like the following in the terminal when the application is running. + +```console +server-1 | You can now view your Streamlit app in your browser. +server-1 | +server-1 | URL: http://0.0.0.0:8501 +server-1 | +``` + +Open a browser and view the application at [http://localhost:8501](http://localhost:8501). You should see a simple Streamlit application. + +The application requires a Qdrant database service and an LLM service to +function. If you have access to services that you ran outside of Docker, specify +the connection information and try it out. If you don't have the services +running, continue with this guide to learn how you can run some or all of these +services with Docker. + +In the terminal, press `ctrl`+`c` to stop the application. + +## Summary + +In this section, you learned how you can containerize and run your RAG +application using Docker. + +## Next steps + +In the next section, you'll learn how you can run your application, database, and LLM service all locally using Docker. + +{{< button text="Develop your application" url="develop.md" >}} diff --git a/content/guides/use-case/rag-ollama/develop.md b/content/guides/use-case/rag-ollama/develop.md new file mode 100644 index 000000000000..be198f470e16 --- /dev/null +++ b/content/guides/use-case/rag-ollama/develop.md @@ -0,0 +1,157 @@ +--- +title: Use containers for generative RAG development +keywords: python, local, development, generative ai, genai, llm, rag, ollama, langchain, openai +description: Learn how to develop your generative RAG application locally. +--- + +## Prerequisites + +Complete [Containerize a generative AI application](containerize.md). + +## Overview + +In this section, you'll learn how to set up a development environment to access all the services that your generative RAG application needs. This includes: + +- Adding a local database +- Adding a local or remote LLM service + +> **Note** +> +> You can see more samples of containerized GenAI applications in the [GenAI Stack](https://github.com/docker/genai-stack) demo applications. + +## Add a local database + +You can use containers to set up local services, like a database. In this section, you'll update the `compose.yaml` file to define a database service. In addition, you'll specify an environment variables file to load the database connection information rather than manually entering the information every time. + +To run the database service: + +1. In the cloned repository's directory, open the `compose.yaml` file in an IDE or text editor. + +2. In the `compose.yaml` file, add the following: + - Add instructions to run a Qdrant database + - Specify the volume and the port + + The following is the updated `compose.yaml` file. All comments have been removed. + + ```yaml{hl_lines=["3-10"]} + services: + qdrant: + image: qdrant/qdrant + container_name: qdrant + ports: + - "6333:6333" + volumes: + - qdrant_data:/qdrant/storage + ``` + + > **Note** + > + > To learn more about Qdrant, see the [Qdrant Official Docker Image](https://hub.docker.com/r/qdrant/qdrant). + +3. Run the application. Inside the `winy` directory, +run the following command in a terminal. + + ```console + $ docker compose up --build + ``` + +4. Access the application. Open a browser and view the application at [http://localhost:8501](http://localhost:8501). You should see a simple Streamlit application. + +5. Stop the application. In the terminal, press `ctrl`+`c` to stop the application. + +## Add a local or remote LLM service + +The sample application supports both [Ollama](https://ollama.ai/). This guide provides instructions for the following scenarios: +- Run Ollama in a container +- Run Ollama outside of a container + +While all platforms can use any of the previous scenarios, the performance and +GPU support may vary. You can use the following guidelines to help you choose the appropriate option: +- Run Ollama in a container if you're on Linux, and using a native installation of the Docker Engine, or Windows 10/11, and using Docker Desktop, you + have a CUDA-supported GPU, and your system has at least 8 GB of RAM. +- Run Ollama outside of a container on a Linux Machine. + +Choose one of the following options for your LLM service. + +{{< tabs >}} +{{< tab name="Run Ollama in a container" >}} + +When running Ollama in a container, you should have a CUDA-supported GPU. While you can run Ollama in a container without a supported GPU, the performance may not be acceptable. Only Linux and Windows 11 support GPU access to containers. + +To run Ollama in a container and provide GPU access: +1. Install the prerequisites. + - For Docker Engine on Linux, install the [NVIDIA Container Toolkilt](https://github.com/NVIDIA/nvidia-container-toolkit). + - For Docker Desktop on Windows 10/11, install the latest [NVIDIA driver](https://www.nvidia.com/Download/index.aspx) and make sure you are using the [WSL2 backend](../../../desktop/wsl/index.md/#turn-on-docker-desktop-wsl-2) +2. Add the Ollama service and a volume in your `compose.yaml`. The following is + the updated `compose.yaml`: + + ```yaml {hl_lines=["23-34"]} + ollama: + image: ollama/ollama + container_name: ollama + ports: + - "8000:8000" + deploy: + resources: + reservations: + devices: + - driver: nvidia + count: 1 + capabilities: [gpu] + ``` + + > **Note** + > + > For more details about the Compose instructions, see [Turn on GPU access with Docker Compose](../../../compose/gpu-support.md). + +3. Once the Ollama container is up and running it is possible to use the `download_model.sh` inside the `tools` folder with this command: +```console +. ./download_model.sh +``` +Pulling ollama model could take several minutes. + +{{< /tab >}} +{{< tab name="Run Ollama outside of a container" >}} + +To run Ollama outside of a container: +1. [Install](https://github.com/jmorganca/ollama) and run Ollama on your host + machine. +2. Pull the model to Ollama using the following command. + ```console + $ ollama pull llama2 + ``` + +{{< /tab >}} + +{{< /tabs >}} + +## Run your RAG application + +At this point, you have the following services in your Compose file: +- Server service for your main RAG application +- Database service to store vectors in a Qdrant database +- (optional) Ollama service to run the LLM +- (optional) Ollama-pull service to automatically pull the model for the Ollama + service + + +Once the application is running, open a browser and access the application at [http://localhost:8501](http://localhost:8501). + +Depending on your system and the LLM service that you chose, it may take several +minutes to answer. + +## Summary + +In this section, you learned how to set up a development environment to provide +access all the services that your GenAI application needs. + +Related information: + - [Dockerfile reference](../../../reference/dockerfile.md) + - [Compose file reference](../../../compose/compose-file/_index.md) + - [Ollama Docker image](https://hub.docker.com/r/ollama/ollama) + - [GenAI Stack demo applications](https://github.com/docker/genai-stack) + +## Next steps + +See samples of more GenAI applications in the [GenAI Stack demo applications](https://github.com/docker/genai-stack). + From 447aa411e96b35fbb7d94920f106f1670b51fa92 Mon Sep 17 00:00:00 2001 From: mfranzon Date: Wed, 10 Jul 2024 15:48:26 +0200 Subject: [PATCH 02/30] modify index --- content/guides/use-case/_index.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/content/guides/use-case/_index.md b/content/guides/use-case/_index.md index e8f55f27fa15..ce3143243a62 100644 --- a/content/guides/use-case/_index.md +++ b/content/guides/use-case/_index.md @@ -37,6 +37,10 @@ grid_genai: description: Explore an app that can summarize text. link: /guides/use-case/nlp/text-summarization/ icon: summarize +- title: RAG Ollama application + description: Explore how to containerize a RAG application + link: /guides/use-case/rag-ollama + icon: article --- Explore this collection of use-case guides designed to help you leverage Docker From fa2cb77c3cb9da04741a437d47c7395f6a60a875 Mon Sep 17 00:00:00 2001 From: mfranzon Date: Mon, 29 Jul 2024 10:45:53 +0200 Subject: [PATCH 03/30] add guide to _index and toc --- content/guides/use-case/_index.md | 2 +- data/toc.yaml | 8 ++++++++ 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/content/guides/use-case/_index.md b/content/guides/use-case/_index.md index ce3143243a62..3a2808f6faac 100644 --- a/content/guides/use-case/_index.md +++ b/content/guides/use-case/_index.md @@ -39,7 +39,7 @@ grid_genai: icon: summarize - title: RAG Ollama application description: Explore how to containerize a RAG application - link: /guides/use-case/rag-ollama + link: /guides/use-case/rag-ollama/ icon: article --- diff --git a/data/toc.yaml b/data/toc.yaml index 6b6c515f2a68..be383efa4b5b 100644 --- a/data/toc.yaml +++ b/data/toc.yaml @@ -212,6 +212,14 @@ Guides: title: Text classification - path: /guides/use-case/nlp/text-summarization/ title: Text summarization + - sectiontitle: RAG Ollama application + section: + - path: /guides/use-case/rag-ollama/ + title: Overview + - path: /guides/use-case/rag-ollama/containerize/ + title: Containerize your app + - path: /guides/use-case/rag-ollama/develop/ + title: Develop your app - path: /guides/use-case/jupyter/ title: Data science with JupyterLab - path: /scout/guides/vex/ From e4a0d7714260c021662b2f97f9bc09a98ef25f25 Mon Sep 17 00:00:00 2001 From: Marco Franzon <43796979+mfranzon@users.noreply.github.com> Date: Mon, 29 Jul 2024 11:18:28 +0200 Subject: [PATCH 04/30] Update content/guides/use-case/rag-ollama/_index.md Co-authored-by: Craig Osterhout <103533812+craig-osterhout@users.noreply.github.com> --- content/guides/use-case/rag-ollama/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/guides/use-case/rag-ollama/_index.md b/content/guides/use-case/rag-ollama/_index.md index 3bd3d8ffec08..104d82c15f1f 100644 --- a/content/guides/use-case/rag-ollama/_index.md +++ b/content/guides/use-case/rag-ollama/_index.md @@ -6,7 +6,7 @@ toc_min: 1 toc_max: 2 --- -The Retrival Augmented Generation (RAG) guide teaches you how to containerize an existing RAG application using Docker. The example application is a RAG that acts like a sommelier, giving you the best pairings between wines and food. In this guide, you’ll learn how to: +The Retrieval Augmented Generation (RAG) guide teaches you how to containerize an existing RAG application using Docker. The example application is a RAG that acts like a sommelier, giving you the best pairings between wines and food. In this guide, you’ll learn how to: * Containerize and run a RAG application * Set up a local environment to run the complete RAG stack locally for development From 8ab130acc9429a7812ea9ba27350f5b83656c1b3 Mon Sep 17 00:00:00 2001 From: Marco Franzon <43796979+mfranzon@users.noreply.github.com> Date: Mon, 29 Jul 2024 11:20:47 +0200 Subject: [PATCH 05/30] Update content/guides/use-case/rag-ollama/develop.md Co-authored-by: Craig Osterhout <103533812+craig-osterhout@users.noreply.github.com> --- content/guides/use-case/rag-ollama/develop.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/guides/use-case/rag-ollama/develop.md b/content/guides/use-case/rag-ollama/develop.md index be198f470e16..5df7c171e281 100644 --- a/content/guides/use-case/rag-ollama/develop.md +++ b/content/guides/use-case/rag-ollama/develop.md @@ -25,7 +25,7 @@ You can use containers to set up local services, like a database. In this sectio To run the database service: -1. In the cloned repository's directory, open the `compose.yaml` file in an IDE or text editor. +1. In the cloned repository's directory, open the `docker-compose.yaml` file in an IDE or text editor. 2. In the `compose.yaml` file, add the following: - Add instructions to run a Qdrant database From 9127fdbc89bb7ef7bd0c5650520e7402373fbd29 Mon Sep 17 00:00:00 2001 From: Marco Franzon <43796979+mfranzon@users.noreply.github.com> Date: Mon, 29 Jul 2024 11:21:10 +0200 Subject: [PATCH 06/30] Update content/guides/use-case/rag-ollama/develop.md Co-authored-by: Craig Osterhout <103533812+craig-osterhout@users.noreply.github.com> --- content/guides/use-case/rag-ollama/develop.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/guides/use-case/rag-ollama/develop.md b/content/guides/use-case/rag-ollama/develop.md index 5df7c171e281..e7d64d384f57 100644 --- a/content/guides/use-case/rag-ollama/develop.md +++ b/content/guides/use-case/rag-ollama/develop.md @@ -27,7 +27,7 @@ To run the database service: 1. In the cloned repository's directory, open the `docker-compose.yaml` file in an IDE or text editor. -2. In the `compose.yaml` file, add the following: +2. In the `docker-compose.yaml` file, you'll see the following: - Add instructions to run a Qdrant database - Specify the volume and the port From a6e596adb70e0e30b6175f4f368367b872aeca18 Mon Sep 17 00:00:00 2001 From: Marco Franzon <43796979+mfranzon@users.noreply.github.com> Date: Mon, 29 Jul 2024 11:21:51 +0200 Subject: [PATCH 07/30] Update content/guides/use-case/rag-ollama/develop.md Co-authored-by: Craig Osterhout <103533812+craig-osterhout@users.noreply.github.com> --- content/guides/use-case/rag-ollama/develop.md | 1 - 1 file changed, 1 deletion(-) diff --git a/content/guides/use-case/rag-ollama/develop.md b/content/guides/use-case/rag-ollama/develop.md index e7d64d384f57..269774f467a0 100644 --- a/content/guides/use-case/rag-ollama/develop.md +++ b/content/guides/use-case/rag-ollama/develop.md @@ -29,7 +29,6 @@ To run the database service: 2. In the `docker-compose.yaml` file, you'll see the following: - Add instructions to run a Qdrant database - - Specify the volume and the port The following is the updated `compose.yaml` file. All comments have been removed. From 43c013c88e83bea4ca82b9774e7ae5aca76f8271 Mon Sep 17 00:00:00 2001 From: Marco Franzon <43796979+mfranzon@users.noreply.github.com> Date: Mon, 29 Jul 2024 11:22:19 +0200 Subject: [PATCH 08/30] Update content/guides/use-case/rag-ollama/develop.md Co-authored-by: Craig Osterhout <103533812+craig-osterhout@users.noreply.github.com> --- content/guides/use-case/rag-ollama/develop.md | 1 - 1 file changed, 1 deletion(-) diff --git a/content/guides/use-case/rag-ollama/develop.md b/content/guides/use-case/rag-ollama/develop.md index 269774f467a0..0add83884eeb 100644 --- a/content/guides/use-case/rag-ollama/develop.md +++ b/content/guides/use-case/rag-ollama/develop.md @@ -30,7 +30,6 @@ To run the database service: 2. In the `docker-compose.yaml` file, you'll see the following: - Add instructions to run a Qdrant database - The following is the updated `compose.yaml` file. All comments have been removed. ```yaml{hl_lines=["3-10"]} services: From 6f5a99d79d39b72b46a55cef7f642bcd2422e517 Mon Sep 17 00:00:00 2001 From: Marco Franzon <43796979+mfranzon@users.noreply.github.com> Date: Mon, 29 Jul 2024 11:23:07 +0200 Subject: [PATCH 09/30] Update content/guides/use-case/rag-ollama/develop.md Co-authored-by: Craig Osterhout <103533812+craig-osterhout@users.noreply.github.com> --- content/guides/use-case/rag-ollama/develop.md | 1 - 1 file changed, 1 deletion(-) diff --git a/content/guides/use-case/rag-ollama/develop.md b/content/guides/use-case/rag-ollama/develop.md index 0add83884eeb..6f83c0dad814 100644 --- a/content/guides/use-case/rag-ollama/develop.md +++ b/content/guides/use-case/rag-ollama/develop.md @@ -129,7 +129,6 @@ At this point, you have the following services in your Compose file: - Server service for your main RAG application - Database service to store vectors in a Qdrant database - (optional) Ollama service to run the LLM -- (optional) Ollama-pull service to automatically pull the model for the Ollama service From 5378dee4ed1cf0a4541771f5b1a528dd1056b9f4 Mon Sep 17 00:00:00 2001 From: Marco Franzon <43796979+mfranzon@users.noreply.github.com> Date: Mon, 29 Jul 2024 11:23:27 +0200 Subject: [PATCH 10/30] Update content/guides/use-case/rag-ollama/develop.md Co-authored-by: Craig Osterhout <103533812+craig-osterhout@users.noreply.github.com> --- content/guides/use-case/rag-ollama/develop.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/guides/use-case/rag-ollama/develop.md b/content/guides/use-case/rag-ollama/develop.md index 6f83c0dad814..3f84da95dec3 100644 --- a/content/guides/use-case/rag-ollama/develop.md +++ b/content/guides/use-case/rag-ollama/develop.md @@ -81,7 +81,7 @@ To run Ollama in a container and provide GPU access: - For Docker Engine on Linux, install the [NVIDIA Container Toolkilt](https://github.com/NVIDIA/nvidia-container-toolkit). - For Docker Desktop on Windows 10/11, install the latest [NVIDIA driver](https://www.nvidia.com/Download/index.aspx) and make sure you are using the [WSL2 backend](../../../desktop/wsl/index.md/#turn-on-docker-desktop-wsl-2) 2. Add the Ollama service and a volume in your `compose.yaml`. The following is - the updated `compose.yaml`: + the updated `docker-compose.yaml`: ```yaml {hl_lines=["23-34"]} ollama: From 92bf8950e510c243c0e95559bd95f9c279d90a51 Mon Sep 17 00:00:00 2001 From: Marco Franzon <43796979+mfranzon@users.noreply.github.com> Date: Mon, 29 Jul 2024 11:23:41 +0200 Subject: [PATCH 11/30] Update content/guides/use-case/rag-ollama/develop.md Co-authored-by: Craig Osterhout <103533812+craig-osterhout@users.noreply.github.com> --- content/guides/use-case/rag-ollama/develop.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/guides/use-case/rag-ollama/develop.md b/content/guides/use-case/rag-ollama/develop.md index 3f84da95dec3..dcbd04dbb442 100644 --- a/content/guides/use-case/rag-ollama/develop.md +++ b/content/guides/use-case/rag-ollama/develop.md @@ -6,7 +6,7 @@ description: Learn how to develop your generative RAG application locally. ## Prerequisites -Complete [Containerize a generative AI application](containerize.md). +Complete [Containerize a RAG application](containerize.md). ## Overview From f3c7cdec20ec84f0a69661b8f575ce1832a056e5 Mon Sep 17 00:00:00 2001 From: Marco Franzon <43796979+mfranzon@users.noreply.github.com> Date: Mon, 29 Jul 2024 11:23:50 +0200 Subject: [PATCH 12/30] Update content/guides/use-case/rag-ollama/develop.md Co-authored-by: Craig Osterhout <103533812+craig-osterhout@users.noreply.github.com> --- content/guides/use-case/rag-ollama/develop.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/guides/use-case/rag-ollama/develop.md b/content/guides/use-case/rag-ollama/develop.md index dcbd04dbb442..f0a23734ce17 100644 --- a/content/guides/use-case/rag-ollama/develop.md +++ b/content/guides/use-case/rag-ollama/develop.md @@ -1,5 +1,5 @@ --- -title: Use containers for generative RAG development +title: Use containers for RAG development keywords: python, local, development, generative ai, genai, llm, rag, ollama, langchain, openai description: Learn how to develop your generative RAG application locally. --- From bd22b41867f89ee488f5e9704df49e9482ff1ee4 Mon Sep 17 00:00:00 2001 From: Marco Franzon <43796979+mfranzon@users.noreply.github.com> Date: Mon, 29 Jul 2024 11:24:07 +0200 Subject: [PATCH 13/30] Update content/guides/use-case/rag-ollama/develop.md Co-authored-by: Craig Osterhout <103533812+craig-osterhout@users.noreply.github.com> --- content/guides/use-case/rag-ollama/develop.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/guides/use-case/rag-ollama/develop.md b/content/guides/use-case/rag-ollama/develop.md index f0a23734ce17..252e3f305163 100644 --- a/content/guides/use-case/rag-ollama/develop.md +++ b/content/guides/use-case/rag-ollama/develop.md @@ -1,6 +1,6 @@ --- title: Use containers for RAG development -keywords: python, local, development, generative ai, genai, llm, rag, ollama, langchain, openai +keywords: python, local, development, generative ai, genai, llm, rag, ollama description: Learn how to develop your generative RAG application locally. --- From ad4757d08931f041d76fff166282e754ee435669 Mon Sep 17 00:00:00 2001 From: Marco Franzon <43796979+mfranzon@users.noreply.github.com> Date: Mon, 29 Jul 2024 11:25:19 +0200 Subject: [PATCH 14/30] Update content/guides/use-case/rag-ollama/develop.md Co-authored-by: Craig Osterhout <103533812+craig-osterhout@users.noreply.github.com> --- content/guides/use-case/rag-ollama/develop.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/guides/use-case/rag-ollama/develop.md b/content/guides/use-case/rag-ollama/develop.md index 252e3f305163..f6262c7a868f 100644 --- a/content/guides/use-case/rag-ollama/develop.md +++ b/content/guides/use-case/rag-ollama/develop.md @@ -80,7 +80,7 @@ To run Ollama in a container and provide GPU access: 1. Install the prerequisites. - For Docker Engine on Linux, install the [NVIDIA Container Toolkilt](https://github.com/NVIDIA/nvidia-container-toolkit). - For Docker Desktop on Windows 10/11, install the latest [NVIDIA driver](https://www.nvidia.com/Download/index.aspx) and make sure you are using the [WSL2 backend](../../../desktop/wsl/index.md/#turn-on-docker-desktop-wsl-2) -2. Add the Ollama service and a volume in your `compose.yaml`. The following is +2. The `docker-compose.yaml` file already contains the necessary instructions. In your own apps, you'll need to add the Ollama service in your `docker-compose.yaml`. The following is the updated `docker-compose.yaml`: ```yaml {hl_lines=["23-34"]} From 03926c0bffbe351737d57334444eb5bbc870161c Mon Sep 17 00:00:00 2001 From: Marco Franzon <43796979+mfranzon@users.noreply.github.com> Date: Mon, 29 Jul 2024 11:25:40 +0200 Subject: [PATCH 15/30] Update content/guides/use-case/rag-ollama/develop.md Co-authored-by: Craig Osterhout <103533812+craig-osterhout@users.noreply.github.com> --- content/guides/use-case/rag-ollama/develop.md | 1 - 1 file changed, 1 deletion(-) diff --git a/content/guides/use-case/rag-ollama/develop.md b/content/guides/use-case/rag-ollama/develop.md index f6262c7a868f..319648eadcc5 100644 --- a/content/guides/use-case/rag-ollama/develop.md +++ b/content/guides/use-case/rag-ollama/develop.md @@ -28,7 +28,6 @@ To run the database service: 1. In the cloned repository's directory, open the `docker-compose.yaml` file in an IDE or text editor. 2. In the `docker-compose.yaml` file, you'll see the following: - - Add instructions to run a Qdrant database ```yaml{hl_lines=["3-10"]} From 36ea5d4c5684ffc0cc6c230c198c9e029f25aa77 Mon Sep 17 00:00:00 2001 From: Marco Franzon <43796979+mfranzon@users.noreply.github.com> Date: Mon, 29 Jul 2024 11:26:15 +0200 Subject: [PATCH 16/30] Update content/guides/use-case/rag-ollama/develop.md Co-authored-by: Craig Osterhout <103533812+craig-osterhout@users.noreply.github.com> --- content/guides/use-case/rag-ollama/develop.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/guides/use-case/rag-ollama/develop.md b/content/guides/use-case/rag-ollama/develop.md index 319648eadcc5..9c74c4729b66 100644 --- a/content/guides/use-case/rag-ollama/develop.md +++ b/content/guides/use-case/rag-ollama/develop.md @@ -82,7 +82,7 @@ To run Ollama in a container and provide GPU access: 2. The `docker-compose.yaml` file already contains the necessary instructions. In your own apps, you'll need to add the Ollama service in your `docker-compose.yaml`. The following is the updated `docker-compose.yaml`: - ```yaml {hl_lines=["23-34"]} + ```yaml ollama: image: ollama/ollama container_name: ollama From 78e7b4582782d8541471b1a5979dede9d4d19624 Mon Sep 17 00:00:00 2001 From: Marco Franzon <43796979+mfranzon@users.noreply.github.com> Date: Mon, 29 Jul 2024 11:26:32 +0200 Subject: [PATCH 17/30] Update content/guides/use-case/rag-ollama/develop.md Co-authored-by: Craig Osterhout <103533812+craig-osterhout@users.noreply.github.com> --- content/guides/use-case/rag-ollama/develop.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/guides/use-case/rag-ollama/develop.md b/content/guides/use-case/rag-ollama/develop.md index 9c74c4729b66..81da9871df0f 100644 --- a/content/guides/use-case/rag-ollama/develop.md +++ b/content/guides/use-case/rag-ollama/develop.md @@ -30,7 +30,7 @@ To run the database service: 2. In the `docker-compose.yaml` file, you'll see the following: - ```yaml{hl_lines=["3-10"]} + ```yaml services: qdrant: image: qdrant/qdrant From fe41f128f0faf84157cd04c89c7af2e1c47a323e Mon Sep 17 00:00:00 2001 From: Marco Franzon <43796979+mfranzon@users.noreply.github.com> Date: Mon, 29 Jul 2024 11:27:51 +0200 Subject: [PATCH 18/30] Update content/guides/use-case/rag-ollama/develop.md Co-authored-by: Craig Osterhout <103533812+craig-osterhout@users.noreply.github.com> --- content/guides/use-case/rag-ollama/develop.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/guides/use-case/rag-ollama/develop.md b/content/guides/use-case/rag-ollama/develop.md index 81da9871df0f..326fc2a27594 100644 --- a/content/guides/use-case/rag-ollama/develop.md +++ b/content/guides/use-case/rag-ollama/develop.md @@ -105,7 +105,7 @@ To run Ollama in a container and provide GPU access: ```console . ./download_model.sh ``` -Pulling ollama model could take several minutes. +Pulling an Ollama model can take several minutes. {{< /tab >}} {{< tab name="Run Ollama outside of a container" >}} From fc94aed8f1641a3b131b7c584abf02b7eeebeb04 Mon Sep 17 00:00:00 2001 From: mfranzon Date: Fri, 2 Aug 2024 11:11:16 +0200 Subject: [PATCH 19/30] add essentials --- content/guides/use-case/rag-ollama/containerize.md | 8 ++++++++ content/guides/use-case/rag-ollama/develop.md | 2 +- 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/content/guides/use-case/rag-ollama/containerize.md b/content/guides/use-case/rag-ollama/containerize.md index 4d06cd88f1c7..71e13eb98b3e 100644 --- a/content/guides/use-case/rag-ollama/containerize.md +++ b/content/guides/use-case/rag-ollama/containerize.md @@ -44,6 +44,14 @@ You should now have the following files in your `winy` directory. │ └── README.md ``` +## Containerizing your application: Essentials + +Containerizing an application involves packaging it along with its dependencies into a container, which ensures consistency across different environments. Here’s what you need to containerize an app like Winy : + +1. Dockerfile: A Dockerfile that contains instructions on how to build a Docker image for your application. It specifies the base image, dependencies, configuration files, and the command to run your application. + +2. Docker Compose File: Docker Compose is a tool for defining and running multi-container Docker applications. A Compose file, typically named docker-compose.yml, allows you to configure your application's services, networks, and volumes in a single file. + ## Run the application Inside the `winy` directory, run the following command in a diff --git a/content/guides/use-case/rag-ollama/develop.md b/content/guides/use-case/rag-ollama/develop.md index 326fc2a27594..ff1aa1e4f120 100644 --- a/content/guides/use-case/rag-ollama/develop.md +++ b/content/guides/use-case/rag-ollama/develop.md @@ -21,7 +21,7 @@ In this section, you'll learn how to set up a development environment to access ## Add a local database -You can use containers to set up local services, like a database. In this section, you'll update the `compose.yaml` file to define a database service. In addition, you'll specify an environment variables file to load the database connection information rather than manually entering the information every time. +You can use containers to set up local services, like a database. In this section, you'll update the `docker-compose.yaml` file to define a database service. In addition, you'll specify an environment variables file to load the database connection information rather than manually entering the information every time. To run the database service: From 227551f0fc66249949931c341b0b4770dde28a17 Mon Sep 17 00:00:00 2001 From: mfranzon Date: Fri, 2 Aug 2024 12:10:10 +0200 Subject: [PATCH 20/30] add missing info about env variables --- .../use-case/rag-ollama/containerize.md | 27 ++++++++++++++----- content/guides/use-case/rag-ollama/develop.md | 8 +++++- 2 files changed, 28 insertions(+), 7 deletions(-) diff --git a/content/guides/use-case/rag-ollama/containerize.md b/content/guides/use-case/rag-ollama/containerize.md index 71e13eb98b3e..afd3f3f1e61b 100644 --- a/content/guides/use-case/rag-ollama/containerize.md +++ b/content/guides/use-case/rag-ollama/containerize.md @@ -72,11 +72,26 @@ server-1 | Open a browser and view the application at [http://localhost:8501](http://localhost:8501). You should see a simple Streamlit application. -The application requires a Qdrant database service and an LLM service to -function. If you have access to services that you ran outside of Docker, specify -the connection information and try it out. If you don't have the services -running, continue with this guide to learn how you can run some or all of these -services with Docker. +The application requires a Qdrant database service and an LLM service to work properly. If you have access to services that you ran outside of Docker, specify the connection information in the `docker-compose.yaml`. + +```yaml +winy: +build: + context: ./app + dockerfile: Dockerfile +environment: + - QDRANT_CLIENT=http://qdrant:6333 # Specifies the url for the qdrant database + - OLLAMA=http://ollama:11434 # Specifies the url for the ollama service +container_name: winy +ports: + - "8501:8501" +depends_on: + - qdrant + - ollama +``` + +If you don't have the services running, continue with this guide to learn how you can run some or all of these services with Docker. +Remember that the `ollama` service is empty, it has not any model, for this reason you had to pull a model before starting to use the RAG application. All the instruction are in the following page! In the terminal, press `ctrl`+`c` to stop the application. @@ -87,6 +102,6 @@ application using Docker. ## Next steps -In the next section, you'll learn how you can run your application, database, and LLM service all locally using Docker. +In the next section, you'll learn how to properly configure the application with your preferred LLM model, completely locally, using Docker. {{< button text="Develop your application" url="develop.md" >}} diff --git a/content/guides/use-case/rag-ollama/develop.md b/content/guides/use-case/rag-ollama/develop.md index ff1aa1e4f120..404301f45af7 100644 --- a/content/guides/use-case/rag-ollama/develop.md +++ b/content/guides/use-case/rag-ollama/develop.md @@ -21,7 +21,7 @@ In this section, you'll learn how to set up a development environment to access ## Add a local database -You can use containers to set up local services, like a database. In this section, you'll update the `docker-compose.yaml` file to define a database service. In addition, you'll specify an environment variables file to load the database connection information rather than manually entering the information every time. +You can use containers to set up local services, like a database. In this section, you'll explore the database service in the `docker-compose.yaml` file. To run the database service: @@ -117,6 +117,12 @@ To run Ollama outside of a container: ```console $ ollama pull llama2 ``` +3. Remove the `ollama` service from the `docker-compose.yaml` and update properly the connection variables in `winy` service: + +```diff +- OLLAMA=http://ollama:11434 ++ OLLAMA= +``` {{< /tab >}} From b0aca89e39e74e51353c3e8d112079d8e51645ea Mon Sep 17 00:00:00 2001 From: Marco Franzon <43796979+mfranzon@users.noreply.github.com> Date: Wed, 4 Sep 2024 16:32:36 +0200 Subject: [PATCH 21/30] Update content/guides/use-case/_index.md Co-authored-by: Craig Osterhout <103533812+craig-osterhout@users.noreply.github.com> --- content/guides/use-case/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/guides/use-case/_index.md b/content/guides/use-case/_index.md index 3a2808f6faac..5691bae1cf66 100644 --- a/content/guides/use-case/_index.md +++ b/content/guides/use-case/_index.md @@ -38,7 +38,7 @@ grid_genai: link: /guides/use-case/nlp/text-summarization/ icon: summarize - title: RAG Ollama application - description: Explore how to containerize a RAG application + description: Explore how to containerize a RAG application. link: /guides/use-case/rag-ollama/ icon: article --- From 43b9dd2aae2adbb1549667bd233044968ffc941a Mon Sep 17 00:00:00 2001 From: Marco Franzon <43796979+mfranzon@users.noreply.github.com> Date: Wed, 4 Sep 2024 16:32:46 +0200 Subject: [PATCH 22/30] Update content/guides/use-case/rag-ollama/containerize.md Co-authored-by: Craig Osterhout <103533812+craig-osterhout@users.noreply.github.com> --- content/guides/use-case/rag-ollama/containerize.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/guides/use-case/rag-ollama/containerize.md b/content/guides/use-case/rag-ollama/containerize.md index afd3f3f1e61b..d54c5890cb00 100644 --- a/content/guides/use-case/rag-ollama/containerize.md +++ b/content/guides/use-case/rag-ollama/containerize.md @@ -50,7 +50,7 @@ Containerizing an application involves packaging it along with its dependencies 1. Dockerfile: A Dockerfile that contains instructions on how to build a Docker image for your application. It specifies the base image, dependencies, configuration files, and the command to run your application. -2. Docker Compose File: Docker Compose is a tool for defining and running multi-container Docker applications. A Compose file, typically named docker-compose.yml, allows you to configure your application's services, networks, and volumes in a single file. +2. Docker Compose File: Docker Compose is a tool for defining and running multi-container Docker applications. A Compose file allows you to configure your application's services, networks, and volumes in a single file. ## Run the application From 1d99c9c19f3824e8bd66d303a4acfb427768ca3f Mon Sep 17 00:00:00 2001 From: Marco Franzon <43796979+mfranzon@users.noreply.github.com> Date: Wed, 4 Sep 2024 16:32:57 +0200 Subject: [PATCH 23/30] Update content/guides/use-case/rag-ollama/containerize.md Co-authored-by: Craig Osterhout <103533812+craig-osterhout@users.noreply.github.com> --- content/guides/use-case/rag-ollama/containerize.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/guides/use-case/rag-ollama/containerize.md b/content/guides/use-case/rag-ollama/containerize.md index d54c5890cb00..208e1d3d3062 100644 --- a/content/guides/use-case/rag-ollama/containerize.md +++ b/content/guides/use-case/rag-ollama/containerize.md @@ -91,7 +91,7 @@ depends_on: ``` If you don't have the services running, continue with this guide to learn how you can run some or all of these services with Docker. -Remember that the `ollama` service is empty, it has not any model, for this reason you had to pull a model before starting to use the RAG application. All the instruction are in the following page! +Remember that the `ollama` service is empty; it doesn't have any model. For this reason you need to pull a model before starting to use the RAG application. All the instructions are in the following page. In the terminal, press `ctrl`+`c` to stop the application. From f6e87d29a2174c3ec2beb498aff23103caa3f69d Mon Sep 17 00:00:00 2001 From: Marco Franzon <43796979+mfranzon@users.noreply.github.com> Date: Wed, 4 Sep 2024 16:33:09 +0200 Subject: [PATCH 24/30] Update content/guides/use-case/rag-ollama/develop.md Co-authored-by: Craig Osterhout <103533812+craig-osterhout@users.noreply.github.com> --- content/guides/use-case/rag-ollama/develop.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/guides/use-case/rag-ollama/develop.md b/content/guides/use-case/rag-ollama/develop.md index 404301f45af7..b53526f9ba49 100644 --- a/content/guides/use-case/rag-ollama/develop.md +++ b/content/guides/use-case/rag-ollama/develop.md @@ -66,7 +66,7 @@ While all platforms can use any of the previous scenarios, the performance and GPU support may vary. You can use the following guidelines to help you choose the appropriate option: - Run Ollama in a container if you're on Linux, and using a native installation of the Docker Engine, or Windows 10/11, and using Docker Desktop, you have a CUDA-supported GPU, and your system has at least 8 GB of RAM. -- Run Ollama outside of a container on a Linux Machine. +- Run Ollama outside of a container if running Docker Desktop on a Linux Machine. Choose one of the following options for your LLM service. From 40c12cfa8125d5ca980e45927739ee7b64c3a9c9 Mon Sep 17 00:00:00 2001 From: Marco Franzon <43796979+mfranzon@users.noreply.github.com> Date: Wed, 4 Sep 2024 16:33:20 +0200 Subject: [PATCH 25/30] Update content/guides/use-case/rag-ollama/containerize.md Co-authored-by: Craig Osterhout <103533812+craig-osterhout@users.noreply.github.com> --- content/guides/use-case/rag-ollama/containerize.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/guides/use-case/rag-ollama/containerize.md b/content/guides/use-case/rag-ollama/containerize.md index 208e1d3d3062..e16784027734 100644 --- a/content/guides/use-case/rag-ollama/containerize.md +++ b/content/guides/use-case/rag-ollama/containerize.md @@ -83,7 +83,7 @@ environment: - QDRANT_CLIENT=http://qdrant:6333 # Specifies the url for the qdrant database - OLLAMA=http://ollama:11434 # Specifies the url for the ollama service container_name: winy -ports: + ports: - "8501:8501" depends_on: - qdrant From 86147cf858fb6d5af2fb653c1818ce966330b0f0 Mon Sep 17 00:00:00 2001 From: Marco Franzon <43796979+mfranzon@users.noreply.github.com> Date: Wed, 4 Sep 2024 16:33:29 +0200 Subject: [PATCH 26/30] Update content/guides/use-case/rag-ollama/containerize.md Co-authored-by: Craig Osterhout <103533812+craig-osterhout@users.noreply.github.com> --- content/guides/use-case/rag-ollama/containerize.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/guides/use-case/rag-ollama/containerize.md b/content/guides/use-case/rag-ollama/containerize.md index e16784027734..114c2faa2c93 100644 --- a/content/guides/use-case/rag-ollama/containerize.md +++ b/content/guides/use-case/rag-ollama/containerize.md @@ -79,7 +79,7 @@ winy: build: context: ./app dockerfile: Dockerfile -environment: + environment: - QDRANT_CLIENT=http://qdrant:6333 # Specifies the url for the qdrant database - OLLAMA=http://ollama:11434 # Specifies the url for the ollama service container_name: winy From 181c0c01b83209816c4afe8cf3095a5ab24d59d1 Mon Sep 17 00:00:00 2001 From: Marco Franzon <43796979+mfranzon@users.noreply.github.com> Date: Wed, 4 Sep 2024 16:33:40 +0200 Subject: [PATCH 27/30] Update content/guides/use-case/rag-ollama/develop.md Co-authored-by: Craig Osterhout <103533812+craig-osterhout@users.noreply.github.com> --- content/guides/use-case/rag-ollama/develop.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/guides/use-case/rag-ollama/develop.md b/content/guides/use-case/rag-ollama/develop.md index b53526f9ba49..c683bb31416e 100644 --- a/content/guides/use-case/rag-ollama/develop.md +++ b/content/guides/use-case/rag-ollama/develop.md @@ -102,7 +102,7 @@ To run Ollama in a container and provide GPU access: > For more details about the Compose instructions, see [Turn on GPU access with Docker Compose](../../../compose/gpu-support.md). 3. Once the Ollama container is up and running it is possible to use the `download_model.sh` inside the `tools` folder with this command: -```console + ```console . ./download_model.sh ``` Pulling an Ollama model can take several minutes. From 90d3357e5a9e614fa8cf8e1732321fd594c98302 Mon Sep 17 00:00:00 2001 From: Marco Franzon <43796979+mfranzon@users.noreply.github.com> Date: Wed, 4 Sep 2024 16:33:52 +0200 Subject: [PATCH 28/30] Update content/guides/use-case/rag-ollama/containerize.md Co-authored-by: Craig Osterhout <103533812+craig-osterhout@users.noreply.github.com> --- content/guides/use-case/rag-ollama/containerize.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/guides/use-case/rag-ollama/containerize.md b/content/guides/use-case/rag-ollama/containerize.md index 114c2faa2c93..06f8a9345a19 100644 --- a/content/guides/use-case/rag-ollama/containerize.md +++ b/content/guides/use-case/rag-ollama/containerize.md @@ -85,7 +85,7 @@ build: container_name: winy ports: - "8501:8501" -depends_on: + depends_on: - qdrant - ollama ``` From b2ea91fc28f091f2d83ccda32e5bbbc3745b39a2 Mon Sep 17 00:00:00 2001 From: Marco Franzon <43796979+mfranzon@users.noreply.github.com> Date: Wed, 4 Sep 2024 16:38:43 +0200 Subject: [PATCH 29/30] Update content/guides/use-case/rag-ollama/containerize.md Co-authored-by: Craig Osterhout <103533812+craig-osterhout@users.noreply.github.com> --- content/guides/use-case/rag-ollama/containerize.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/guides/use-case/rag-ollama/containerize.md b/content/guides/use-case/rag-ollama/containerize.md index 06f8a9345a19..e6e35baa465b 100644 --- a/content/guides/use-case/rag-ollama/containerize.md +++ b/content/guides/use-case/rag-ollama/containerize.md @@ -76,7 +76,7 @@ The application requires a Qdrant database service and an LLM service to work pr ```yaml winy: -build: + build: context: ./app dockerfile: Dockerfile environment: From 9b28afbabd83914afe1e936abdc8ca024bc48334 Mon Sep 17 00:00:00 2001 From: Marco Franzon <43796979+mfranzon@users.noreply.github.com> Date: Wed, 4 Sep 2024 16:39:04 +0200 Subject: [PATCH 30/30] Update content/guides/use-case/rag-ollama/containerize.md Co-authored-by: Craig Osterhout <103533812+craig-osterhout@users.noreply.github.com> --- content/guides/use-case/rag-ollama/containerize.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/guides/use-case/rag-ollama/containerize.md b/content/guides/use-case/rag-ollama/containerize.md index e6e35baa465b..683fcecd0996 100644 --- a/content/guides/use-case/rag-ollama/containerize.md +++ b/content/guides/use-case/rag-ollama/containerize.md @@ -82,7 +82,7 @@ winy: environment: - QDRANT_CLIENT=http://qdrant:6333 # Specifies the url for the qdrant database - OLLAMA=http://ollama:11434 # Specifies the url for the ollama service -container_name: winy + container_name: winy ports: - "8501:8501" depends_on: