-
Notifications
You must be signed in to change notification settings - Fork 21
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
chore(docs): add helm chart deployment instructions (#45)
Add deployment instructions for 1. Docker compose 2. Helm Chart 3. Ametnes Platform Closes #25
- Loading branch information
Showing
15 changed files
with
377 additions
and
96 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,43 @@ | ||
| # Ametnes Platform | ||
|
|
||
| The Ametnes Platform helps orchestrate your business applications wherever you host them. This can be in your private | ||
| data-center, in AWS, GCP or Azure. | ||
|
|
||
| Nesis is available on the Ametnes Platform and can be deployed in your kubernetes cluster wherever you host it. | ||
|
|
||
| The first step is to setup your kubernetes cluster as an Ametnes Application Location. See these detailed <a href="https://cloud.ametnes.com/docs/concepts/data-service-location/" target="_blank">instructions</a>. | ||
|
|
||
| ## Create the service | ||
|
|
||
| Log into your Ametnes Cloud console at <a href="https://cloud.ametnes.com/console/signin" target="_blank">here</a> | ||
| or sign up <a href="https://cloud.ametnes.com/console/signup" target="_blank">here</a> if you do not have one. | ||
|
|
||
| 1. Using the **Services** left menu, navigate to the service management dashboard. | ||
| 2. Click **New Service**. | ||
| 3. Enter the **__Nesis__** to filter from the list and select **__Create__** | ||
| 4. In the displayed form, enter the following info. | ||
| 1. Enter the **Name**: `Nesis-Service-DSL1` and **Description**: `Nesis-Service-DSL1`. | ||
| 2. Select a **Version** from the list. | ||
| 3. Select the **Location**. | ||
| 4. Set the `OPENAI_API_KEY` and the `HF_TOKEN` keys. | ||
| 4. Click `Create`. | ||
|
|
||
| ## Test connectivity | ||
|
|
||
| 1. Using the **Services** left menu, navigate to the service management dashboard. Your service should be listed. | ||
|
|
||
| !!! note "Service not showing" | ||
| If your service list is empty, use the filter at the top right corner, to filter for ALL services. | ||
|
|
||
| 2. After a while, your data service status will change to `ready`. | ||
| 2. To the right of your service, click the `Admin` button and you will navigate to your service's details page. | ||
| 3. At the bottom of the page, copy the endpoint of your service as well as your username/key and password. | ||
| 4. In your browser, paste the URL `https://<your.instance.host.name>/`. | ||
| 5. You should get a prompt to login. | ||
|
|
||
| ## Clean up | ||
|
|
||
| ### Delete all services | ||
| 1. In your Ametnes Cloud console, navigate to the **Admin** section of each service | ||
| 2. Delete the service. | ||
|
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,164 @@ | ||
| # Docker Compose | ||
| For your quick local test, a <a href="https://github.com/ametnes/nesis/blob/main/compose.yml" target="_blank">docker compose</a> file is provided which you can | ||
| run to standup a local instance of Nesis. | ||
|
|
||
| ## Compose File | ||
|
|
||
| ```yaml title="compose.yml" linenums="1" | ||
|
|
||
| version: '3' | ||
|
|
||
|
|
||
| networks: | ||
| nesis: | ||
| driver: overlay | ||
| attachable: true | ||
|
|
||
| services: | ||
|
|
||
| nesis_frontend: | ||
| image: ametnes/nesis:latest-frontend | ||
| ports: | ||
| - "58000:8000" | ||
| environment: | ||
| API_URL: http://nesis_api:6000 | ||
| networks: | ||
| - nesis | ||
| depends_on: | ||
| - nesis_api | ||
| nesis_api: | ||
| image: ametnes/nesis:latest-api | ||
| ports: | ||
| - "56000:6000" | ||
| environment: | ||
| NESIS_API_PORT: "6000" | ||
| NESIS_API_DATABASE_URL: "postgresql://postgres:password@database:5432/nesis" | ||
| NESIS_API_TASKS_JOB_STORES_URL: "postgresql://postgres:password@database:5432/nesis" | ||
| NESIS_ADMIN_EMAIL: "some.email@domain.com" | ||
| NESIS_ADMIN_PASSWORD: "password" | ||
| NESIS_MEMCACHE_HOSTS: memcached:11211 | ||
| NESIS_API_RAG_ENDPOINT: http://nesis_rag:8080 | ||
| networks: | ||
| - nesis | ||
| depends_on: | ||
| database: | ||
| condition: service_healthy | ||
| links: | ||
| - database | ||
| - memcached | ||
| - nesis_rag | ||
| nesis_rag: | ||
| image: ametnes/nesis:latest-rag | ||
| ports: | ||
| - "58080:8080" | ||
| environment: | ||
| NESIS_RAG_SERVER_PORT: "8080" | ||
| NESIS_RAG_PGVECTOR_URL: postgresql://postgres:password@database:5432/nesis | ||
| HF_TOKEN: <your-huggingface-token> | ||
| # 1. local mode uses hugging face. Other options | ||
| NESIS_RAG_EMBEDDING_MODE: local | ||
| NESIS_RAG_EMBEDDING_DIMENSIONS: "384" | ||
|
|
||
| # 2. openai - for OpenAI Embeddings | ||
| # NESIS_RAG_EMBEDDING_MODE: openai | ||
| # NESIS_RAG_EMBEDDING_DIMENSIONS: "1536" | ||
|
|
||
| # 3. sagemaker - for Sagemaker | ||
| OPENAI_API_KEY: <your-api-key> | ||
| # OPENAI_API_BASE: <your-api-base> | ||
| networks: | ||
| - nesis | ||
| depends_on: | ||
| - database | ||
| memcached: | ||
| image: bitnami/memcached:1.6.19 | ||
| ports: | ||
| - "11211:11211" | ||
| networks: | ||
| - nesis | ||
| samba: | ||
| image: andyzhangx/samba:win-fix | ||
| command: ["-u", "username;password", "-s", "share;/smbshare/;yes;no;no;all;none", "-p"] | ||
| ports: | ||
| - '2445:445' | ||
| networks: | ||
| - nesis | ||
| volumes: | ||
| - 'samba_data2:/smbshare' | ||
| environment: | ||
| - USERNAME=username | ||
| - PASSWORD=password | ||
| minio: | ||
| image: docker.io/bitnami/minio:2022 | ||
| ports: | ||
| - '59000:9000' | ||
| - '59001:9001' | ||
| networks: | ||
| - nesis | ||
| volumes: | ||
| - 'minio_data:/data' | ||
| environment: | ||
| - MINIO_ROOT_USER=your_username | ||
| - MINIO_ROOT_PASSWORD=your_password | ||
| - MINIO_DEFAULT_BUCKETS=documents | ||
| database: | ||
| image: "ametnes/postgresql:16-debian-12" | ||
| ports: | ||
| - "65432:5432" | ||
| environment: | ||
| POSTGRES_USER: postgres | ||
| POSTGRES_PASSWORD: password | ||
| POSTGRES_DB: nesis | ||
| volumes: | ||
| - database_data:/var/lib/postgresql/data | ||
| restart: on-failure | ||
| healthcheck: | ||
| test: ["CMD-SHELL", "pg_isready -U postgres"] | ||
| interval: 5s | ||
| timeout: 5s | ||
| retries: 5 | ||
| networks: | ||
| - nesis | ||
| volumes: | ||
| minio_data: | ||
| samba_data2: | ||
| database_data: | ||
|
|
||
| ``` | ||
|
|
||
| ## Steps | ||
|
|
||
| 1. Obtain your **OPENAI_API_KEY** from https://platform.openai.com/api-keys: | ||
| - and update the `compose.yml` file entry. | ||
|
|
||
| 2. Start all services locally with the provided docker compose file. | ||
|
|
||
| ```commandline | ||
| docker-compose -f compose.yml up | ||
| ``` | ||
|
|
||
| 2. Then connect to your instance via http://localhost:58000 with the following login credentials: | ||
| - *email* = `some.email@domain.com` | ||
| - *password* = `password` | ||
|
|
||
| 3. Connect to your minio instance via http://localhost:59001/ with the following login credentials: | ||
| - *username* = `YOUR_USERNAME` | ||
| - *password* = `YOUR_PASSWORD` | ||
|
|
||
|
|
||
| 4. Upload some documents into your minio `documents` bucket. | ||
|
|
||
| 5. Back on your Nesis page, register the minio datasource with | ||
| 1. Navigate to **Settings** -> **Datasource** -> **Add** | ||
| 2. Enter the details; | ||
|
|
||
| 1. Type: **S3 Compatible** | ||
| 4. Name: **documents** | ||
| 5. Host: **http://minio:9000/** | ||
| 6. Username: `YOUR_USERNAME` | ||
| 7. Password: `YOUR_PASSWORD` | ||
| 8. Click **Create** | ||
| 9. Then, run an adhoc ingestion by clicking the **Ingest** button of the datasource. | ||
|
|
||
| - *Note*: Replace `YOUR_USERNAME` and `YOUR_PASSWORD` with the correct values of your `username` and `password`. | ||
|
|
Oops, something went wrong.