Skip to content

Commit 2a2ff45

Browse files
letonghanpre-commit-ci[bot]
letonghan
and
pre-commit-ci[bot]
authored
Explain Default Model in ChatQnA and CodeTrans READMEs (#694)
* explain default model in CodeTrans READMEs Signed-off-by: letonghan <letong.han@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * explain default model in ChatQnA READMEs Signed-off-by: letonghan <letong.han@intel.com> * add required models Signed-off-by: letonghan <letong.han@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: letonghan <letong.han@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
1 parent 6a679ba commit 2a2ff45

File tree

10 files changed

+112
-5
lines changed

10 files changed

+112
-5
lines changed

‎ChatQnA/README.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -121,6 +121,18 @@ Currently we support two ways of deploying ChatQnA services with docker compose:
121121

122122
2. Start services using the docker images `built from source`: [Guide](./docker)
123123

124+
### Required Models
125+
126+
By default, the embedding, reranking and LLM models are set to a default value as listed below:
127+
128+
| Service | Model |
129+
| --------- | ------------------------- |
130+
| Embedding | BAAI/bge-base-en-v1.5 |
131+
| Reranking | BAAI/bge-reranker-base |
132+
| LLM | Intel/neural-chat-7b-v3-3 |
133+
134+
Change the `xxx_MODEL_ID` in `docker/xxx/set_env.sh` for your needs.
135+
124136
### Setup Environment Variable
125137

126138
To set up environment variables for deploying ChatQnA services, follow these steps:

‎ChatQnA/docker/gaudi/README.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -159,6 +159,18 @@ If Guardrails docker image is built, you will find one more image:
159159

160160
## 🚀 Start MicroServices and MegaService
161161

162+
### Required Models
163+
164+
By default, the embedding, reranking and LLM models are set to a default value as listed below:
165+
166+
| Service | Model |
167+
| --------- | ------------------------- |
168+
| Embedding | BAAI/bge-base-en-v1.5 |
169+
| Reranking | BAAI/bge-reranker-base |
170+
| LLM | Intel/neural-chat-7b-v3-3 |
171+
172+
Change the `xxx_MODEL_ID` below for your needs.
173+
162174
### Setup Environment Variables
163175

164176
Since the `compose.yaml` will consume some environment variables, you need to setup them in advance as below.

‎ChatQnA/docker/gpu/README.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -87,6 +87,18 @@ Then run the command `docker images`, you will have the following 7 Docker Image
8787

8888
## 🚀 Start MicroServices and MegaService
8989

90+
### Required Models
91+
92+
By default, the embedding, reranking and LLM models are set to a default value as listed below:
93+
94+
| Service | Model |
95+
| --------- | ------------------------- |
96+
| Embedding | BAAI/bge-base-en-v1.5 |
97+
| Reranking | BAAI/bge-reranker-base |
98+
| LLM | Intel/neural-chat-7b-v3-3 |
99+
100+
Change the `xxx_MODEL_ID` below for your needs.
101+
90102
### Setup Environment Variables
91103

92104
Since the `compose.yaml` will consume some environment variables, you need to setup them in advance as below.

‎ChatQnA/docker/xeon/README.md

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -161,6 +161,18 @@ Then run the command `docker images`, you will have the following 7 Docker Image
161161

162162
## 🚀 Start Microservices
163163

164+
### Required Models
165+
166+
By default, the embedding, reranking and LLM models are set to a default value as listed below:
167+
168+
| Service | Model |
169+
| --------- | ------------------------- |
170+
| Embedding | BAAI/bge-base-en-v1.5 |
171+
| Reranking | BAAI/bge-reranker-base |
172+
| LLM | Intel/neural-chat-7b-v3-3 |
173+
174+
Change the `xxx_MODEL_ID` below for your needs.
175+
164176
### Setup Environment Variables
165177

166178
Since the `compose.yaml` will consume some environment variables, you need to setup them in advance as below.
@@ -183,7 +195,7 @@ export your_hf_api_token="Your_Huggingface_API_Token"
183195

184196
**Append the value of the public IP address to the no_proxy list**
185197

186-
```
198+
```bash
187199
export your_no_proxy=${your_no_proxy},"External_Public_IP"
188200
```
189201

‎ChatQnA/docker/xeon/README_qdrant.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -148,6 +148,18 @@ Then run the command `docker images`, you will have the following 7 Docker Image
148148

149149
## 🚀 Start Microservices
150150

151+
### Required Models
152+
153+
By default, the embedding, reranking and LLM models are set to a default value as listed below:
154+
155+
| Service | Model |
156+
| --------- | ------------------------- |
157+
| Embedding | BAAI/bge-base-en-v1.5 |
158+
| Reranking | BAAI/bge-reranker-base |
159+
| LLM | Intel/neural-chat-7b-v3-3 |
160+
161+
Change the `xxx_MODEL_ID` below for your needs.
162+
151163
### Setup Environment Variables
152164

153165
Since the `compose.yaml` will consume some environment variables, you need to setup them in advance as below.

‎CodeTrans/README.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,16 @@ Currently we support two ways of deploying Code Translation services on docker:
2222

2323
2. Start services using the docker images `built from source`: [Guide](./docker)
2424

25+
### Required Models
26+
27+
By default, the LLM model is set to a default value as listed below:
28+
29+
| Service | Model |
30+
| ------- | ----------------------------- |
31+
| LLM | HuggingFaceH4/mistral-7b-grok |
32+
33+
Change the `LLM_MODEL_ID` in `docker/set_env.sh` for your needs.
34+
2535
### Setup Environment Variable
2636

2737
To set up environment variables for deploying Code Translation services, follow these steps:

‎CodeTrans/docker/gaudi/README.md

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -42,9 +42,17 @@ Then run the command `docker images`, you will have the following Docker Images:
4242

4343
## 🚀 Start Microservices
4444

45-
### Setup Environment Variables
45+
### Required Models
46+
47+
By default, the LLM model is set to a default value as listed below:
48+
49+
| Service | Model |
50+
| ------- | ----------------------------- |
51+
| LLM | HuggingFaceH4/mistral-7b-grok |
4652

47-
Since the `compose.yaml` will consume some environment variables, you need to setup them in advance as below. Notice that the `LLM_MODEL_ID` indicates the LLM model used for TGI service.
53+
Change the `LLM_MODEL_ID` below for your needs.
54+
55+
### Setup Environment Variables
4856

4957
```bash
5058
export no_proxy=${your_no_proxy}

‎CodeTrans/docker/xeon/README.md

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -50,9 +50,17 @@ Then run the command `docker images`, you will have the following Docker Images:
5050

5151
## 🚀 Start Microservices
5252

53-
### Setup Environment Variables
53+
### Required Models
54+
55+
By default, the LLM model is set to a default value as listed below:
56+
57+
| Service | Model |
58+
| ------- | ----------------------------- |
59+
| LLM | HuggingFaceH4/mistral-7b-grok |
5460

55-
Since the `compose.yaml` will consume some environment variables, you need to setup them in advance as below. Notice that the `LLM_MODEL_ID` indicates the LLM model used for TGI service.
61+
Change the `LLM_MODEL_ID` below for your needs.
62+
63+
### Setup Environment Variables
5664

5765
```bash
5866
export no_proxy=${your_no_proxy}

‎CodeTrans/kubernetes/README.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,9 +7,20 @@ Please install GMC in your Kubernetes cluster, if you have not already done so,
77
If you have only Intel Xeon machines you could use the codetrans_xeon.yaml file or if you have a Gaudi cluster you could use codetrans_gaudi.yaml
88
In the below example we illustrate on Xeon.
99

10+
## Required Models
11+
12+
By default, the LLM model is set to a default value as listed below:
13+
14+
|Service |Model |
15+
|---------|-------------------------|
16+
|LLM |HuggingFaceH4/mistral-7b-grok|
17+
18+
Change the `MODEL_ID` in `codetrans_xeon.yaml` for your needs.
19+
1020
## Deploy the RAG application
1121

1222
1. Create the desired namespace if it does not already exist and deploy the application
23+
1324
```bash
1425
export APP_NAMESPACE=CT
1526
kubectl create ns $APP_NAMESPACE

‎CodeTrans/kubernetes/manifests/README.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,16 @@
88
99
> You need to make sure you have created the directory `/mnt/opea-models` to save the cached model on the node where the CodeTrans workload is running. Otherwise, you need to modify the `codetrans.yaml` file to change the `model-volume` to a directory that exists on the node.
1010
11+
## Required Models
12+
13+
By default, the LLM model is set to a default value as listed below:
14+
15+
|Service |Model |
16+
|---------|-------------------------|
17+
|LLM |HuggingFaceH4/mistral-7b-grok|
18+
19+
Change the `MODEL_ID` in `codetrans.yaml` for your needs.
20+
1121
## Deploy On Xeon
1222

1323
```bash

0 commit comments

Comments
 (0)