Skip to content

Commit 00241d0

Browse files
Update README.md for quick start guide (#1355)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
1 parent ed2b8ed commit 00241d0

File tree

1 file changed

+81
-2
lines changed
  • FaqGen/docker_compose/intel/hpu/gaudi

1 file changed

+81
-2
lines changed

FaqGen/docker_compose/intel/hpu/gaudi/README.md

Lines changed: 81 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,85 @@
22

33
This document outlines the deployment process for a FAQ Generation application utilizing the [GenAIComps](https://github.com/opea-project/GenAIComps.git) microservice pipeline on Intel Gaudi server. The steps include Docker image creation, container deployment via Docker Compose, and service execution to integrate microservices such as llm. We will publish the Docker images to Docker Hub, which will simplify the deployment process for this service.
44

5+
## Quick Start:
6+
7+
1. Set up the environment variables.
8+
2. Run Docker Compose.
9+
3. Consume the ChatQnA Service.
10+
11+
### Quick Start: 1.Setup Environment Variable
12+
13+
To set up environment variables for deploying ChatQnA services, follow these steps:
14+
15+
1. Set the required environment variables:
16+
17+
```bash
18+
# Example: host_ip="192.168.1.1"
19+
export host_ip=$(hostname -I | awk '{print $1}')
20+
export HUGGINGFACEHUB_API_TOKEN="Your_Huggingface_API_Token"
21+
```
22+
23+
2. If you are in a proxy environment, also set the proxy-related environment variables:
24+
25+
```bash
26+
export http_proxy="Your_HTTP_Proxy"
27+
export https_proxy="Your_HTTPs_Proxy"
28+
# Example: no_proxy="localhost, 127.0.0.1,192.168.1.1"
29+
export no_proxy="localhost, 127.0.0.1,192.168.1.1, ${host_ip}"
30+
```
31+
32+
3. Set up other environment variables:
33+
34+
```bash
35+
export LLM_MODEL_ID="meta-llama/Meta-Llama-3-8B-Instruct"
36+
export TGI_LLM_ENDPOINT="http://${host_ip}:8008"
37+
export MEGA_SERVICE_HOST_IP=${host_ip}
38+
export LLM_SERVICE_HOST_IP=${host_ip}
39+
export LLM_SERVICE_PORT=9000
40+
export BACKEND_SERVICE_ENDPOINT="http://${host_ip}:8888/v1/faqgen"
41+
```
42+
43+
### Quick Start: 2.Run Docker Compose
44+
45+
```bash
46+
docker compose up -d
47+
```
48+
49+
It will automatically download the docker image on `docker hub`, please check the images' status by the commands
50+
51+
```bash
52+
docker ps -a
53+
docker logs tgi-gaudi-server -t
54+
```
55+
56+
it may take some time to download the model.
57+
In following cases, you could build docker image from source by yourself.
58+
59+
- Failed to download the docker image.
60+
61+
- If you want to use a specific version of Docker image.
62+
63+
Please refer to 'Build Docker Images' in below.
64+
65+
### QuickStart: 3.Consume the Service
66+
67+
```bash
68+
curl localhost:8008/generate \
69+
-X POST \
70+
-d '{"inputs":"What is Deep Learning?","parameters":{"max_new_tokens":64, "do_sample": true}}' \
71+
-H 'Content-Type: application/json'
72+
```
73+
74+
here we just test the service on the host machine for a quick start.
75+
If all networks work fine, please try
76+
77+
```bash
78+
curl http://${host_ip}:8008/generate \
79+
-X POST \
80+
-d '{"inputs":"What is Deep Learning?","parameters":{"max_new_tokens":64, "do_sample": true}}' \
81+
-H 'Content-Type: application/json'
82+
```
83+
584
## 🚀 Build Docker Images
685

786
First of all, you need to build Docker Images locally. This step can be ignored once the Docker images are published to Docker hub.
@@ -76,7 +155,7 @@ export no_proxy=${your_no_proxy}
76155
export http_proxy=${your_http_proxy}
77156
export https_proxy=${your_http_proxy}
78157
export LLM_MODEL_ID="meta-llama/Meta-Llama-3-8B-Instruct"
79-
export TGI_LLM_ENDPOINT="http://${your_ip}:8008"
158+
export TGI_LLM_ENDPOINT="http://${host_ip}:8008"
80159
export HUGGINGFACEHUB_API_TOKEN=${your_hf_api_token}
81160
export MEGA_SERVICE_HOST_IP=${host_ip}
82161
export LLM_SERVICE_HOST_IP=${host_ip}
@@ -98,7 +177,7 @@ docker compose up -d
98177
1. TGI Service
99178

100179
```bash
101-
curl http://${your_ip}:8008/generate \
180+
curl http://${host_ip}:8008/generate \
102181
-X POST \
103182
-d '{"inputs":"What is Deep Learning?","parameters":{"max_new_tokens":64, "do_sample": true}}' \
104183
-H 'Content-Type: application/json'

0 commit comments

Comments
 (0)