Skip to content

Commit 0a6bad0

Browse files
add k8s support for audioqna (#583)
* add k8s support for audioqna * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
1 parent 4f7fc39 commit 0a6bad0

File tree

3 files changed

+866
-0
lines changed

3 files changed

+866
-0
lines changed
Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
# Deploy VisualQnA in a Kubernetes Cluster
2+
3+
> [NOTE]
4+
> The following values must be set before you can deploy:
5+
> HUGGINGFACEHUB_API_TOKEN
6+
> You can also customize the "MODEL_ID" and "model-volume"
7+
8+
## Deploy On Xeon
9+
```
10+
cd GenAIExamples/AudioQnA/kubernetes/manifests/xeon
11+
export HUGGINGFACEHUB_API_TOKEN="YourOwnToken"
12+
sed -i "s/insert-your-huggingface-token-here/${HUGGINGFACEHUB_API_TOKEN}/g" audioqna.yaml
13+
kubectl apply -f audioqna.yaml
14+
```
15+
## Deploy On Gaudi
16+
```
17+
cd GenAIExamples/AudioQnA/kubernetes/manifests/gaudi
18+
export HUGGINGFACEHUB_API_TOKEN="YourOwnToken"
19+
sed -i "s/insert-your-huggingface-token-here/${HUGGINGFACEHUB_API_TOKEN}/g" audioqna.yaml
20+
kubectl apply -f audioqna.yaml
21+
```
22+
23+
24+
## Verify Services
25+
26+
Make sure all the pods are running, and restart the audioqna-xxxx pod if necessary.
27+
28+
```bash
29+
kubectl get pods
30+
31+
curl http://${host_ip}:3008/v1/audioqna -X POST -d '{"audio": "UklGRigAAABXQVZFZm10IBIAAAABAAEARKwAAIhYAQACABAAAABkYXRhAgAAAAEA", "max_tokens":64}' -H 'Content-Type: application/json'
32+
```

0 commit comments

Comments
 (0)