Skip to content

Commit

Permalink
[Doc][Serving]serving doc update version to 1.0.0 (#755)
Browse files Browse the repository at this point in the history
serving doc update version to 1.0.0
  • Loading branch information
heliqi committed Nov 30, 2022
1 parent 8399092 commit e79dc86
Show file tree
Hide file tree
Showing 10 changed files with 35 additions and 25 deletions.
6 changes: 3 additions & 3 deletions examples/text/ernie-3.0/serving/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,10 +51,10 @@ models
# GPU镜像
docker pull paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10
# CPU镜像
docker pull paddlepaddle/fastdeploy:z.y.z-cpu-only-21.10
docker pull paddlepaddle/fastdeploy:x.y.z-cpu-only-21.10

# 运行
docker run -it --net=host --name fastdeploy_server --shm-size="1g" -v /path/serving/models:/models paddlepaddle/fastdeploy:0.6.0-cpu-only-21.10 bash
docker run -it --net=host --name fastdeploy_server --shm-size="1g" -v /path/serving/models:/models paddlepaddle/fastdeploy:x.y.z-cpu-only-21.10 bash
```

## 部署模型
Expand All @@ -67,7 +67,7 @@ token_cls_rpc_client.py # 序列标注任务发送pipeline预测请求的脚
```

*注意*:启动服务时,Server的每个python后端进程默认申请`64M`内存,默认启动的docker无法启动多个python后端节点。有两个解决方案:
- 1.启动容器时设置`shm-size`参数, 比如:`docker run -it --net=host --name fastdeploy_server --shm-size="1g" -v /path/serving/models:/models paddlepaddle/fastdeploy:0.6.0-gpu-cuda11.4-trt8.4-21.10 bash`
- 1.启动容器时设置`shm-size`参数, 比如:`docker run -it --net=host --name fastdeploy_server --shm-size="1g" -v /path/serving/models:/models paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10 bash`
- 2.启动服务时设置python后端的`shm-default-byte-size`参数, 设置python后端的默认内存为10M: `tritonserver --model-repository=/models --backend-config=python,shm-default-byte-size=10485760`

### 分类任务
Expand Down
4 changes: 2 additions & 2 deletions examples/text/uie/serving/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,10 +34,10 @@ models
# GPU镜像
docker pull paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10
# CPU镜像
docker pull paddlepaddle/fastdeploy:z.y.z-cpu-only-21.10
docker pull paddlepaddle/fastdeploy:x.y.z-cpu-only-21.10

# 运行容器.容器名字为 fd_serving, 并挂载当前目录为容器的 /uie_serving 目录
docker run -it --net=host --name fastdeploy_server --shm-size="1g" -v `pwd`/:/uie_serving paddlepaddle/fastdeploy:0.6.0-gpu-cuda11.4-trt8.4-21.10 bash
docker run -it --net=host --name fastdeploy_server --shm-size="1g" -v `pwd`/:/uie_serving paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10 bash

# 启动服务(不设置CUDA_VISIBLE_DEVICES环境变量,会拥有所有GPU卡的调度权限)
CUDA_VISIBLE_DEVICES=0 fastdeployserver --model-repository=/uie_serving/models --backend-config=python,shm-default-byte-size=10485760
Expand Down
4 changes: 2 additions & 2 deletions examples/vision/classification/paddleclas/serving/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,10 @@ mv ResNet50_vd_infer/inference.pdiparams models/runtime/1/model.pdiparams
# GPU镜像
docker pull paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10
# CPU镜像
docker pull paddlepaddle/fastdeploy:z.y.z-cpu-only-21.10
docker pull paddlepaddle/fastdeploy:x.y.z-cpu-only-21.10

# 运行容器.容器名字为 fd_serving, 并挂载当前目录为容器的 /serving 目录
nvidia-docker run -it --net=host --name fd_serving -v `pwd`/:/serving paddlepaddle/fastdeploy:0.6.0-gpu-cuda11.4-trt8.4-21.10 bash
nvidia-docker run -it --net=host --name fd_serving -v `pwd`/:/serving paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10 bash

# 启动服务(不设置CUDA_VISIBLE_DEVICES环境变量,会拥有所有GPU卡的调度权限)
CUDA_VISIBLE_DEVICES=0 fastdeployserver --model-repository=/serving/models --backend-config=python,shm-default-byte-size=10485760
Expand Down
4 changes: 2 additions & 2 deletions examples/vision/detection/yolov5/serving/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,10 @@ mv yolov5s.onnx models/runtime/1/model.onnx
# GPU镜像
docker pull paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10
# CPU镜像
docker pull paddlepaddle/fastdeploy:z.y.z-cpu-only-21.10
docker pull paddlepaddle/fastdeploy:x.y.z-cpu-only-21.10

# 运行容器.容器名字为 fd_serving, 并挂载当前目录为容器的 /yolov5_serving 目录
nvidia-docker run -it --net=host --name fd_serving -v `pwd`/:/yolov5_serving paddlepaddle/fastdeploy:0.6.0-gpu-cuda11.4-trt8.4-21.10 bash
nvidia-docker run -it --net=host --name fd_serving -v `pwd`/:/yolov5_serving paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10 bash

# 启动服务(不设置CUDA_VISIBLE_DEVICES环境变量,会拥有所有GPU卡的调度权限)
CUDA_VISIBLE_DEVICES=0 fastdeployserver --model-repository=/yolov5_serving/models --backend-config=python,shm-default-byte-size=10485760
Expand Down
6 changes: 3 additions & 3 deletions examples/vision/detection/yolov5/serving/README_EN.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,11 @@ wget https://bj.bcebos.com/paddlehub/fastdeploy/yolov5s.onnx
# Save the model under models/infer/1 and rename it as model.onnx
mv yolov5s.onnx models/infer/1/

# Pull fastdeploy image
docker pull paddlepaddle/fastdeploy:0.6.0-gpu-cuda11.4-trt8.4-21.10
# Pull fastdeploy image, x.y.z is FastDeploy version, example 1.0.0.
docker pull paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10

# Run the docker. The docker name is fd_serving, and the current directory is mounted as the docker's /yolov5_serving directory
nvidia-docker run -it --net=host --name fd_serving -v `pwd`/:/yolov5_serving paddlepaddle/fastdeploy:0.6.0-gpu-cuda11.4-trt8.4-21.10 bash
nvidia-docker run -it --net=host --name fd_serving -v `pwd`/:/yolov5_serving paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10 bash

# Start the service (Without setting the CUDA_VISIBLE_DEVICES environment variable, it will have scheduling privileges for all GPU cards)
CUDA_VISIBLE_DEVICES=0 fastdeployserver --model-repository=models --backend-config=python,shm-default-byte-size=10485760
Expand Down
10 changes: 7 additions & 3 deletions examples/vision/ocr/PP-OCRv3/serving/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# PP-OCR服务化部署示例

在服务化部署前,需确认

- 1. 服务化镜像的软硬件环境要求和镜像拉取命令请参考[FastDeploy服务化部署](../../../../../serving/README_CN.md)

## 介绍
本文介绍了使用FastDeploy搭建OCR文字识别服务的方法.

Expand Down Expand Up @@ -48,9 +52,9 @@ mv ppocr_keys_v1.txt models/rec_postprocess/1/

wget https://gitee.com/paddlepaddle/PaddleOCR/raw/release/2.6/doc/imgs/12.jpg


docker pull paddlepaddle/fastdeploy:0.6.0-gpu-cuda11.4-trt8.4-21.10
docker run -dit --net=host --name fastdeploy --shm-size="1g" -v $PWD:/ocr_serving paddlepaddle/fastdeploy:0.6.0-gpu-cuda11.4-trt8.4-21.10 bash
# x.y.z为镜像版本号,需参照serving文档替换为数字
docker pull paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10
docker run -dit --net=host --name fastdeploy --shm-size="1g" -v $PWD:/ocr_serving paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10 bash
docker exec -it -u root fastdeploy bash
```

Expand Down
4 changes: 2 additions & 2 deletions serving/README_CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,13 @@ FastDeploy基于[Triton Inference Server](https://github.com/triton-inference-se
#### CPU镜像
CPU镜像仅支持Paddle/ONNX模型在CPU上进行服务化部署,支持的推理后端包括OpenVINO、Paddle Inference和ONNX Runtime
``` shell
docker pull paddlepaddle/fastdeploy:0.6.0-cpu-only-21.10
docker pull paddlepaddle/fastdeploy:1.0.0-cpu-only-21.10
```

#### GPU镜像
GPU镜像支持Paddle/ONNX模型在GPU/CPU上进行服务化部署,支持的推理后端包括OpenVINO、TensorRT、Paddle Inference和ONNX Runtime
```
docker pull paddlepaddle/fastdeploy:0.6.0-gpu-cuda11.4-trt8.4-21.10
docker pull paddlepaddle/fastdeploy:1.0.0-gpu-cuda11.4-trt8.4-21.10
```

用户也可根据自身需求,参考如下文档自行编译镜像
Expand Down
4 changes: 2 additions & 2 deletions serving/README_EN.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,15 +20,15 @@ FastDeploy builds an end-to-end serving deployment based on [Triton Inference Se
CPU images only support Paddle/ONNX models for serving deployment on CPUs, and supported inference backends include OpenVINO, Paddle Inference, and ONNX Runtime

```shell
docker pull paddlepaddle/fastdeploy:0.6.0-cpu-only-21.10
docker pull paddlepaddle/fastdeploy:1.0.0-cpu-only-21.10
```

#### GPU Image

GPU images support Paddle/ONNX models for serving deployment on GPU and CPU, and supported inference backends including OpenVINO, TensorRT, Paddle Inference, and ONNX Runtime

```
docker pull paddlepaddle/fastdeploy:0.6.0-gpu-cuda11.4-trt8.4-21.10
docker pull paddlepaddle/fastdeploy:1.0.0-gpu-cuda11.4-trt8.4-21.10
```

Users can also compile the image by themselves according to their own needs, referring to the following documents:
Expand Down
9 changes: 6 additions & 3 deletions serving/docs/EN/compile-en.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,9 @@ cd serving
bash scripts/build.sh

# Exit to the FastDeploy home directory and create the image
# x.y.z is FastDeploy version, example: 1.0.0
cd ../
docker build -t paddlepaddle/fastdeploy:0.6.0-gpu-cuda11.4-trt8.4-21.10 -f serving/Dockerfile .
docker build -t paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10 -f serving/Dockerfile .
```

## CPU Image
Expand All @@ -25,8 +26,9 @@ cd serving
bash scripts/build.sh OFF

# Exit to the FastDeploy home directory and create the image
# x.y.z is FastDeploy version, example: 1.0.0
cd ../
docker build -t paddlepaddle/fastdeploy:0.6.0-cpu-only-21.10 -f serving/Dockerfile_cpu .
docker build -t paddlepaddle/fastdeploy:x.y.z-cpu-only-21.10 -f serving/Dockerfile_cpu .
```

## IPU Image
Expand All @@ -37,6 +39,7 @@ cd serving
bash scripts/build_fd_ipu.sh

# Exit to the FastDeploy home directory and create the image
# x.y.z is FastDeploy version, example: 1.0.0
cd ../
docker build -t paddlepaddle/fastdeploy:0.6.0-ipu-only-21.10 -f serving/Dockerfile_ipu .
docker build -t paddlepaddle/fastdeploy:x.y.z-ipu-only-21.10 -f serving/Dockerfile_ipu .
```
9 changes: 6 additions & 3 deletions serving/docs/zh_CN/compile.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,9 @@ cd serving
bash scripts/build.sh
# 退出到FastDeploy主目录,制作镜像
# x.y.z为FastDeploy版本号,可根据情况自己确定。比如: 1.0.0
cd ../
docker build -t paddlepaddle/fastdeploy:0.6.0-gpu-cuda11.4-trt8.4-21.10 -f serving/Dockerfile .
docker build -t paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10 -f serving/Dockerfile .
```

## 制作CPU镜像
Expand All @@ -24,8 +25,9 @@ cd serving
bash scripts/build.sh OFF
# 退出到FastDeploy主目录,制作镜像
# x.y.z为FastDeploy版本号,可根据情况自己确定。比如: 1.0.0
cd ../
docker build -t paddlepaddle/fastdeploy:0.6.0-cpu-only-21.10 -f serving/Dockerfile_cpu .
docker build -t paddlepaddle/fastdeploy:x.y.z-cpu-only-21.10 -f serving/Dockerfile_cpu .
```

## 制作IPU镜像
Expand All @@ -36,6 +38,7 @@ cd serving
bash scripts/build_fd_ipu.sh
# 退出到FastDeploy主目录,制作镜像
# x.y.z为FastDeploy版本号,可根据情况自己确定。比如: 1.0.0
cd ../
docker build -t paddlepaddle/fastdeploy:0.6.0-ipu-only-21.10 -f serving/Dockerfile_ipu .
docker build -t paddlepaddle/fastdeploy:x.y.z-ipu-only-21.10 -f serving/Dockerfile_ipu .
```

0 comments on commit e79dc86

Please sign in to comment.