How to deploy PaddleOCR using triton?

[bdeng3]

Hey,

I'm struggling with how to deploy PaddleOCR in triton serving.

I understand I could use either Triton Server Paddlebackend. However, if I understand it right, it is not able to support PaddleOCR as a whole (det->cls->rec). For sure I could upload only one model, say the detection model, to a triton server of PaddleBackend. But that means I have to serve three models (det, cls , rec) separately and connect them together manually.

I also thought about using Triton Server Pythonbackend and package a conda environment with PaddleOCR into the server. However, the triton server won't be able unpack the environment as long as there is PaddlePaddle package in it.

Is there any other way we could deploy PaddleOCR as a whole using triton?

[HexToString]

First, triton can support PaddleOCR as a whole (det->cls->rec).
You can refer to the url:
https://github.com/PaddlePaddle/PaddleNLP/tree/develop/model_zoo/ernie-3.0/deploy/triton

Triton support several backend or platform. You can just install your python package inside the triton docker.

If you want to use the PaddlePaddle to load your model, you can also use the Triton-Paddlepaddle docker(which is now not an official version).
Pull the docker use
docker pull paddlepaddle/triton_paddle:21.10

[bdeng3]

Thanks for your reply, really appreciate it.

I was wondering if, by any chance, you also try preparing PaddlePaddle environment inside triton docker (python backend) using the suggested way to package conda environment?

I've tried couple of times preparing the environment with or without PaddlePaddle. Every time I added PaddlePaddle package in the conda environment package, Triton server fails to unpack it. As long as I removed Paddlepaddle, the environment was set up successfully. Did anyone get similar experience like I did?

[tink2123]

It is recommended to use pip to install directly, not conda.