Can't run paddlex commands in Dockerfiles

[mattiasarro]

I have the following Dockerfile, which aims to install PaddleOCR plugin as part of the image:

FROM ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlex/paddlex:paddlex3.0.0rc0-paddlepaddle3.0.0rc0-gpu-cuda12.3-cudnn9.0-trt8.6

WORKDIR /paddle

# install PaddleX plugin PaddleOCR
RUN git clone https://github.com/PaddlePaddle/PaddleX.git
RUN pip install -e PaddleX
RUN paddlex --install PaddleOCR

which fails on the line paddlex --install PaddleOCR because it can't find CUDA:

109.6 Error: Can not import paddle core while this file exists: /usr/local/lib/python3.10/dist-packages/paddle/base/libpaddle.so
109.6 Traceback (most recent call last):
109.6   File "/usr/local/bin/paddlex", line 8, in <module>
109.6     sys.exit(console_entry())
109.6   File "/paddle/PaddleX/paddlex/__main__.py", line 26, in console_entry
109.6     main()
109.6   File "/paddle/PaddleX/paddlex/paddlex_cli.py", line 439, in main
109.6     install(args)
109.6   File "/paddle/PaddleX/paddlex/paddlex_cli.py", line 306, in install
109.6     setup(
109.6   File "/paddle/PaddleX/paddlex/repo_manager/core.py", line 202, in setup
109.6     installer.install(
109.6   File "/paddle/PaddleX/paddlex/repo_manager/repo.py", line 282, in install
109.6     repo.install_packages()
109.6   File "/paddle/PaddleX/paddlex/repo_manager/repo.py", line 138, in install_packages
109.6     install_external_deps(self.name, self.root_dir)
109.6   File "/paddle/PaddleX/paddlex/repo_manager/utils.py", line 71, in install_external_deps
109.6     import paddle
109.6   File "/usr/local/lib/python3.10/dist-packages/paddle/__init__.py", line 38, in <module>
109.6     from .base import core  # noqa: F401
109.6   File "/usr/local/lib/python3.10/dist-packages/paddle/base/__init__.py", line 38, in <module>
109.6     from . import (  # noqa: F401
109.6   File "/usr/local/lib/python3.10/dist-packages/paddle/base/backward.py", line 28, in <module>
109.6     from . import core, framework, log_helper, unique_name
109.6   File "/usr/local/lib/python3.10/dist-packages/paddle/base/core.py", line 387, in <module>
109.6     raise e
109.6   File "/usr/local/lib/python3.10/dist-packages/paddle/base/core.py", line 267, in <module>
109.6     from . import libpaddle
109.6 ImportError: libcuda.so.1: cannot open shared object file: No such file or directory
------
Dockerfile.paddle:8
--------------------
   6 |     RUN git clone https://github.com/PaddlePaddle/PaddleX.git
   7 |     RUN pip install -e PaddleX
   8 | >>> RUN paddlex --install PaddleOCR
   9 |
  10 |     RUN mkdir -p /root/.paddlex/official_models
--------------------
ERROR: failed to solve: process "/bin/sh -c paddlex --install PaddleOCR" did not complete successfully: exit code: 1

However, paddlex commands work fine inside running containers. For example if I run docker run --gpus all -it ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlex/paddlex:paddlex3.0.0rc0-paddlepaddle3.0.0rc0-gpu-cuda12.3-cudnn9.0-trt8.6 bash and inside the running container, the below works:

git clone https://github.com/PaddlePaddle/PaddleX.git
pip install -e PaddleX
paddlex --install PaddleOCR

The problem is that paddlex commands access CUDA, which is not available at Docker build time. But it makes sense to use paddlex commands (such as paddlex --install PaddleOCR) to install plugins, such that I can build a base image which would be used for training models. Otherwise I will need to run paddlex --install PaddleOCR every time I start a new container, which is time-consuming.

Are there plans to support paddlex commands as part of Dockerfiles?

(BTW this is a duplicate issue of 2071, but it was never properly resolved)

[changdazhou]

Execute the nvidia-smi command to check if your driver supports CUDA 12.3.

[mattiasarro]

It has:

| NVIDIA-SMI 550.144.03             Driver Version: 550.144.03     CUDA Version: 12.4     |

Which should be compatible, otherwise it would throw a similar error at container run time, where I can successfully run paddlex commands. The error only occurs at Docker build time.

[changdazhou]

The reason for this problem is that when you install and build the docker image, you don't have a real CUDA dependency. Our official approach is to first copy the local CUDA dependency to /lib/x86_64-linux-gnu/ libcuda.so-1, and then delete it after the Docker is built. Just add two lines of command:

COPY libcuda.so.xxx.xx.x /lib/x86_64-linux-gnu/libcuda.so.1

#...
# Your Commands
#...

RUN rm -rf /lib/x86_64-linux-gnu/libcuda.so.1

In addition, under the /root of the official image, there is already installed PaddleX, which can be used directly

[mattiasarro]

Thanks, this approach works.