How to avoid installation of problematic dependecies if running with remote llama.cpp GPU server

[elfarolab]

Hello,

I am trying to understand how to get a clean installation without unneccessary packages into my virtualenv.

I have a setup with 2 servers:

• A is with AMD CPU to run docling
• B is an Nvidia Jetson ORIN running llama.cpp connected via LAN.

During the installation of docling, pip installs a lot of dependencies including some packages that will never be able to run onto server A:

pip install docling

``` ... Using cached docling-2.120.1-py3-none-any.whl (5.2 kB) Using cached docling_slim-2.120.1-py3-none-any.whl (729 kB) Using cached accelerate-1.14.0-py3-none-any.whl (389 kB) Using cached beautifulsoup4-4.15.0-py3-none-any.whl (109 kB) Using cached defusedxml-0.7.1-py2.py3-none-any.whl (25 kB) Using cached docling_core-2.91.0-py3-none-any.whl (286 kB) Using cached doclang-0.7.3-py3-none-any.whl (32 kB) Using cached docling_ibm_models-3.14.0-py3-none-any.whl (94 kB) Using cached docling_parse-7.13.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (10.6 MB) Using cached filetype-1.2.0-py2.py3-none-any.whl (19 kB) Using cached httpx-0.28.1-py3-none-any.whl (73 kB) Using cached httpcore-1.0.9-py3-none-any.whl (78 kB) Using cached huggingface_hub-1.27.0-py3-none-any.whl (784 kB) Using cached click-8.4.2-py3-none-any.whl (119 kB) Using cached hf_xet-1.6.0-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (4.5 MB) Using cached jsonlines-4.0.0-py3-none-any.whl (8.7 kB) Using cached jsonref-1.1.0-py3-none-any.whl (9.4 kB) Using cached jsonschema-4.26.0-py3-none-any.whl (90 kB) Using cached latex2mathml-3.81.0-py3-none-any.whl (79 kB) Using cached mail_parser-4.6.2-py3-none-any.whl (46 kB) Using cached marko-2.2.4-py3-none-any.whl (46 kB) Using cached numpy-2.5.2-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (16.7 MB) Using cached openpyxl-3.1.5-py2.py3-none-any.whl (250 kB) Using cached pandas-3.0.5-cp314-cp314-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (11.0 MB) Using cached pillow-12.3.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (6.9 MB) Using cached pluggy-1.6.0-py3-none-any.whl (20 kB) Using cached pydantic-2.13.4-py3-none-any.whl (472 kB) Using cached pydantic_core-2.46.4-cp314-cp314-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.1 MB) Using cached pydantic_settings-2.15.0-py3-none-any.whl (69 kB) Using cached pylatexenc-2.11-py2.py3-none-any.whl (137 kB) Using cached pypdfium2-5.13.0-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.7 MB) Using cached python_docx-1.2.0-py3-none-any.whl (252 kB) Using cached python_dotenv-1.2.3-py3-none-any.whl (22 kB) Using cached python_oxmsg-0.0.2-py3-none-any.whl (31 kB) Using cached python_pptx-1.0.2-py3-none-any.whl (472 kB) Using cached pyyaml-6.0.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (794 kB) Using cached rapidocr-3.9.2-py3-none-any.whl (27.3 MB) Using cached requests-2.34.2-py3-none-any.whl (73 kB) Using cached charset_normalizer-3.5.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (251 kB) Using cached idna-3.18-py3-none-any.whl (65 kB) Using cached rtree-1.4.1-py3-none-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (507 kB) Using cached safetensors-0.8.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (516 kB) Using cached scipy-1.18.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (35.3 MB) Using cached semchunk-3.2.5-py3-none-any.whl (13 kB) Using cached tabulate-0.10.0-py3-none-any.whl (39 kB) Using cached torch-2.13.0-cp314-cp314-manylinux_2_28_x86_64.whl (526.6 MB) Using cached cuda_toolkit-13.0.3.0-py2.py3-none-any.whl (2.5 kB) Using cached nvidia_cudnn_cu13-9.20.0.48-py3-none-manylinux_2_27_x86_64.whl (366.2 MB) Using cached nvidia_cusparselt_cu13-0.8.1-py3-none-manylinux2014_x86_64.whl (170.1 MB) Using cached nvidia_nccl_cu13-2.29.7-py3-none-manylinux_2_18_x86_64.whl (206.0 MB) Using cached nvidia_nvshmem_cu13-3.4.5-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (60.4 MB) Using cached triton-3.7.1-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (197.7 MB) Using cached cuda_bindings-13.3.1-cp314-cp314-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (6.5 MB) Downloading nvidia_cublas-13.1.1.3-py3-none-manylinux_2_27_x86_64.whl (423.1 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 423.1/423.1 MB 74.5 MB/s 0:00:05 Downloading nvidia_cuda_cupti-13.0.85-py3-none-manylinux_2_25_x86_64.whl (10.7 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.7/10.7 MB 80.3 MB/s 0:00:00 Downloading nvidia_cuda_nvrtc-13.0.88-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl (90.2 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 90.2/90.2 MB 82.8 MB/s 0:00:01 Downloading nvidia_cuda_runtime-13.0.96-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (2.2 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.2/2.2 MB 87.8 MB/s 0:00:00 Downloading nvidia_cufft-12.0.0.61-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (214.1 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 214.1/214.1 MB 79.8 MB/s 0:00:02 Downloading nvidia_cufile-1.15.1.6-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (1.2 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 7.1 MB/s 0:00:00 Downloading nvidia_curand-10.4.0.35-py3-none-manylinux_2_27_x86_64.whl (59.5 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 59.5/59.5 MB 69.9 MB/s 0:00:00 Downloading nvidia_cusolver-12.0.4.66-py3-none-manylinux_2_27_x86_64.whl (200.9 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 200.9/200.9 MB 80.6 MB/s 0:00:02 Downloading nvidia_cusparse-12.6.3.3-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (145.9 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 145.9/145.9 MB 68.4 MB/s 0:00:02 Downloading nvidia_nvjitlink-13.3.33-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl (40.7 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.7/40.7 MB 73.5 MB/s 0:00:00 Downloading nvidia_nvtx-13.0.85-py3-none-manylinux1_x86_64.manylinux_2_5_x86_64.whl (148 kB) Downloading torchvision-0.28.0-cp314-cp314-manylinux_2_28_x86_64.whl (7.7 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.7/7.7 MB 69.5 MB/s 0:00:00 Downloading tqdm-4.70.0-py3-none-any.whl (80 kB) Downloading transformers-5.15.0-py3-none-any.whl (11.7 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.7/11.7 MB 73.1 MB/s 0:00:00 Downloading tokenizers-0.22.2-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.3 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.3/3.3 MB 68.8 MB/s 0:00:00 Downloading tree_sitter-0.26.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (668 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 668.3/668.3 kB 56.5 MB/s 0:00:00 Downloading typer-0.26.8-py3-none-any.whl (122 kB) Using cached typing_extensions-4.16.0-py3-none-any.whl (45 kB) Using cached urllib3-2.7.0-py3-none-any.whl (131 kB) Downloading websockets-16.1.1-cp314-cp314-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl (187 kB) Using cached annotated_doc-0.0.5-py3-none-any.whl (5.3 kB) Using cached annotated_types-0.8.0-py3-none-any.whl (13 kB) Using cached attrs-26.1.0-py3-none-any.whl (67 kB) Using cached certifi-2026.7.22-py3-none-any.whl (136 kB) Downloading cuda_pathfinder-1.6.0-py3-none-any.whl (54 kB) Downloading filelock-3.32.3-py3-none-any.whl (98 kB) Downloading fsspec-2026.7.0-py3-none-any.whl (206 kB) Using cached h11-0.16.0-py3-none-any.whl (37 kB) Using cached jsonschema_specifications-2025.9.1-py3-none-any.whl (18 kB) Downloading lxml-6.1.1-cp314-cp314-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl (5.2 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.2/5.2 MB 69.7 MB/s 0:00:00 Downloading networkx-3.6.1-py3-none-any.whl (2.1 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 76.9 MB/s 0:00:00 Downloading omegaconf-2.3.1-py3-none-any.whl (79 kB) Downloading opencv_python-5.0.0.93-cp37-abi3-manylinux_2_28_x86_64.whl (73.8 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 73.8/73.8 MB 75.4 MB/s 0:00:00 Using cached packaging-26.3-py3-none-any.whl (129 kB) Downloading polyfactory-3.3.0-py3-none-any.whl (62 kB) Downloading faker-40.36.0-py3-none-any.whl (2.1 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 70.2 MB/s 0:00:00 Downloading pyclipper-1.4.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (968 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 968.2/968.2 kB 66.9 MB/s 0:00:00 Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) Using cached referencing-0.37.0-py3-none-any.whl (26 kB) Downloading regex-2026.7.19-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (801 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 801.3/801.3 kB 62.6 MB/s 0:00:00 Using cached rich-15.0.0-py3-none-any.whl (310 kB) Using cached pygments-2.20.0-py3-none-any.whl (1.2 MB) Using cached markdown_it_py-4.2.0-py3-none-any.whl (91 kB) Using cached mdurl-0.1.2-py3-none-any.whl (10.0 kB) Using cached rpds_py-2026.6.3-cp314-cp314-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (367 kB) Using cached setuptools-84.0.0-py3-none-any.whl (818 kB) Downloading shapely-2.1.2-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (3.1 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.1/3.1 MB 69.7 MB/s 0:00:00 Using cached shellingham-1.5.4-py2.py3-none-any.whl (9.8 kB) Using cached six-1.17.0-py2.py3-none-any.whl (11 kB) Downloading soupsieve-2.9.2-py3-none-any.whl (37 kB) Using cached sympy-1.14.0-py3-none-any.whl (6.3 MB) Using cached mpmath-1.3.0-py3-none-any.whl (536 kB) Downloading tree_sitter_c-0.24.2-cp310-abi3-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl (94 kB) Downloading tree_sitter_javascript-0.25.0-cp310-abi3-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl (99 kB) Downloading tree_sitter_python-0.25.0-cp310-abi3-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl (108 kB) Downloading tree_sitter_typescript-0.23.2-cp39-abi3-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (344 kB) Using cached typing_inspection-0.4.4-py3-none-any.whl (14 kB) Downloading xlsxwriter-3.2.9-py3-none-any.whl (175 kB) Using cached anyio-4.14.2-py3-none-any.whl (125 kB) Downloading colorlog-6.12.0-py3-none-any.whl (12 kB) Downloading et_xmlfile-2.0.0-py3-none-any.whl (18 kB) Using cached jinja2-3.1.6-py3-none-any.whl (134 kB) Using cached markupsafe-3.0.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (23 kB) Downloading mpire-2.10.2-py3-none-any.whl (272 kB) Downloading multiprocess-0.70.19-py314-none-any.whl (160 kB) Using cached dill-0.4.1-py3-none-any.whl (120 kB) Downloading olefile-0.47-py2.py3-none-any.whl (114 kB) Using cached psutil-7.2.2-cp36-abi3-manylinux2010_x86_64.manylinux_2_12_x86_64.manylinux_2_28_x86_64.whl (155 kB) Building wheels for collected packages: antlr4-python3-runtime Building wheel for antlr4-python3-runtime (pyproject.toml) ... done Created wheel for antlr4-python3-runtime: filename=antlr4_python3_runtime-4.9.3-py3-none-any.whl size=144590 sha256=3b9b631d81e49c2535003a30927b3521d289f74fb6525a57ff4f3b610eb1fce6 Stored in directory: /home/seaman/.cache/pip/wheels/ea/ca/a6/5ad163e6c66e23b25d4e6a81dba06bce77047ef6257d643728 Successfully built antlr4-python3-runtime Installing collected packages: pylatexenc, nvidia-cusparselt-cu13, mpmath, filetype, cuda-toolkit, antlr4-python3-runtime, XlsxWriter, websockets, urllib3, typing-extensions, triton, tree-sitter-typescript, tree-sitter-python, tree-sitter-javascript, tree-sitter-c, tree-sitter, tqdm, tabulate, sympy, soupsieve, six, shellingham, setuptools, safetensors, rtree, rpds-py, regex, pyyaml, python-dotenv, pypdfium2, pygments, pyclipper, psutil, pluggy, pillow, packaging, olefile, nvidia-nvtx, nvidia-nvshmem-cu13, nvidia-nvjitlink, nvidia-nccl-cu13, nvidia-curand, nvidia-cufile, nvidia-cuda-runtime, nvidia-cuda-nvrtc, nvidia-cuda-cupti, numpy, networkx, mdurl, MarkupSafe, marko, mail-parser, lxml, latex2mathml, jsonref, idna, hf-xet, h11, fsspec, filelock, faker, et-xmlfile, dill, defusedxml, cuda-pathfinder, colorlog, click, charset_normalizer, certifi, attrs, annotated-types, annotated-doc, typing-inspection, Shapely, scipy, requests, referencing, python-pptx, python-oxmsg, python-docx, python-dateutil, pydantic-core, polyfactory, openpyxl, opencv_python, omegaconf, nvidia-cusparse, nvidia-cufft, nvidia-cublas, multiprocess, mpire, markdown-it-py, jsonlines, jinja2, httpcore, cuda-bindings, beautifulsoup4, anyio, rich, rapidocr, pydantic, pandas, nvidia-cusolver, nvidia-cudnn-cu13, jsonschema-specifications, httpx, typer, semchunk, pydantic-settings, jsonschema, huggingface-hub, torch, tokenizers, doclang, transformers, torchvision, docling-core, accelerate, docling-slim, docling-parse, docling-ibm-models, docling ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━ 101/122 [pandas]^C ```

as you can see it is installing also:

"""
...
Using cached cuda_toolkit-13.0.3.0-py2.py3-none-any.whl (2.5 kB)
Using cached nvidia_cudnn_cu13-9.20.0.48-py3-none-manylinux_2_27_x86_64.whl (366.2 MB)
Using cached nvidia_cusparselt_cu13-0.8.1-py3-none-manylinux2014_x86_64.whl (170.1 MB)
Using cached nvidia_nccl_cu13-2.29.7-py3-none-manylinux_2_18_x86_64.whl (206.0 MB)
Using cached nvidia_nvshmem_cu13-3.4.5-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (60.4 MB)
Using cached triton-3.7.1-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (197.7 MB)
Using cached cuda_bindings-13.3.1-cp314-cp314-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (6.5 MB)
Downloading nvidia_cublas-13.1.1.3-py3-none-manylinux_2_27_x86_64.whl (423.1 MB)
...
"""

Those packages will never work on server A and they will probably generate errors and problems.
Also, they should not be necessary because inference is running at the remote GPU server B.

So my question is:

in a remote llama.cpp server backend configuration, how to avoid installing those unnecessary dependencies?

Thank you so much!

[dolfim-ibm]

You might want to look into the docling-slim package. However, the layout and table model won't run on Ollama.

[elfarolab]

@dolfim-ibm
Hello, your message is so much appreciated.
I am trying to understand the architecture of Docling, the main package.

At the official website, in the documentation section, I found the procedure for configuring Docling with a remote llama.cpp server.
However, Docling seems to me not designed for server-client architecture.

I may be wrong, and I really hope so... but it looks like the local inference is so tightly integrated into the docling that it is very difficult to separate the parts. That's why customizing the installation is so complex. Without proper installation, I am running into impossible package requirements.

Initially, I thought it was just a configuration job. I would enable the remote inference via LAN running on my GPU llama.cpp server, but I still cannot solve the installation problems. I cannot figure out how to do a clean installation for client-server architecture.
In my setup, the client host running Docling is just a Linux box with an AMD CPU and without any GPU.
I have llama.cpp running on another Linux box with an Nvidia GPU connected via LAN, perfectly serving LLM responses to my laboratory.

I would like to achieve what the official documentation says but without running any local inference onto the AMD Linux box.

Thank you so much to everybody!

[dolfim-ibm]

if you want to run Docling remotely, you can use our API server https://docling-project.github.io/docling/usage/api_server/

[elfarolab]

yeah, I was referring exactly to that, but it doesn't work becaus ethe installation problems, also I would rather prefer the opposite architecture, where Docling is client making requests to a remote inference server:

https://docling-project.github.io/docling/usage/gpu/

This because you cannot install whatever version of package into a Nvidia Jetson platform or DGX SPARK, because CUDA is tightly integrated into the OS, you cannot update the versions without updating the Jetpack SDK. So Running Docling as server is impossible due the installation problems.