fix(backends): pin grpcio-tools to installed grpcio in runProtogen (protobuf gencode mismatch)

[localai-bot]

Problem

The vLLM backend (all accelerations) fails to start with:

google.protobuf.runtime_version.VersionError: Detected incompatible Protobuf
Gencode/Runtime versions when loading backend.proto: gencode 7.35.0 runtime 6.33.6.
Runtime version cannot be older than the linked gencode version.

The backend crashes on import backend_pb2 before it can serve, surfacing to the user as failed to load model with internal loader: grpc service not ready.

Reported as a ROCm / R9700 gfx1201 failure (#10718), but it is not GPU-specific — the process never reaches any GPU code. It affects every vLLM variant (and any Python backend that caps the protobuf runtime below the latest grpcio-tools gencode).

Root cause

runProtogen (backend/python/common/libbackend.sh) runs after the backend's requirements are installed, and installs grpcio-tools unpinned:

uv pip install grpcio-tools   # pulls the newest release

The protoc bundled in the newest grpcio-tools stamps backend_pb2.py with Protobuf gencode 7.35.0. vLLM's dependencies pin the protobuf runtime down to 6.33.6. Protobuf enforces runtime >= gencode at import, so the generated module refuses to load.

Fix

Pin grpcio-tools to the grpcio version the backend already installed — the two are released in lockstep, so the bundled protoc's gencode stays in step with the protobuf runtime. Falls back to unpinned when grpcio isn't present (behaviour unchanged for those backends).

grpcio_version="$(python -c 'import importlib.metadata as m; print(m.version("grpcio"))' 2>/dev/null || true)"
[ -n "$grpcio_version" ] && grpcio_tools_spec="grpcio-tools==${grpcio_version}"

Testing

• bash -n clean; shellcheck -S warning reports nothing new on the changed lines (pre-existing findings elsewhere untouched).
• Fallback path verified (grpcio absent → unpinned, as before).
• Full runtime validation requires a backend image build (ROCm/vLLM) — CI territory; cannot be reproduced locally.

Closes #10718

🤖 Generated with Claude Code

[aTosser]

#10718 is not resolved. There exists still the protobuf gencode/runtime mismatch error in both rocm-vllm and rocm-vllm-development still. grpcio-tools is not pinned to 1.78.0 for some reason. This patch seems ineffective.

[Jpc-Automation]

#10718 is not resolved. There exists still the protobuf gencode/runtime mismatch error in both rocm-vllm and rocm-vllm-development still. grpcio-tools is not pinned to 1.78.0 for some reason. This patch seems ineffective.

I can confirm the issue still persist in exactly the same way.
#10718
#10642
#10940

Is there a way to set the grpc version without compiling the whole project/backend?

@mudler should we reopen the issue?

best regards

[localai-bot]

@Jpc-Automation thanks for the ping, and for spotting that grpcio-tools was not landing on 1.78.0 — that observation was the thread that unravelled it.

Answering the direct question first: no need to reopen. There is no defect left in master. The two backends you named are failing for two different reasons:

• rocm-vllm-development (tracks master): already fixed by fix(vllm): generate protobuf 6 compatible stubs #10944. I read backend_pb2.py straight out of the published master-gpu-rocm-hipblas-vllm image (built 2026-07-21) and it stamps gencode 6.31.1, well under the 6.33.6 runtime. It cannot produce the 7.35.0 error. If you are still seeing it, the backend installed under BackendsPath is older than that image — LocalAI installs a backend once and a restart does not re-pull it. Delete the backend and reinstall it.
• rocm-vllm (tracks latest): genuinely still broken. That image is from 2026-07-14 and predates both fixes, so nothing local will help; it needs the next release build.

On why this PR's pin was ineffective: it pinned grpcio-tools to the installed grpcio version, but grpcio-tools' version tracks grpcio while the gencode its protoc emits tracks protobuf, and the two move independently. Since requirements.txt pins grpcio==1.82.1, the pin resolved to grpcio-tools==1.82.1, which requires protobuf>=7.35.1 and stamps gencode 7.35.0 — precisely the broken generator. #10944 then fixed it by hardcoding 1.78.0.

I have opened #11057 to make that robust rather than lucky: it derives the generator from the installed protobuf runtime (so it cannot rot as protobuf advances), regenerates the stubs after the late vllm install, clears __pycache__ (a stale .pyc can shadow a regenerated stub, since CPython validates by mtime and size and the gencode triple is the same byte width), and fails the build if the generated stub cannot be imported.

Full detail and a stopgap for rocm-vllm in #10940 (comment).