Skip to content

Fix openarc serve start failing due to wrong dep version#120

Open
solidDoWant wants to merge 1 commit into
SearchSavior:mainfrom
solidDoWant:fix/wrong-optimum-version-1
Open

Fix openarc serve start failing due to wrong dep version#120
solidDoWant wants to merge 1 commit into
SearchSavior:mainfrom
solidDoWant:fix/wrong-optimum-version-1

Conversation

@solidDoWant
Copy link
Copy Markdown
Contributor

This is a lockfile update that includes bumping optimum to 2.0.0, which is needed with a recently-added pytorch bump. Without this, openarc serve start fails because optimum attempts to access a now non-existent pytorch field.

I'm running a build with this and it's working with my specific configuration and use case, but might need additional users to test prior to merge.

Signed-off-by: solidDoWant <fred.heinecke@yahoo.com>
@SearchSavior
Copy link
Copy Markdown
Owner

@solidDoWant Hey thanks for checking this out. So a lock file update should be accompanied by a pyproject.toml change right? Was there a specific error you saw? I think there is one about ``OVFeatureExtraction` (something like this) that prevents the server from working.

Lately there have also been issues with transformers versions between gemma4 and qwen35/36 requiring different versions for export and sometimes for inference. It'a annoying, we need the nightlies from optimum but they pin old transformers. so it breaks the uv dependency resolution, and I havent investigated in a while. Does that sounds closer to what you are seeing?

@solidDoWant
Copy link
Copy Markdown
Contributor Author

solidDoWant commented May 23, 2026

@SearchSavior Looks like optimum is trying to import a private field from torch that is no longer exposed. In torch v2.8.0 (previous version before the bump), this symbol was defined at torch.onnx.symbolic_opset14 here. This changed in v2.9.0 (see same file here), which moved the symbol to an internal package. The function exists, but it's no longer available for optimum to import.

Here is the full startup log when trying to use the Qwen3 model (long)
OPENARC_API_KEY_REQUIRED=False [Clients do not need to authenticate.]
Starting OpenArc server on 0.0.0.0:8000
2026-05-18 20:11:59,033 - INFO - Launching  0.0.0.0:8000
2026-05-18 20:11:59,033 - INFO - --------------------------------
2026-05-18 20:11:59,033 - INFO - OpenArc endpoints:
2026-05-18 20:11:59,033 - INFO -   - POST   /openarc/load           Load a model
2026-05-18 20:11:59,033 - INFO -   - POST   /openarc/unload         Unload a model
2026-05-18 20:11:59,033 - INFO -   - GET    /openarc/status         Get model status
2026-05-18 20:11:59,033 - INFO -   - GET    /openarc/metrics            Get hardware telemetry
2026-05-18 20:11:59,033 - INFO -   - POST   /openarc/models/update      Update model configuration
2026-05-18 20:11:59,034 - INFO -   - POST   /openarc/bench              Run inference benchmark
2026-05-18 20:11:59,034 - INFO -   - GET    /openarc/downloader         List active model downloads
2026-05-18 20:11:59,034 - INFO -   - POST   /openarc/downloader         Start a model download
2026-05-18 20:11:59,034 - INFO -   - DELETE /openarc/downloader         Cancel a model download
2026-05-18 20:11:59,034 - INFO -   - POST   /openarc/downloader/pause   Pause a model download
2026-05-18 20:11:59,034 - INFO -   - POST   /openarc/downloader/resume  Resume a model download
2026-05-18 20:11:59,034 - INFO - --------------------------------
2026-05-18 20:11:59,034 - INFO - OpenAI compatible endpoints:
2026-05-18 20:11:59,034 - INFO -   - GET    /v1/models
2026-05-18 20:11:59,034 - INFO -   - POST   /v1/chat/completions
2026-05-18 20:11:59,034 - INFO -   - POST   /v1/audio/transcriptions: Whisper only
2026-05-18 20:11:59,034 - INFO -   - POST   /v1/audio/speech: Kokoro only
2026-05-18 20:11:59,034 - INFO -   - POST   /v1/embeddings
2026-05-18 20:11:59,034 - INFO -   - POST   /v1/rerank
2026-05-18 20:12:00,366 - WARNING - Failed to create openvino_telemetry file. Please allow write access to the following directory: /app
2026-05-18 20:12:00,366 - WARNING - Failed to create openvino_telemetry file. Please allow write access to the following directory: /app
2026-05-18 20:12:00,366 - WARNING - Could not create directory for storing client ID. No data will be sent.
2026-05-18 20:12:03,715 - WARNING - mkdir -p failed for path /app/.config/matplotlib: [Errno 30] Read-only file system: '/app/.config'
2026-05-18 20:12:03,715 - WARNING - Matplotlib created a temporary cache directory at /tmp/matplotlib-n6cwezsa because there was an issue with the default path (/app/.config/matplotlib); it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the import of Matplotlib and to better support multiprocessing.
Traceback (most recent call last):
  File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/bin/openarc", line 10, in <module>
    sys.exit(cli())
             ^^^^^
  File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/rich_click/rich_command.py", line 404, in __call__
    return super().__call__(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/click/core.py", line 1442, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/rich_click/rich_command.py", line 187, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/click/core.py", line 1830, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/click/core.py", line 1830, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/click/core.py", line 1226, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/click/core.py", line 794, in invoke
    return callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/click/decorators.py", line 34, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/src/cli/groups/serve.py", line 84, in start
    start_server(host=host, port=port)
  File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/src/cli/modules/launch_server.py", line 116, in start_server
    uvicorn.run(
  File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/uvicorn/main.py", line 580, in run
    server.run()
  File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/uvicorn/server.py", line 67, in run
    return asyncio.run(self.serve(sockets=sockets))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/h3q2g9wq4x3q84164qsfm3lz5djj0bf3-python3-3.12.13/lib/python3.12/asyncio/runners.py", line 195, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/nix/store/h3q2g9wq4x3q84164qsfm3lz5djj0bf3-python3-3.12.13/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/h3q2g9wq4x3q84164qsfm3lz5djj0bf3-python3-3.12.13/lib/python3.12/asyncio/base_events.py", line 691, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/uvicorn/server.py", line 71, in serve
    await self._serve(sockets)
  File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/uvicorn/server.py", line 78, in _serve
    config.load()
  File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/uvicorn/config.py", line 436, in load
    self.loaded_app = import_from_string(self.app)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/uvicorn/importer.py", line 19, in import_from_string
    module = importlib.import_module(module_str)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/h3q2g9wq4x3q84164qsfm3lz5djj0bf3-python3-3.12.13/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 999, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/src/server/main.py", line 17, in <module>
    from src.server.deps import _registry
  File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/src/server/deps.py", line 8, in <module>
    from src.server.worker_registry import WorkerRegistry
  File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/src/server/worker_registry.py", line 18, in <module>
    from src.engine.optimum.optimum_emb import Optimum_EMB
  File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/src/engine/optimum/optimum_emb.py", line 13, in <module>
    from optimum.intel import OVModelForFeatureExtraction
  File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/transformers/utils/import_utils.py", line 2045, in __getattr__
    module = self._get_module(self._class_to_module[name])
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/transformers/utils/import_utils.py", line 2075, in _get_module
    raise e
  File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/transformers/utils/import_utils.py", line 2073, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/h3q2g9wq4x3q84164qsfm3lz5djj0bf3-python3-3.12.13/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/optimum/intel/openvino/__init__.py", line 50, in <module>
    from .quantization import OVCalibrationDataset, OVQuantizer
  File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/optimum/intel/openvino/quantization.py", line 51, in <module>
    from ...exporters.openvino import export, export_pytorch_via_onnx
  File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/optimum/exporters/openvino/__init__.py", line 15, in <module>
    import optimum.exporters.openvino.model_configs
  File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/optimum/exporters/openvino/model_configs.py", line 23, in <module>
    from optimum.exporters.onnx.base import ConfigBehavior
  File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/optimum/exporters/onnx/base.py", line 52, in <module>
    from .model_patcher import DecoderModelPatcher, ModelPatcher, Seq2SeqModelPatcher
  File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/optimum/exporters/onnx/model_patcher.py", line 346, in <module>
    from torch.onnx.symbolic_opset14 import (  # noqa: E402
ImportError: cannot import name '_attention_scale' from 'torch.onnx.symbolic_opset14' (/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/torch/onnx/symbolic_opset14.py)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants