Fix openarc serve start failing due to wrong dep version#120
Fix openarc serve start failing due to wrong dep version#120solidDoWant wants to merge 1 commit into
openarc serve start failing due to wrong dep version#120Conversation
Signed-off-by: solidDoWant <fred.heinecke@yahoo.com>
|
@solidDoWant Hey thanks for checking this out. So a lock file update should be accompanied by a Lately there have also been issues with transformers versions between |
|
@SearchSavior Looks like optimum is trying to import a private field from torch that is no longer exposed. In torch v2.8.0 (previous version before the bump), this symbol was defined at Here is the full startup log when trying to use the Qwen3 model (long)OPENARC_API_KEY_REQUIRED=False [Clients do not need to authenticate.]
Starting OpenArc server on 0.0.0.0:8000
2026-05-18 20:11:59,033 - INFO - Launching 0.0.0.0:8000
2026-05-18 20:11:59,033 - INFO - --------------------------------
2026-05-18 20:11:59,033 - INFO - OpenArc endpoints:
2026-05-18 20:11:59,033 - INFO - - POST /openarc/load Load a model
2026-05-18 20:11:59,033 - INFO - - POST /openarc/unload Unload a model
2026-05-18 20:11:59,033 - INFO - - GET /openarc/status Get model status
2026-05-18 20:11:59,033 - INFO - - GET /openarc/metrics Get hardware telemetry
2026-05-18 20:11:59,033 - INFO - - POST /openarc/models/update Update model configuration
2026-05-18 20:11:59,034 - INFO - - POST /openarc/bench Run inference benchmark
2026-05-18 20:11:59,034 - INFO - - GET /openarc/downloader List active model downloads
2026-05-18 20:11:59,034 - INFO - - POST /openarc/downloader Start a model download
2026-05-18 20:11:59,034 - INFO - - DELETE /openarc/downloader Cancel a model download
2026-05-18 20:11:59,034 - INFO - - POST /openarc/downloader/pause Pause a model download
2026-05-18 20:11:59,034 - INFO - - POST /openarc/downloader/resume Resume a model download
2026-05-18 20:11:59,034 - INFO - --------------------------------
2026-05-18 20:11:59,034 - INFO - OpenAI compatible endpoints:
2026-05-18 20:11:59,034 - INFO - - GET /v1/models
2026-05-18 20:11:59,034 - INFO - - POST /v1/chat/completions
2026-05-18 20:11:59,034 - INFO - - POST /v1/audio/transcriptions: Whisper only
2026-05-18 20:11:59,034 - INFO - - POST /v1/audio/speech: Kokoro only
2026-05-18 20:11:59,034 - INFO - - POST /v1/embeddings
2026-05-18 20:11:59,034 - INFO - - POST /v1/rerank
2026-05-18 20:12:00,366 - WARNING - Failed to create openvino_telemetry file. Please allow write access to the following directory: /app
2026-05-18 20:12:00,366 - WARNING - Failed to create openvino_telemetry file. Please allow write access to the following directory: /app
2026-05-18 20:12:00,366 - WARNING - Could not create directory for storing client ID. No data will be sent.
2026-05-18 20:12:03,715 - WARNING - mkdir -p failed for path /app/.config/matplotlib: [Errno 30] Read-only file system: '/app/.config'
2026-05-18 20:12:03,715 - WARNING - Matplotlib created a temporary cache directory at /tmp/matplotlib-n6cwezsa because there was an issue with the default path (/app/.config/matplotlib); it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the import of Matplotlib and to better support multiprocessing.
Traceback (most recent call last):
File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/bin/openarc", line 10, in <module>
sys.exit(cli())
^^^^^
File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/rich_click/rich_command.py", line 404, in __call__
return super().__call__(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/click/core.py", line 1442, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/rich_click/rich_command.py", line 187, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/click/core.py", line 1830, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/click/core.py", line 1830, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/click/core.py", line 1226, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/click/core.py", line 794, in invoke
return callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/click/decorators.py", line 34, in new_func
return f(get_current_context(), *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/src/cli/groups/serve.py", line 84, in start
start_server(host=host, port=port)
File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/src/cli/modules/launch_server.py", line 116, in start_server
uvicorn.run(
File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/uvicorn/main.py", line 580, in run
server.run()
File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/uvicorn/server.py", line 67, in run
return asyncio.run(self.serve(sockets=sockets))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/nix/store/h3q2g9wq4x3q84164qsfm3lz5djj0bf3-python3-3.12.13/lib/python3.12/asyncio/runners.py", line 195, in run
return runner.run(main)
^^^^^^^^^^^^^^^^
File "/nix/store/h3q2g9wq4x3q84164qsfm3lz5djj0bf3-python3-3.12.13/lib/python3.12/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/nix/store/h3q2g9wq4x3q84164qsfm3lz5djj0bf3-python3-3.12.13/lib/python3.12/asyncio/base_events.py", line 691, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/uvicorn/server.py", line 71, in serve
await self._serve(sockets)
File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/uvicorn/server.py", line 78, in _serve
config.load()
File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/uvicorn/config.py", line 436, in load
self.loaded_app = import_from_string(self.app)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/uvicorn/importer.py", line 19, in import_from_string
module = importlib.import_module(module_str)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/nix/store/h3q2g9wq4x3q84164qsfm3lz5djj0bf3-python3-3.12.13/lib/python3.12/importlib/__init__.py", line 90, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 999, in exec_module
File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/src/server/main.py", line 17, in <module>
from src.server.deps import _registry
File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/src/server/deps.py", line 8, in <module>
from src.server.worker_registry import WorkerRegistry
File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/src/server/worker_registry.py", line 18, in <module>
from src.engine.optimum.optimum_emb import Optimum_EMB
File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/src/engine/optimum/optimum_emb.py", line 13, in <module>
from optimum.intel import OVModelForFeatureExtraction
File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/transformers/utils/import_utils.py", line 2045, in __getattr__
module = self._get_module(self._class_to_module[name])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/transformers/utils/import_utils.py", line 2075, in _get_module
raise e
File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/transformers/utils/import_utils.py", line 2073, in _get_module
return importlib.import_module("." + module_name, self.__name__)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/nix/store/h3q2g9wq4x3q84164qsfm3lz5djj0bf3-python3-3.12.13/lib/python3.12/importlib/__init__.py", line 90, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/optimum/intel/openvino/__init__.py", line 50, in <module>
from .quantization import OVCalibrationDataset, OVQuantizer
File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/optimum/intel/openvino/quantization.py", line 51, in <module>
from ...exporters.openvino import export, export_pytorch_via_onnx
File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/optimum/exporters/openvino/__init__.py", line 15, in <module>
import optimum.exporters.openvino.model_configs
File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/optimum/exporters/openvino/model_configs.py", line 23, in <module>
from optimum.exporters.onnx.base import ConfigBehavior
File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/optimum/exporters/onnx/base.py", line 52, in <module>
from .model_patcher import DecoderModelPatcher, ModelPatcher, Seq2SeqModelPatcher
File "/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/optimum/exporters/onnx/model_patcher.py", line 346, in <module>
from torch.onnx.symbolic_opset14 import ( # noqa: E402
ImportError: cannot import name '_attention_scale' from 'torch.onnx.symbolic_opset14' (/nix/store/5pnypvlz4mbz5vm6rxvcqfak88yiyjg8-openarc-env/lib/python3.12/site-packages/torch/onnx/symbolic_opset14.py) |
This is a lockfile update that includes bumping optimum to 2.0.0, which is needed with a recently-added pytorch bump. Without this,
openarc serve startfails because optimum attempts to access a now non-existent pytorch field.I'm running a build with this and it's working with my specific configuration and use case, but might need additional users to test prior to merge.