[Bug]: openAI endpoint crashing on "no locator available"

[mrseeker]

Your current environment

🐛 Describe the bug

Getting this error when starting an openAI endpoint on the docker container:

Traceback (most recent call last):
  File “/usr/lib/python3.10/runpy.py”, line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File “/usr/lib/python3.10/runpy.py”, line 86, in _run_code
    exec(code, run_globals)
  File “/app/aphrodite-engine/aphrodite/endpoints/openai/api_server.py”, line 30, in <module>
    from aphrodite.endpoints.openai.serving_chat import OpenAIServingChat
  File “/app/aphrodite-engine/aphrodite/endpoints/openai/serving_chat.py”, line 16, in <module>
    from aphrodite.modeling.outlines_decoding import get_guided_decoding_logits_processor
  File “/app/aphrodite-engine/aphrodite/modeling/outlines_decoding.py”, line 12, in <module>
    from aphrodite.modeling.outlines_logits_processors import JSONLogitsProcessor, RegexLogitsProcessor
  File “/app/aphrodite-engine/aphrodite/modeling/outlines_logits_processors.py”, line 24, in <module>
    from outlines.fsm.fsm import RegexFSM
  File “/usr/local/lib/python3.10/dist-packages/outlines/__init__.py”, line 2, in <module>
    import outlines.generate
  File “/usr/local/lib/python3.10/dist-packages/outlines/generate/__init__.py”, line 2, in <module>
    from .cfg import cfg
  File “/usr/local/lib/python3.10/dist-packages/outlines/generate/cfg.py”, line 3, in <module>
    from outlines.fsm.guide import CFGGuide
  File “/usr/local/lib/python3.10/dist-packages/outlines/fsm/guide.py”, line 9, in <module>
    from outlines.fsm.regex import create_fsm_index_tokenizer, make_deterministic_fsm
  File “/usr/local/lib/python3.10/dist-packages/outlines/fsm/regex.py”, line 96, in <module>
    def create_fsm_info(
  File “/usr/local/lib/python3.10/dist-packages/numba/core/decorators.py”, line 229, in wrapper
    disp.enable_caching()
  File “/usr/local/lib/python3.10/dist-packages/numba/core/dispatcher.py”, line 856, in enable_caching
    self._cache = FunctionCache(self.py_func)
  File “/usr/local/lib/python3.10/dist-packages/numba/core/caching.py”, line 601, in __init__
    self._impl = self._impl_class(py_func)
  File “/usr/local/lib/python3.10/dist-packages/numba/core/caching.py”, line 337, in __init__
    raise RuntimeError(“cannot cache function %r: no locator available ”
RuntimeError: cannot cache function ‘create_fsm_info’: no locator available for file ‘/usr/local/lib/python3.10/dist-packages/outlines/fsm/regex.py

[mrseeker]

Issue is solvable, add:

NUMBA_CACHE_DIR=/tmp/numba_cache