Huggingface server fails to start on OpenShift cluster

[eyalcha]

/kind bug

Probably need to add something like that for OpenShift

# For OpenShift deployment to allow mkdir
ENV HUGGINGFACE_HUB_CACHE="/tmp/huggingface/hub"

There was a problem when trying to write in your cache folder (/.cache/huggingface/hub). You should set the environment variable TRANSFORMERS_CACHE to a writable directory.
Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/huggingfaceserver/huggingfaceserver/__main__.py", line 69, in <module>
    model.load()
  File "/huggingfaceserver/huggingfaceserver/model.py", line 114, in load
    model_config = AutoConfig.from_pretrained(model_id_or_path)
  File "/prod_venv/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 1100, in from_pretrained
    config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/prod_venv/lib/python3.10/site-packages/transformers/configuration_utils.py", line 634, in get_config_dict
    config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/prod_venv/lib/python3.10/site-packages/transformers/configuration_utils.py", line 689, in _get_config_dict
    resolved_config_file = cached_file(
  File "/prod_venv/lib/python3.10/site-packages/transformers/utils/hub.py", line 385, in cached_file
    resolved_file = hf_hub_download(
  File "/prod_venv/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 119, in _inner_fn
    return fn(*args, **kwargs)
  File "/prod_venv/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1418, in hf_hub_download
    os.makedirs(os.path.dirname(blob_path), exist_ok=True)
  File "/usr/lib/python3.10/os.py", line 215, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/usr/lib/python3.10/os.py", line 215, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/usr/lib/python3.10/os.py", line 215, in makedirs
    makedirs(head, exist_ok=exist_ok)
  [Previous line repeated 1 more time]
  File "/usr/lib/python3.10/os.py", line 225, in makedirs
    mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/.cache'

[spolti]

@terrytangyuan @israel-hdez f.y.i

[eyalcha]

This works for now

  containers:
    - name: kserve-container
      image: huggingfaceserver:replace
      args:
        - --model_name={{.Name}}
      env:
        - name: HUGGINGFACE_HUB_CACHE
          value: /tmp/huggingface/hub
      resources:
        requests:
          cpu: "1"
          memory: 2Gi
        limits:
          cpu: "1"
          memory: 2Gi

[terrytangyuan]

Thanks. Sent a PR to add this #3576