Can't set model alias

[hongyin163]

Prerequisites

Please answer the following questions for yourself before submitting an issue.

• [y ] I am running the latest code. Development is very rapid so there are no tagged versions as of now.
• [y] I carefully followed the README.md.
• [y] I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• [y] I reviewed the Discussions, and have a new bug or useful enhancement to share.

Expected Behavior

I run command as follow to start server

python -m llama_cpp.server --model ../llama.cpp/models/airoboros-7b-gpt4/airoboros-7b-gpt4-1.4.ggmlv3.q4_0.bin --model_alias gpt-4

I can run it if I don't set --model_alias parameter,but if I set --model_alias as above, there will be an error as follow.

Current Behavior

The execute result is follow:

/Users/hongyin/anaconda3/envs/llmcpp/lib/python3.11/site-packages/pydantic/_internal/_fields.py:126: UserWarning: Field "model_alias" has conflict with protected namespace "model_".

You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ('settings_',)`.
  warnings.warn(
usage: __main__.py [-h] [--model MODEL] [--model_alias MODEL_ALIAS] [--n_ctx N_CTX] [--n_gpu_layers N_GPU_LAYERS]
                   [--tensor_split TENSOR_SPLIT] [--rope_freq_base ROPE_FREQ_BASE] [--rope_freq_scale ROPE_FREQ_SCALE]
                   [--seed SEED] [--n_batch N_BATCH] [--n_threads N_THREADS] [--f16_kv F16_KV] [--use_mlock USE_MLOCK]
                   [--use_mmap USE_MMAP] [--embedding EMBEDDING] [--low_vram LOW_VRAM]
                   [--last_n_tokens_size LAST_N_TOKENS_SIZE] [--logits_all LOGITS_ALL] [--cache CACHE]
                   [--cache_type CACHE_TYPE] [--cache_size CACHE_SIZE] [--vocab_only VOCAB_ONLY] [--verbose VERBOSE]
                   [--host HOST] [--port PORT] [--interrupt_requests INTERRUPT_REQUESTS]
__main__.py: error: argument --model_alias: invalid Optional value: 'gpt-4'
(llmcpp) hongyin@honyindeMacBook-Pro pyai % python --version
Python 3.11.4

Environment and Context

Please provide detailed information about your computer setup. This is important in case the issue is not reproducible except for under certain specific conditions.

• Physical (or virtual) hardware you are using, e.g. for Linux:

$ lscpu

• Operating System, e.g. for Linux:

$ uname -a
Darwin honyindeMacBook-Pro.local 22.3.0 Darwin Kernel Version 22.3.0: Mon Jan 30 20:39:46 PST 2023; root:xnu-8792.81.3~2/RELEASE_ARM64_T6020 arm64

• SDK version, e.g. for Linux:

$ python3 --version 3.11.4
$ make --version 3.81
$ g++ --version 14.0.0

Failure Information (for bugs)

/Users/hongyin/anaconda3/envs/llmcpp/lib/python3.11/site-packages/pydantic/_internal/_fields.py:126: UserWarning: Field "model_alias" has conflict with protected namespace "model_".

You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ('settings_',)`.
  warnings.warn(
usage: __main__.py [-h] [--model MODEL] [--model_alias MODEL_ALIAS] [--n_ctx N_CTX] [--n_gpu_layers N_GPU_LAYERS]
                   [--tensor_split TENSOR_SPLIT] [--rope_freq_base ROPE_FREQ_BASE] [--rope_freq_scale ROPE_FREQ_SCALE]
                   [--seed SEED] [--n_batch N_BATCH] [--n_threads N_THREADS] [--f16_kv F16_KV] [--use_mlock USE_MLOCK]
                   [--use_mmap USE_MMAP] [--embedding EMBEDDING] [--low_vram LOW_VRAM]
                   [--last_n_tokens_size LAST_N_TOKENS_SIZE] [--logits_all LOGITS_ALL] [--cache CACHE]
                   [--cache_type CACHE_TYPE] [--cache_size CACHE_SIZE] [--vocab_only VOCAB_ONLY] [--verbose VERBOSE]
                   [--host HOST] [--port PORT] [--interrupt_requests INTERRUPT_REQUESTS]
__main__.py: error: argument --model_alias: invalid Optional value: 'gpt-4'
(llmcpp) hongyin@honyindeMacBook-Pro pyai % python --version
Python 3.11.4

Steps to Reproduce

1. run command python -m llama_cpp.server --model models/airoboros-7b-gpt4/airoboros-7b-gpt4-1.4.ggmlv3.q4_0.bin --model_alias gpt-4

Failure Logs

llama-cpp-python$ git log | head -1
commit a4fe3fe3502fa6d73575f77220b8694a420c7ebd

llama-cpp-python$ python3 --version
Python 3.11.4

llama-cpp-python$ pip list | egrep "uvicorn|fastapi|sse-starlette|numpy"
uvicorn=0.23.1
anyio=3.7.1
starlette=0.30.0
fastapi=0.100.0
pydantic_settings=2.0.2
sse_starlette=1.6.1

I try to debug the code to solve this problem, if doesn't set type of function add_argument, that will be ok.

import os
import argparse

import uvicorn

from llama_cpp.server.app import create_app, Settings

if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    for name, field in Settings.model_fields.items():
        
        description = field.description
        if field.default is not None and description is not None:
            description += f" (default: {field.default})"
        print(name)
        print(field.annotation)
        parser.add_argument(
            f"--{name}",
            dest=name,
            type=field.annotation if field.annotation is not None else str, # If I comment out this line of code, it is working fine
            help=description,
        )

    args = parser.parse_args() 

    settings = Settings(**{k: v for k, v in vars(args).items() if v is not None})
    app = create_app(settings=settings)

    uvicorn.run(
        app, host=os.getenv("HOST", settings.host), port=int(os.getenv("PORT", settings.port))
    )

[abetlen]

@hongyin163 thanks for reporting this, it was an unintended breaking change when updating to pydantic v2 which reserves any parameters starting with model_ for itself, I'll update model_alias to just alias.

[anujcb]

Hi, am i running into the same issue? I am on Windows environment, i am able to get the "python ./examples/low_level_api/Chat.py" running fine.

python -m llama_cpp.server --model ./../../llama.cpp/models/v1/7B/ggml-model-q4_0.bin --port 7777 --host 192.168.0.1 --n_gpu_layers 30 --n_threads 4 --n_ctx 2048

ggml_init_cublas: found 1 CUDA devices:
Device 0: NVIDIA GeForce GTX 1070, compute capability 6.1
F:\ProgramData\Anaconda3\envs\scrapalot-research-assistant\lib\site-packages\pydantic_internal_fields.py:127: UserWarning: Field "model_alias" has conflict with protected namespace "model_".

You may be able to resolve this warning by setting model_config['protected_namespaces'] = ('settings_',).
warnings.warn(
llama.cpp: loading model from ./../../llama.cpp/models/v1/7B/ggml-model-q4_0.bin
(magic, version) combination: 67676a74, 00000003
llama_model_load_internal: format = ggjt v3 (latest)
llama_model_load_internal: n_vocab = 32000
llama_model_load_internal: n_ctx = 2048
llama_model_load_internal: n_embd = 4096
llama_model_load_internal: n_mult = 256
llama_model_load_internal: n_head = 32
llama_model_load_internal: n_head_kv = 32
llama_model_load_internal: n_layer = 32
llama_model_load_internal: n_rot = 128
llama_model_load_internal: n_gqa = 1
llama_model_load_internal: rnorm_eps = 5.0e-06
llama_model_load_internal: n_ff = 11008
llama_model_load_internal: freq_base = 10000.0
llama_model_load_internal: freq_scale = 1
llama_model_load_internal: ftype = 2 (mostly Q4_0)
llama_model_load_internal: model size = 7B
llama_model_load_internal: ggml ctx size = 0.08 MB
Traceback (most recent call last):
File "F:\ProgramData\Anaconda3\envs\scrapalot-research-assistant\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "F:\ProgramData\Anaconda3\envs\scrapalot-research-assistant\lib\runpy.py", line 86, in run_code
exec(code, run_globals)
File "F:\work\scrapalot-research-assistant\llama-cpp-python\llama_cpp\server_main.py", line 46, in
app = create_app(settings=settings)
File "F:\work\scrapalot-research-assistant\llama-cpp-python\llama_cpp\server\app.py", line 313, in create_app
llama = llama_cpp.Llama(
File "F:\work\scrapalot-research-assistant\llama-cpp-python\llama_cpp\llama.py", line 309, in init
self.model = llama_cpp.llama_load_model_from_file(
File "F:\work\scrapalot-research-assistant\llama-cpp-python\llama_cpp\llama_cpp.py", line 428, in llama_load_model_from_file
return _lib.llama_load_model_from_file(path_model, params)
OSError: [WinError -1073741795] Windows Error 0xc000001d
Exception ignored in: <function Llama.del at 0x000002D67EFF08B0>
Traceback (most recent call last):
File "F:\work\scrapalot-research-assistant\llama-cpp-python\llama_cpp\llama.py", line 1506, in del
if self.model is not None:
AttributeError: 'Llama' object has no attribute 'model'
(scrapalot-research-assistant) PS F:\work\scrapalot-research-assistant\llama-cpp-python> pip show llama-cpp-python
Name: llama-cpp-python
Version: 0.1.72
Summary: A Python wrapper for llama.cpp
Home-page:
Author: Andrei Betlen
Author-email: abetlen@gmail.com
License: MIT
Location: f:\programdata\anaconda3\envs\scrapalot-research-assistant\lib\site-packages
Requires: diskcache, numpy, typing-extensions
Required-by:
(scrapalot-research-assistant) PS F:\work\scrapalot-research-assistant\llama-cpp-python> @@

[anujcb]

model_alias

I was able to change the model_alias to alias in the app.py and the warning went away

But the error
"AttributeError: 'Llama' object has no attribute 'model'"

still there

[nullnuller]

Same issue, any update?

[anujcb]

try setting

[piperino11]

I have the same problem with:docker run --rm -it -p 8001:8001 -v ./modellollama:/models -e MODEL=/models/llama-2-7b-chat.ggmlv3.q2_K.bin ghcr.io/abetlen/llama-cpp-python:latest
You may be able to resolve this warning by setting model_config['protected_namespaces'] = ('settings_',).
warnings.warn(
gguf_init_from_file: invalid magic number 67676a74
error loading model: llama_model_loader: failed to load model from /models/llama-2-7b-chat.ggmlv3.q2_K.bin

llama_load_model_from_file: failed to load model
Traceback (most recent call last):
File "/usr/local/bin/uvicorn", line 8, in
sys.exit(main())
^^^^^^
File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1157, in call
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/uvicorn/main.py", line 416, in main
run(
File "/usr/local/lib/python3.11/site-packages/uvicorn/main.py", line 587, in run
server.run()
File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 61, in run
return asyncio.run(self.serve(sockets=sockets))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run
return runner.run(main)
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 68, in serve
config.load()
File "/usr/local/lib/python3.11/site-packages/uvicorn/config.py", line 473, in load
self.loaded_app = self.loaded_app()
^^^^^^^^^^^^^^^^^
File "/app/llama_cpp/server/app.py", line 317, in create_app
llama = llama_cpp.Llama(
^^^^^^^^^^^^^^^^
File "/app/llama_cpp/llama.py", line 323, in init
assert self.model is not None
^^^^^^^^^^^^^^^^^^^^^^
AssertionError


[steveoon]

same issue when add --model_alias

export MODEL_ALIAS=bitwise-34b-code
export MODEL=/Users/rensiwen/Library/Mobile\ Documents/com~apple~CloudDocs/LLMs-cloud/phind-codellama-34b-v2.Q8_0.gguf
python3 -m llama_cpp.server --model $MODEL --model_alias $MODEL_ALIAS --model_config '{"protected_namespaces": ["settings_"]}' --n_gpu_layers 1

then got error:

/Users/rensiwen/miniforge3/envs/llama-python-server/lib/python3.10/site-packages/pydantic/_internal/_fields.py:127: UserWarning: Field "model_alias" has conflict with protected namespace "model_".

You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ('settings_',)`.
  warnings.warn(
usage: __main__.py [-h] [--model MODEL] [--model_alias MODEL_ALIAS] [--n_ctx N_CTX] [--n_gpu_layers N_GPU_LAYERS]
                   [--tensor_split TENSOR_SPLIT] [--rope_freq_base ROPE_FREQ_BASE] [--rope_freq_scale ROPE_FREQ_SCALE]
                   [--seed SEED] [--n_batch N_BATCH] [--n_threads N_THREADS] [--f16_kv F16_KV] [--use_mlock USE_MLOCK]
                   [--use_mmap USE_MMAP] [--embedding EMBEDDING] [--low_vram LOW_VRAM]
                   [--last_n_tokens_size LAST_N_TOKENS_SIZE] [--logits_all LOGITS_ALL] [--cache CACHE]
                   [--cache_type CACHE_TYPE] [--cache_size CACHE_SIZE] [--vocab_only VOCAB_ONLY] [--verbose VERBOSE]
                   [--host HOST] [--port PORT] [--interrupt_requests INTERRUPT_REQUESTS] [--n_gqa N_GQA]
                   [--rms_norm_eps RMS_NORM_EPS] [--mul_mat_q MUL_MAT_Q]
__main__.py: error: argument --model_alias: invalid Optional value: 'bitwisecode'

[dylan-sh]

bump

[kawnah]

Still running into the same issue. What is the location of the "model_alias" variable to change?

`2023-11-16 17:37:06 /usr/local/lib/python3.10/dist-packages/pydantic/internal/fields.py:128: UserWarning: Field "model_alias" has conflict with protected namespace "model".
2023-11-16 17:37:06
2023-11-16 17:37:06 You may be able to resolve this warning by setting model_config['protected_namespaces'] = ('settings',)```

[abetlen]

@kawnah are you running the latest version of llama-cpp-python? This originally broke because of some changes to how cli args were being parsed and then a conflict with pydantic but it should be resolved in the latest pypi release.

[sweetcard]

@kawnah are you running the latest version of llama-cpp-python? This originally broke because of some changes to how cli args were being parsed and then a conflict with pydantic but it should be resolved in the latest pypi release.

Still running into the same issue.
llama_cpp_python: version 0.2.64
@abetlen