server: add more test cases for multiple models testing

[ngxson]

the test cases are inside test_router.py

currently, there is one one case: loading one model.

it can be interesting to add more of them (but not too many tests - as it will make the CI runs too slow)

these test cases can be interesting to add:

• test unloading model
• set --models-max 2, load 2 models at the same time, load one more model, the first model should be unloaded automatically
• test with --no-models-autoload
• test with API key

[ngxson]

tagging @ServeurpersoCom if you're interested. these small number of tests can already be useful, while not costing too much of running time on CI

[ServeurpersoCom]

(root|~/llama.cpp.pascal) cat test.sh
#!/bin/bash

cd tools/server/tests

source .venv/bin/activate
#pip install -r requirements.txt
pytest unit/test_router.py -v
deactivate

Current :

(root|~/llama.cpp.pascal) ./test.sh
===================================================================== test session starts ======================================================================
platform linux -- Python 3.11.2, pytest-8.3.5, pluggy-1.6.0 -- /root/llama.cpp.pascal/tools/server/tests/.venv/bin/python3
cachedir: .pytest_cache
rootdir: /root/llama.cpp.pascal/tools/server/tests
configfile: pytest.ini
plugins: anyio-4.12.0
collected 2 items

unit/test_router.py::test_router_chat_completion_stream[ggml-org/tinygemma3-GGUF:Q8_0-True] PASSED                                                       [ 50%]
unit/test_router.py::test_router_chat_completion_stream[non-existent/model-False] PASSED                                                                 [100%]

====================================================================== 2 passed in 6.74s =======================================================================
(root|~/llama.cpp.pascal)

Add 4 test cases for model router:
- test_router_unload_model: explicit model unloading
- test_router_models_max_evicts_lru: LRU eviction with --models-max
- test_router_no_models_autoload: --no-models-autoload flag behavior
- test_router_api_key_required: API key authentication

Tests use async model loading with polling and graceful skip when
insufficient models available for eviction testing.

utils.py changes:
- Add models_max, models_dir, no_models_autoload attributes to ServerProcess
- Handle JSONDecodeError for non-JSON error responses (fallback to text)

(root|~/llama.cpp.pascal) ./test.sh
===================================================================== test session starts ======================================================================
platform linux -- Python 3.11.2, pytest-8.3.5, pluggy-1.6.0 -- /root/llama.cpp.pascal/tools/server/tests/.venv/bin/python3
cachedir: .pytest_cache
rootdir: /root/llama.cpp.pascal/tools/server/tests
configfile: pytest.ini
plugins: anyio-4.12.0
collected 6 items

unit/test_router.py::test_router_chat_completion_stream[ggml-org/tinygemma3-GGUF:Q8_0-True] PASSED                                                       [ 16%]
unit/test_router.py::test_router_chat_completion_stream[non-existent/model-False] PASSED                                                                 [ 33%]
unit/test_router.py::test_router_unload_model PASSED                                                                                                     [ 50%]
unit/test_router.py::test_router_models_max_evicts_lru SKIPPED (Not enough models could be loaded to exercise eviction)                                  [ 66%]
unit/test_router.py::test_router_no_models_autoload PASSED                                                                                               [ 83%]
unit/test_router.py::test_router_api_key_required PASSED                                                                                                 [100%]

================================================================ 5 passed, 1 skipped in 13.85s =================================================================
(root|~/llama.cpp.pascal)

[ngxson]

test_router_models_max_evicts_lru SKIPPED (Not enough models could be loaded to exercise eviction)

Hmm yeah I'll move 2 models more test models to dedicated repo, so that it will show up on the list of cached models

[ngxson]

2 models with hf_repo are:

• ggml-org/test-model-stories260K
• ggml-org/test-model-stories260K-infill

Remember to set model_hf_file = None, code below (we can update existing test models):

    @staticmethod
    def tinyllama2() -> ServerProcess:
        server = ServerProcess()
        server.model_hf_repo = "ggml-org/test-model-stories260K"
        server.model_hf_file = None
        server.model_alias = "tinyllama-2"
        server.n_ctx = 512
        server.n_batch = 32
        server.n_slots = 2
        server.n_predict = 64
        server.seed = 42
        return server

...

    @staticmethod
    def tinyllama_infill() -> ServerProcess:
        server = ServerProcess()
        server.offline = True # will be downloaded by load_all()
        server.model_hf_repo = "ggml-org/test-model-stories260K-infill"
        server.model_hf_file = None
        server.model_alias = "tinyllama-infill"
        server.n_ctx = 2048
        server.n_batch = 1024
        server.n_slots = 1
        server.n_predict = 64
        server.temperature = 0.0
        server.seed = 42
        return server

[ngxson]

btw, hugging face backend technically support this syntax: <user>/<model>:<path_to_file>, for example: ggml-org/models:tinyllamas/stories260K.gguf

I completely forgot that until now, but I think the dedicated repo ggml-org/test-model-stories260K looks cleaner anyway