66 feat load unload model with config at runtime

[tikikun]

Example curl to load model

curl -X POST 'http://localhost:3928/inferences/llamacpp/loadModel' \
-H 'Content-Type: application/json' \
-d '{
    "llama_model_path": "/Users/alandao/Documents/codes/nitro.cpp_temp/models/llama2_7b_chat_uncensored.Q4_0.gguf",
    "ctx_len": 2048,
    "ngl": 100,
    "embedding": true
}'

[tikikun]

Example curl to load model

curl -X POST 'http://localhost:3928/inferences/llamacpp/loadModel' \
-H 'Content-Type: application/json' \
-d '{
    "llama_model_path": "/Users/alandao/Documents/codes/nitro.cpp_temp/models/llama2_7b_chat_uncensored.Q4_0.gguf",
    "ctx_len": 2048,
    "ngl": 100,
    "embedding": true
}'