Name and Version
release 4604
Operating systems
All
Which llama.cpp modules do you know to be affected?
llama-server
Command line
llama-server --model $MODEL --temp 0
Problem description
When setting prompt sampling parameters (e.g. temperature), they are ignored and silently overridden by the settings of llama-server's web interface.
The user has demonstrated clear intent to change the temperature via command line options, yet it has no effect. If that is intended behavior, the server should throw an error message and refuse operating. A warning would be to lenient, because the server is dumping so much output that nobody is going to read that anyway.
In addition, there should be an option to change the defaults of the web UI via CLI. It requires too much effort to change the settings in the UI manually, especially when providing access to the UI to people who have no basic understanding of LLMs.
Name and Version
release 4604
Operating systems
All
Which llama.cpp modules do you know to be affected?
llama-server
Command line
llama-server --model $MODEL --temp 0Problem description
When setting prompt sampling parameters (e.g. temperature), they are ignored and silently overridden by the settings of llama-server's web interface.
The user has demonstrated clear intent to change the temperature via command line options, yet it has no effect. If that is intended behavior, the server should throw an error message and refuse operating. A warning would be to lenient, because the server is dumping so much output that nobody is going to read that anyway.
In addition, there should be an option to change the defaults of the web UI via CLI. It requires too much effort to change the settings in the UI manually, especially when providing access to the UI to people who have no basic understanding of LLMs.