No output when using command line

[amusingCloud]

When I try to use koboldcpp from command line it exits without any output. I am using the following command:

koboldcpp_nocuda.exe --model Phi-3.5-mini-instruct-Q4_K_M.gguf --usecpu --skiplauncher --prompt "hello world!"

Adding --debugmode did not produce any additional output.

However, when I simply lanuch using koboldcpp_nocuda.exe, I am able to inference using GUI with the same model.

Additional Information:
Koboldcpp 1.75.2
Using on Windows 11

[jojorne]

Is this intended?

if (args.model_param or args.model) and args.prompt and not args.benchmark:
        suppress_stdout()

We have model, we have prompt and we don't have benchmark so suppress it.

[LostRuins]

Hi both, sorry for the late reply. Yes, it's intended that output is suppressed when running with --prompt without benchmark - this is to allow for CLI usage where the LLM generated output is piped into a different program.

However, receiving no response is very strange and possibly indicates an error was encountered.

Can you try use the same command again, with a different prompt? Try the prompt "Hi, my name is" which should definitely give a reply.

If that fails, please run with the --benchmark flag.