Bug: `-ins` command gone from main.exe

[dillfrescott]

What happened?

There seems to be no way to activate instruct mode in main.exe, this causes my scripts to break.

Name and Version

version: 3089 (c90dbe0)
built with MSVC 19.39.33523.0 for x64

What operating system are you seeing the problem on?

Windows

Relevant log output

error: unknown argument: -ins

[deepbluesea]

inp_pfx and inp_sfx are missing as well

any reference of instruct mode is gone

[acasto]

Yeah, I'm having a hard time figuring out what changed, unless I missed something obvious in the git logs. '-ins' is gone, thought maybe it became '-i' which works for most of my stuff but all my previously working scripts needed changes to templates and have a habit of talking to themselves now. Had to turn the temp down by half on codestral to mostly get it to act normal.

[arch-btw]

Instruct was mostly for alpaca models due to the way it prefixed the prompts.
Unless you're using alpaca models, you're most likely looking for --interactive and --interactive-first.

I don't know if that's the reason it was removed though, but here's the commit: #7675

[arch-btw]

You can read more here: https://github.com/ggerganov/llama.cpp/blob/master/examples/main/README.md

It will probably be removed from the README in the future, so here's a screenshot:

[ggerganov]

The reason to remove -ins and -chatml is because they are special-cases of chat templates. It's better to implement general-purpose chat template usage instead

[dillfrescott]

Should I close this because it was intentionally removed and not a bug or accident?

[tjkirch]

The README still has a section titled "instruct mode" that refers to examples/alpaca.sh which uses -ins.

I wish it were clear what we're supposed to use instead of -ins. Clearly -if is required, otherwise it starts generating random text without waiting for a prompt, but even with -if there's no prompt marker like > to indicate that it's ready and waiting for a prompt.

[ggerganov]

main does not support chat / instruct templates - it's something to be done in the future. There might be some poor-man workaround using --in-prefix, --in-suffix + --interactive for some of the chat templates, but one has to figure it out on a case by case basis

[strawberrymelonpanda]

main does not support chat / instruct templates - it's something to be done in the future.

Personally I use custom python scripts to build the prompt (if non-interactive) or to build --in-prefix, --in-suffix + --interactive as you say (if interactive) before sending to main, but patiently waiting for that feature.