Prompt templates

[clarkmcc]

Right now the backend automatically formats all prompts with

USER: {prompt}\nSYSTEM:

but as I understand it, this needs to be configurable based on the model in order for the model to produce the best responses.

[clarkmcc]

Much better performance on llama 2 chat models when using the proper prompt templates

https://huggingface.co/TheBloke/Llama-2-13B-chat-GPTQ
https://huggingface.co/TheBloke/Llama-2-13B-chat-GPTQ/discussions/5

[clarkmcc]

LLaMA 2 Chat

Warmup

[INST] <<SYS>>
You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe.
<</SYS>>
Please answer questions about the following document. Answer only the questions, don’t give a lot of information that was not requested.[/INST]

Template

[INST]{{prompt}}[/INST]

Vicuna

Warmup

A chat between a human and an assistant.

Template

USER: {{prompt}}
ASSISTANT: