How to know what to use for templates in the yaml configuration?

[dpkirchner]

How do I look at a model file (.gguf), or its huggingface page, or whatever, and learn what I should use for the chat and complete templates? I'd rather not share a specific example because I'm looking for a general method that anyone can use when using any model.

[JackBekket]

this should be model-specific .yaml file config

name: wizard-uncensored-13b
context_size: 2048
# -----------------------------------GPU-----------
f16: false # true to GPU acceleration
cuda: true # true to GPU acceleration  basically make sense only for image-generation model, not LLM
gpu_layers: 10 # this model have max 40 layers, 15-20 is reccomended for half-load at NVIDIA 4060 TiTan (more layers -- more VRAM required), (0 is no GPU, CPU only. this is MAIN flag here.)
parameters:
  model: wizard-uncensored-13b.gguf # file of model or link to hugginface
#backend: diffusers
template:

  chat: &template |
    Below is an instruction that describes a task. Write a response that appropriately completes the request.
    Instruction: {{.Input}}
    Response:
  # Modify the prompt template here ^^^ as per your requirements
  completion: *template

If you are looking for templates you can look here: https://github.com/JackBekket/UncensoredGPT_tgbot/tree/master/prompt-templates

You can also run local-ai in debug mode and it will show promt before templating, template itself and what it get after

So, let's say you want to know what templates langchain uses for features like conversation memory -- so you can just run langchain application and look into logs of localai to know they promt templates