-
llama-cpp-python
version0.2.29
has a serious issue abetlen/llama-cpp-python#1089 - Introduce unit tests to update to newerllama-cpp-python
versions confidently. - try https://huggingface.co/TheBloke/Starling-LM-7B-alpha-GGUF (also the beta version).
- try https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf
- try Chat Templates https://medium.com/@ahmet_celebi/demystifying-chat-templates-of-llm-using-llama-cpp-and-ctransformers-f17871569cd6
- make docker container