Pen is a native MacOS application for the Llama 🦙 family of large language models. This app runs locally and is based on the C/C++ port by Georgi Gerganov with some help from Jed Fox.
To get started you'll need access to the Llama weights which you can request here. Follow the instructions here to convert the model to C/C++. That's it!
- Highlight generated tokens with a different color based on token probabilities
- Add support for Alpaca/Vicuna
- Add a Chat mode
- Isolate bug where model loading fails after launching the app.