This Discord bot lets you use GPT-J and LLaMa models to both participate in your conversations and answer your questions in thread channels, in a ChatGPT-like fashion!
On a fresh Debian 11 install, you'll need the following packages: git g++ make cmake zlib1g-dev libssl-dev libsqlite3-dev. For better performance with large batch sizes, libopenblas-dev is recommended.
Then, you can clone this repository including all its submodules:
git clone https://gitlab.com/niansa/discord_llama.git --recursive
Once the command has completed, create a build directory, configure the project and compile it:
cd discord_llama
mkdir build
cd build
cmake ..
make -j$(nproc)
We should now download a model. Let's start with gpt4all-j-v1.3-groovy! I've already written an example model config for it, so let's just grab that and download the model:
mkdir models
cd models
cp ../../example_models/gpt4all-j-v1.3-groovy.txt .
wget https://gpt4all.io/models/ggml-gpt4all-j-v1.3-groovy.bin
cd ..
Next, we need a configuration file. Let's just grab the example config:
cp ../example_config.txt ./config.txt
and edit it. For reference, you can read explained_config.txt.
Please note that gpt4all-j-v1.3-groovy is only capable of instruct mode, so you'll have to set threads_only to true, if you've downloaded just that model.
Finally, start the Discord Bot by passing the config file to the executable:
./discord_llama config.txt
And that's it! Feel free to play around, try different models, tweak the config file, etc... It's really easy! If you still have any questions, please write an issue or contact me on Discord: Tuxifan#0981.
A huge thank you to Nomic AI for helping me get this project to where it is now! Also, this project wouldn't have been possible without the effort of Georgi Gerganov (ggerganov), who has writte the core of it all, the inference code itself.