Install | Download LLM | Create Discord Application | Configuration | Run

Python Discord LLM (Large Language Model) Chatbot

A basic Discord chatbot using Discord.py, LangChain, and GPT4ALL. The bot is designed to interact with Discord channel users in a conversational manner, leveraging large language models (LLMs), and conversational memory to provide engaging responses.

Installation

Important

• Requires atleast Python 3.10 to work
  
• Only tested on a GeForce GPU with 12GB of VRAM. Not sure if cards with less VRAM will work properly using the LLM referenced below.

1. Create and activate a virtual enviroment

conda update conda
conda create -n <env name> python=3.10
conda activate <env name>

2. Create a new folder for this repository

mkdir <repo>
cd <repo>

3. Create a treeless, shallow clone of this repository

git clone -n --depth=1 --filter=tree:0 https://github.com/shaunbarnard/python.git
cd python

4. Enable the sparse-checkout feature and specify the folder you want to clone (discord-llm-chatbot)

git sparse-checkout set --no-cone discord-llm-chatbot

5. Check out the contents of the specified folder (llm-discord-chatbot)

git checkout
cd discord-llm-chatbot

6. Install the necessary Python dependencies

pip install -r requirements.txt

Download a LLM (Large Language Model)

Skip this section if you already have a model, otherwise head over to HuggingFace.co and choose a model to download.

If you have shitty internet, you're gonna want to create a custom download script to avoid wanting to punch a hole through your monitor...

1. Create a new file

sudo nano red.sh

2. Copy and paste the following into the newly created file:

#!/bin/bash

while true
do
    wget -c --retry-connrefused --waitretry=1 --read-timeout=20 --timeout=15 -t 0 "https://huggingface.co/TheBloke/Mistral-7B-v0.1-GGUF/resolve/main/mistral-7b-v0.1.Q4_K_M.gguf?download=true"
    sleep 10s
done

Note: If you need to replace the URL, make sure to include the '?download=true' parameter.

3. Set the download script file permissions to executable

sudo chmod 775 red.sh

4. Run the download script

./red.sh

Note: The download script will contiunally retry and resume failed downloads, even if connection cuts out. Once download is complete, you'll need to manually kill the script.

5. Rename the downloaded model

sudo mv 'mistral-7b-v0.1.Q4_K_M.gguf?download=true' 'mistral-7b-v0.1.Q4_K_M.gguf'

Create a Discord Application:

Create a Discord Bot

1. Log in to the Discord Developer Portal
2. Click on the "New Application" button
3. Give your application a name and confirm the creation

Create a Bot Account

1. Navigate to the "Bot" tab to configure the bot
2. Check the "Public Bot" option if you want others to invite your bot
3. Tick the "bot" checkbox under "scopes" and set the necessary permissions for your bot
4. Copy the bot's token to use it in your bot's code

Add the Bot to a Discord Server

1. Use the bot's token to invite the bot to your Discord server
2. Set the appropriate permissions for your bot

Note: Above steps are a general overview of the process. For detailed instructions and best practices, you can refer to the Offical Discord Documentation

Configuration

1. Set the model path in bot.py to your model's location.

#Set model path
model_path = ("mistral-7b-v0.1.Q4_K_M.gguf")

2. Adjust the remaining configuration settings in bot.py as necessary

BOT_PREFIX_ON = True
BOT_PREFIX = '/b'
BOT_CHANNEL_MESSAGE_ON = True
BOT_CHANNEL_MESSAGE = "Hey guys! Use '/b' followed by your message to interact with me ;)"

3. Paste your API Token into .env file

API_KEY=<your bot's discord token>

Run the bot

python bot.py