Chatify

This is the github repository for the Chatify application, which allows a user to interact in an AI chatbot style conversation to perform various tasks, such as asking questions, summarizing a set of data, and so much more.

More specifically, one of the core functionalities of this applications is the ability to ingest your own data - in a document or file format - and use Retrieval Augment Generation (RAG) to allow the LLM to perform queries on more up-to-date data. This ensures that your interactions with the chatbot is consistent, accurate, and reliable.

NOTE: This application DOES NOT support uploading images files!

Instructions

Before running the application, there are various APIs and tools which have to be installed in order to use this application properly. The next sections will dive into the setup guide for each use case. The technologies and architecture will also be described in a separate section. From localizing the chat history conversation in MySQL to modifying the Knowledge Store that the LLM uses to perform Retrieval Augment Generation (RAG), these allow the application to work offline without an internet connection! This allows the user to have their own data and perform LLM queries privately and securely (provided you aren't connected to the internet).

Setup

This section will go over how to use this repository and the code. Before we use Git to clone this repo into the local machine for testing, you will need to install the following:

1. Ollama
2. MySQL
3. Python Version >= 3.12
4. Environment Variables via Python Env

NOTE: You will either need to have a Windows or Mac computer in order to run this application successfully!

Now we will dive into each service/API and walkthrough how to set it up!

Installing Ollama and Models

DISCLAIMER: Before you try to install Ollama and the models locally onto your machine, make sure that you have sufficient space in your SSD or Hard Drive. Minimum of 23.574 GB is required!

The first service is installing Ollama by using the link:

https://ollama.com/

Once you have installed Ollama, you will be able to start installing various models with different use cases. Now, open your terminal (Windows/Mac) to access Ollama programatically via a command line interface:

From there, please run the following command:

ollama --version

If you have installed it successfully, it should show a version number of the Ollama application!

Now that you have Ollama successfully installed, please install the following models:

"Mistral Nemo - 12B", "Mistral - 7B", "Gemma 2 - 9B

1. Llama 3.1 - 8B
2. Llama 3.2 - 3B
3. Mistral Nemo - 12B
4. "Mistral - 7B"
5. Gemma 2 - 9B

You will also need to install the embeddings model:

nomic-embed-text

If you are unsure the commands to run, feel free to search the respective models and it will dispay the commands to run the models:

https://ollama.com/search

The commands will be of the form:

ollama run MODELNAME

You will see a progress indicator on the status of the model on your local machine.

Once all of the models are installed, run the following command to verify all the models are installed:

ollama list

You have now installed Ollama successuly and the necessary models as part of the application!

Next, we will move on to setting up MySQL.

Install Homebrew, MySQL and Setting up Tables

First, let's focus on setting up MySQL for a Mac computer. You will need to install Homebrew, and if you have not already, use the following instructions:

1. Open the Mac Terminal and run the following command:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

2. Follow the instructions on-screen and once it is done, homebrew is now successfully installed. You can start installing various packages.

3. Now that brew is successfully installed, you will now install MySQL by running the following terminal command:

brew install mysql

4. Homebrew will install the package and will notify you once MySQL is successfully installed through the terminal.

If you want another way to install it, you can use this installer below:

https://dev.mysql.com/downloads/shell/

This will work for both Mac and Windows respectively!

Now, we can start setting up MySQL. Start by following these instructions:

1. Start the MySQL server:

   mysql.server start 
   

2. Connect to the MySQL server:

   mysql -u root -p 
   

   If you don't have a password setup already, MySQL will prompt you to create a user (root in this case) followed by setting your password. NOTE: Make sure to remember your password as you will need it to export as an environmnet variable!

3. Create a database called "Chat Application":

   create database chat_application;
   

4. Use the database

   use database chat_application;
   

5. Create the table "Chat History"

   create table chat_history (id INT AUTO_INCREMENT, user VARCHAR(100) NOT NULL, name VARCHAR(200) NOT NULL, date timestamp NOT NULL, total_tokens INT NOT NULL, total_cost INT NOT NULL, PRIMARY KEY(id)) DEFAULT CHARSET=utf8mb4;
   

6. Create the table "Chat History Messages"

   create table chat_history_messages (id INT AUTO_INCREMENT, group_uuid varchar(8) NOT NULL, role VARCHAR(25), content mediumtext NOT NULL,  PRIMARY KEY(id)) DEFAULT CHARSET=utf8mb4;
   

7. Verify that the character set is valid:

   SELECT default_character_set_name FROM information_schema.SCHEMATA WHERE schema_name = "chat_application";
   

Now you have successfully installed Homebrew (for Mac computers), installed the MySQL Shell, and started the local server with proper credentials along with two tables!

Next, we will move on to the final configurations, which are to install VSCode and Git (if you don't have those already), followed by setting up the python environment.

Setup Python Environment, Install Git and VSCode

Now all that is left is ensuring that you have the proper version of Python installed. Make sure to install a version that is at least >= 3.12. You can try to use older versions of Python but may come across compability issues with packages and/or dependencies

If you have not already installed Git, please follow the instructions on this website:

https://git-scm.com/book/en/v2/Getting-Started-Installing-Git

This will go through how to properly install for either Windows or MacOS users.

Next, for installing VSCode, use this link to download the installer and go through the setup process:

https://code.visualstudio.com/download

Once you have installed both of them, you can now clone this repo by running:

git clone https://github.com/RSB19/Chatify.git

Now that you have cloned the repo in your local machine, open VSCode and open the folder (or directory) where the repo was cloned. In MacOS, it usually resides within your Desktop directory. For Windows, it works a little different so make sure to find it using file explorer.

Next, once you open the repository in VSCode, ensure that you can see all of the files.

From there, you will need to create a python enviromnet. Now is the time to check if python is ready to go in the VSCode terminal below:

python --version

If python is installed (install if you don't see a version number), you can now run this command to create the local python environment:

python3 -m venv .env

This will now create the environment and in order to activate it, run:

source .env/bin/activate

Now, you can install the dependencies to this local enviromnent by running

pip install -r requirements.txt

This command will install all of the necessary package dependencies needed to run the application successfully.

If for some reason you are not able to access pip, please try reinstalling python via Homebrew or other methods if you are on a Windows computer.

Finally, all that is left is the enviromnet variables. Run the following command in the VSCode terminal:

printenv

You will see that there are various enviroment variables that the OS and Python uses to keep track of systemwide processes, applications, etc.

Here, you can export a new variable by typing:

export ENV='debug'

If you want additional terminal outputs, you can keep it in debug or leave it by typing none.

Now, export the MySQL password below:

export MYSQL_ROOT_PASSWORD=`--> PASSWORDHERE <--`

Replace PASSWORDHERE with the actual password that you created when you setup MySQl.

You have now setup all of the required services and APIs needed to run the application.

Running the Application

Before you run the application, assuming that you went through the setup process already, run the following commands in your terminal:

1. Start the MySQL Server

mysql.server start

2. Start Ollama

ollama serve

These are important steps as Langchain will need to recognize your local offline Ollama server. Also, the local MySQL server has to be started for the application to detect!

Finally, in order to run the application, make sure that you are in the directory Application. You can do this by running:

cd Application

You can now run the following command via streamlit to run the application:

streamlit run Chat.py

The application is now running and you can start to use it.

Contact

If you have any questions or suggestions to improve this application, feel free to email or contact on Github below:

EmaiL: bellamrohit@gmail.com

Copyright © Rohit Bellam