Vision Web App w/ Django

Access the app here.

Intro

This Django app uses GPT-4 and OpenCV to analyze video frames, generating a context-aware response based on these frames.

Navigation

• Features
• Installation
  • MySQL Database
  • Django Server
  • Celery and Redis
• Technologies

Features

• Video Frame Analysis: Utilizes OpenCV to analyze each frame of the video and pass it to the OpenAI API
• GPT-4: Uses the vision capabilities of GPT-4 to interpret the analyzed frames, ensuring responses are contextually relevant.
• Custom Instructions: Offers the flexibility to tailor the narration style to fit the video’s tone and audience by giving custom instructions to the model.
• Text-To-Speech: Generates Text-To-Speech audio files from the narration text using OpenAI or ElevenLabs TTS models.

Installation

Make sure you have libgl1-mesa-glx installed on your machine. This is needed for OpenCV.

sudo apt install libgl1-mesa-glx

Database Setup

*Skip this step if you already have a MySQL server or other database set up.

**Adjust your settings.py file accordingly if you are not using MySQL.

The easiest way to set up a MySQL server locally is with docker. Adjust the command to your needs. This command will use root as the only user and the password set in the run command as env variable.

docker run -d \ 
-p 3306:3306 \
-e MYSQL_ROOT_PASSWORD=your-password-here \
-e MYSQL_DATABASE=your-db-name-here \
--name mysql-db \
mysql:latest

You can connect to the mysql or bash shells with the following commands.

• MySQL Shell

docker exec -it mysql-db mysql -u root -p

• Bash Shell

docker exec -it mysql-db bash

SQLite (Alternative)

If you want to use the default SQLite database generated by Django modify DATABASES in settings.py with this.

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': BASE_DIR / 'db.sqlite3',
    }
}

Django Server

1. Create and activate a virtual environment.

   python3 -m venv env && \
   . env/bin/activate
   

2. Install dependencies with pip.

   pip3 install -r requirements.txt
   

3. Set up your environment variables by using the template and information at .env.default.

4. Run database migrations.

   python3 manage.py migrate
   

5. Collect static files.

   python3 manage.py collectstatic
   

   *You don't need to run this if your using the development server and serving static files locally. Only if you use S3 storage or a production server (e.g. gunicorn). Check .env.default file.

6. Run the development server.

   python3 manage.py runserver
   

If everything was set up correctly you should be able to access the app at: http://localhost:8000

Celery and Redis

You need to set up a Redis database in order for celery to work with the app which is used to process videos asynchronously. The server will start even without this and can be accessed but videos won't process.

1. You can use docker to run a Redis instance locally very easily.

   docker run -d -p 6379:6379 --name my-redis redis
   

2. Start a celery worker. You can adjust the parameters as needed.

   celery -A vision_app worker -l info --concurrency=1 --pool=solo 
   

Technologies Used

• Django
• MySQL
• Redis
• Celery
• OpenCV
• OpenAI