Meme Generation

Introduction

This project's intention is to create a model for generating funny memes for the image. The model will be trained on the images and their corresponding captions. For every image we have multiple captions from which the model will learn the context of the image. The project has been picked from - GitHub Repo. They scraped the data from Meme Generator Website which is not active anymore. So we have used the data which they have stored at Path.

Setup

Cloning the Repository with Large Files (Using Git LFS)

Prerequisites
Before cloning the repository, ensure that you have the following prerequisites installed:

1. Git: You need Git installed to clone the repository. If you don't have Git installed, download and install it from Git's official site.

2. Git LFS: Since this repository uses Git Large File Storage (LFS) for managing large files, Git LFS must be installed on your system. You can download it from Git LFS's official site.

Cloning the Repository

1. Clone the Repository using command:

   git clone <repository-url>
   

2. Initialize Git LFS:
   Either globally

   git lfs install
   

   OR local to the specific repository

   cd <repository-name>
   git lfs install --local
   

   When the repository is cloned you would see the .best.pth files, but the size would be in KB which indicates that the large file contents are not downloaded. To download the contents perform step 3.

3. Pull the LFS Files:
   To download the large files stored with LFS, use:

   git lfs pull
   

NOTE: If the git lfs pull fails with the quota error, download the model checkpoints from the following Google Drive links and place them in their specified location:

• Inceptionv3 - https://drive.google.com/file/d/1J4W7MHajN8qROBUOWStM6MOcmOnPprkg/view?usp=drive_link (copy to path /checkpoints_inceptionv3/MemeGenerationInceptionv3)
• ResNet50 - https://drive.google.com/file/d/1ws3YdqAxn9paBYvSYR3s9sBFHH1HwdOO/view?usp=sharing (copy to path /checkpoints_resnet/MemeGenerationResnet)

Seamless Installation of Libraries

To have a seamless installation of libraries and proper installation of the cudatoolkit & cudnn, use the conda environment in VS Code.

1. Install the miniconda
2. Create a conda virtual environment and initialize it in VS Code using commands -
   a. conda create --name venv python=3.10
   b. conda init cmd.exe
3. Activate the virtual environment

   conda activate venv
   

4. Install the cudatoolkit and cudnn using conda

   conda install cudatoolkit cudnn
   

5. Install the other required libraries using requirements.txt file

   pip install -r requirements.txt
   

Meme Generated by Model

The meme generated on 2 unseen images -