Skip to content

alpv95/Dank-Learning

master
Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Dank Learning: Generating Memes Using Deep Neural Networks

Alt text Alt text Alt text Alt text

Overview

This is the code for the paper Dank Learning: Generating Memes Using Deep Neural Networks (https://arxiv.org/abs/1806.04510) and for the associated iOS app Dank Learning (https://danklearning.com/).

Abstract

We introduce a novel meme generation system, which given any image can produce a humorous and relevant caption. Furthermore, the system can be conditioned on not only an image but also a user-defined label relating to the meme template, giving a handle to the user on meme content. The system uses a pretrained Inception-v3 network to return an image embedding which is passed to an attention-based deep-layer LSTM model producing the caption - inspired by the widely recognised Show and Tell Model. We implement a modified beam search to encourage diversity in the captions. We evaluate the quality of our model using perplexity and human assessment on both the quality of memes generated and whether they can be differentiated from real ones. Our model produces original memes that cannot on the whole be differentiated from real ones.

Apple App available!

Try the meme generator out for yourself: see https://danklearning.com/

Just here for the data?

The data we scraped using scraper.py can be found in im2txt/memes (images) and im2txt/captions.txt (associated captions).

Installation

Clone the repo

git clone git@github.com:alpv95/MemeProject

Now cd in

cd Dank-Learning

Create a virtual environment and activate it

python -m virtualenv venv
source venv/bin/activate

Now install dependencies

pip install tensorflow Pillow jupyter

We've forked our own tf-coreml, which you can install in the virtual environment by doing the following:

cd tf-coreml/
pip install -e .
cd ..

Copy in the big files (word embedding matrix & trained DNN weights) from this Google Drive: https://drive.google.com/drive/folders/1R8YRzh0LWno6TiQBTtdoNP73D5GiYviF?usp=sharing

cp [path]/REAL_EMBEDDING_MATRIX im2txt/REAL_EMBEDDING_MATRIX
cp -r [path]/trainlogIncNEW im2txt/trainlogIncNew

Now you should be all set up.

Running

Get Jupyter notebooks fired up

jupyter notebook

Navigate in the browser that launched to im2txt/Meme_Maker.ipynb and follow the steps in the notebook to generate your own memes! Also see im2txt/model_conversion_debug.ipynb for how we converted the model to tf-coreml for use on iOS.

About

Dank Learning codebase, generate a meme from any image using AI. Uses a modified version of the Show and Tell image captioning network

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published