Skip to content


Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time


This repository contains the dataset and code for our ICWSM 2021 paper: Exercise? I thought you said ‘Extra Fries’: Leveraging Sentence Demarcations and Multi-hop Attention for Meme Affect Analysis.

In this paper, we attempt to solve the three tasks suggested in the SemEval’20-Memotion Analysis competition. We propose a multi-hop attention-based deep neural network framework, called MHA-Meme, whose prime objective is to leverage the spatial-domain correspondence between the visual modality and various textual segments of an Internet meme to extract fine-grained feature representations for meme sentiment and affect classification. We evaluate MHA-Meme on the ‘Memotion Analysis’ dataset for all three sub-tasks - sentiment classification, affect classification, and affect class quantification. Our comparative study shows state-of-the-art performances of MHA-Meme for all three tasks compared to the top systems that participated in the competition. Unlike all the baselines which perform inconsistently across all three tasks, MHA-Meme outperforms baselines in all the tasks on average. Moreover, we validate the generalization of MHA-Meme on another set of manually annotated test samples and observe it to be consistent. Finally, we establish the interpretability of MHA-Meme.

MHA-Meme Architecture

Segmented Data Format

The foreground texts are critical in extracting the semantic level information from meme. However, they require special attention depending upon their position in the meme and their reference to a specific region of the background image. Therefore, in the current work, we propose an attentive deep neural network architecture, called MHA-Meme (Multi-Hop Attention for Meme Analysis), to carefully analyze the correspondence between the background image and each text segment at different spatial locations. To do so, at first, we perform OCR (optical character recognition) to extract texts from the meme, and segment them into l sequence of text depending upon their spatial positions. Next, we process each textual segment ti separately by establishing their correspondence with the background image I. The segmented text of each meme in the training ans test set is available at dataset/train_splitted_all_tasks.csv and dataset/test_splitted_all_tasks.csv. Example format of segmented meme text with correspondng label:

Image Name Segment #1 Segment #2 Segment #3 Segment #4 Segment #5 Segment #6 Segment #7 Segment #8 Segment #9 Segment #10 Segment #11 Segment #12 Segment #13 Segment #14 Sentiment Humor Sarcasm Offense Motivation
avengers_new-avenger-endgame-funny-memes-13.jpg YOU ARE BEAUTIFUL GO TO HELL YOU ARE REALLY POWERFUL GO TO HELL CAN YOU LEAD THE AVENGERS REALLY? GO TO HELL 0 0 0 0 0 0 0 positive funny not_sarcastic very_offensive motivational


Please cite the following paper if you find this segmented dataset and MHA-Meme architecture useful in your research:

      title={Exercise? I thought you said 'Extra Fries': Leveraging Sentence Demarcations and Multi-hop Attention for Meme Affect Analysis}, 
      author={Shraman Pramanick and Md Shad Akhtar and Tanmoy Chakraborty},

Run the code

Clone the repository:

git clone
cd MHA-Meme-Affect-Analysis

You will need python >= 3.6. Start by creating a virtual environment to run the code in:

python3 -m venv env
source env/bin/activate

Install all the requirements in the virtual environment:

pip install --upgrade pip
pip install -r requirements.txt
pip install bcolz

Train the model for Sentiment Classification:


Train the model for affect classification:



No description, website, or topics provided.







No releases published


No packages published