Implementation of "Visual Sentiment Prediction based on Automatic Discovery of Affective Regions"
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
caffe-master @ 2a1c552
code
images
.DS_Store
.gitmodules
LICENSE
README.md

README.md

Visual Sentiment Prediction based on Automatic Discovery of Affective Regions

By Jufeng Yang, Dongyu She, Ming Sun, Ming-Ming Cheng, Paul L. Rosin and Liang Wang

Introduction

We propose a framework for sentiment classification and affectiv regions discovery with deep ConvNets.

  • It achieves state-of-the-art performance on sentiment classification, and very competitive results on object discovery.
  • Our code is written by matlab, based on Caffe.

The paper has been accepted by TMM 2018. For more details, please refer to our paper.

Architecture

framework

License

Our framework is released under the MIT License (refer to the LICENSE file for details).

Citing

If you find our framework useful in your research, please consider citing:

@article{tmm2018visual,
	Author = {Jufeng Yang, Dongyu She, Ming Sun, Ming-Ming Cheng, Paul L. Rosin, Liang Wang},
	Title = {Visual Sentiment Prediction based on Automatic Discovery of Affective Regions},
	journal = {TMM},
	volume = 99,
	numer = 99,
	pages = {1-1},
	Year = {2018}
}

Contents

  1. Requirements: software
  2. Requirements: hardware
  3. Basic installation
  4. Extra Downloads (objectness)
  5. Extra Downloads (trainval split)
  6. Usage
  7. Trained models

Requirements: software

  1. Requirements for Caffe and matcaffe (see: Caffe installation instructions)

Note: Caffe must be built with support for Matlab!

# This is required only if you will compile the matlab interface.
# MATLAB directory should contain the mex binary in /bin.
MATLAB_DIR := /usr/local
  1. MATLAB

Requirements: hardware

  1. NVIDIA GTX TITANX (~12G of memory)

Installation

  1. Clone the repository
# Make sure to clone with --recursive
git clone --recursive https://github.com/sherleens/AR_discovery.git
  1. Build Caffe and matcaffe
cd $AR_ROOT/caffe
# Now follow the Caffe installation instructions here
#   http://caffe.berkeleyvision.org/installation.html
# If you're experienced with Caffe and have all of the requirements installed
# and your Makefile.config in place, then simply do:
make all -j 8 && make matcaffe

Download pre-computed object proposals

Pre-computed bounding boxes generated by Objectness, which can also be downloaded here for affective datasets.

Download train val split

Train val split can be downloaded here for affective datasets.

Usage

Train a deep network. For example, train a VGG16 network on FI trainval.

cd prototxt && bash ./train_vgg_FI.sh

Test a deep network. For example, test the VGG 16 network on FI test:

run demo.m

Our trained models

The models trained on the FI dataset can be downloaded from here.