Skip to content

Latest commit

 

History

History
executable file
·
113 lines (77 loc) · 3.47 KB

README.md

File metadata and controls

executable file
·
113 lines (77 loc) · 3.47 KB
Raw

Visual Sentiment Prediction based on Automatic Discovery of Affective Regions

By Jufeng Yang, Dongyu She, Ming Sun, Ming-Ming Cheng, Paul L. Rosin and Liang Wang

Introduction

We propose a framework for sentiment classification and affectiv regions discovery with deep ConvNets.

  • It achieves state-of-the-art performance on sentiment classification, and very competitive results on object discovery.
  • Our code is written by matlab, based on Caffe.

The paper has been accepted by TMM 2018. For more details, please refer to our paper.

Architecture

framework

License

Our framework is released under the MIT License (refer to the LICENSE file for details).

Citing

If you find our framework useful in your research, please consider citing:

@article{tmm2018visual,
	Author = {Jufeng Yang, Dongyu She, Ming Sun, Ming-Ming Cheng, Paul L. Rosin, Liang Wang},
	Title = {Visual Sentiment Prediction based on Automatic Discovery of Affective Regions},
	journal = {TMM},
	volume = 99,
	numer = 99,
	pages = {1-1},
	Year = {2018}
}

Contents

  1. Requirements: software
  2. Requirements: hardware
  3. Basic installation
  4. Extra Downloads (objectness)
  5. Extra Downloads (trainval split)
  6. Usage
  7. Trained models

Requirements: software

  1. Requirements for Caffe and matcaffe (see: Caffe installation instructions)

Note: Caffe must be built with support for Matlab!

# This is required only if you will compile the matlab interface.
# MATLAB directory should contain the mex binary in /bin.
MATLAB_DIR := /usr/local
  1. MATLAB

Requirements: hardware

  1. NVIDIA GTX TITANX (~12G of memory)

Installation

  1. Clone the repository
# Make sure to clone with --recursive
git clone --recursive https://github.com/sherleens/AR_discovery.git
  1. Build Caffe and matcaffe
cd $AR_ROOT/caffe
# Now follow the Caffe installation instructions here
#   http://caffe.berkeleyvision.org/installation.html
# If you're experienced with Caffe and have all of the requirements installed
# and your Makefile.config in place, then simply do:
make all -j 8 && make matcaffe

Download pre-computed object proposals

Pre-computed bounding boxes generated by Objectness, which can also be downloaded here (password: kai7) for affective datasets.

Download dataset config

Train val split can be downloaded here (password: kai7) for affective datasets.

Usage

Train a deep network. For example, train a VGG16 network on FI trainval.

cd prototxt && bash ./train_vgg_FI.sh

Test a deep network. For example, test the VGG 16 network on FI test:

run demo.m in matlab

The final prediction is saved in code/result.txt

Our trained models

The models trained on the FI dataset can be downloaded from here (password: kai7). <<<<<<< HEAD

=======

67f963aaa6ad5c0f2ec83c47960854fd196bdb2f