Skip to content
master
Go to file
Code

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
Oct 24, 2018
Oct 24, 2018
Oct 24, 2018

README.md

Joint Image Emotion Classification and Distribution Learning via Deep Convolutional Neural Network

By Jufeng Yang, Dongyu She, Ming Sun

Introduction

We develop a multi-task deep framework by jointly optimizing classification and distribution prediction

  • It achieves state-of-the-art performance on emotion classification, and LDL prediction tasks.
  • Our code is based on Caffe.

The paper has been accepted by IJCAI 2017. For more details, please refer to our paper.

Architecture

framework

License

Our framework is released under the MIT License (refer to the LICENSE file for details).

Citing

If you find our framework useful in your research, please consider citing:

@inproceedings{ijcai2018joint,
	Author = {Jufeng Yang, Dongyu She, Ming Sun},
	Title = {Joint Image Emotion Classification and Distribution Learning via Deep Convolutional Neural Network},
	booktitle = {IJCAI},
	Year = {2017}
}

Contents

  1. Requirements: software
  2. Requirements: hardware
  3. Basic installation
  4. Extra Downloads (dataset lmdb)
  5. Usage
  6. Trained models

Requirements: software

  1. Requirements for Caffe (see: Caffe installation instructions)
# This is required only if you will compile the matlab interface.

Requirements: hardware

  1. NVIDIA GTX TITANX (~12G of memory)

Installation

  1. Clone the repository
git clone https://github.com/sherleens/EmotionDistributionLearning.git
  1. Change the label number in $ROOT/caffe_KL/src/caffe/layers/data_layer.cpp
line52:    int labelNum = 7; //change to the corresponding label number with the LMDB file
...
line110:   int labelNum = 7; //for example, change to 7 for the Emotion6 dataset, 8 for the Flickr_LDL dataset 
  1. Build Caffe with KLloss
cd $ROOT/caffe_KL
# Now follow the Caffe installation instructions here
#   http://caffe.berkeleyvision.org/installation.html
# If you're experienced with Caffe and have all of the requirements installed
# and your Makefile.config in place, then simply do:
make all -j 8 && make matcaffe

Download dataset lmdb

LMDB file is generated by modified code (caffe_KL/tool/convert_imageset_r) to support multiple ground-truth labels, which can also be downloaded from Baidu, password:b83b.

Contents:

  • dataset_config

    • emotion6_config.mat
    • flickrldl_config.mat
    • twitterldl_config.mat
    • readme
  • lmdb_file

    • train_twitterldl_split1_lmdb.tar 1.5G
    • train_flickrldl_split1_lmdb.tar 1.67G
    • train_emotion6_lmdb.tar 265.4M
    • test_twitterldl_split1_lmdb.tar 384.7M
    • test_flickrldl_split1_lmdb.tar 426.8M
    • test_emotion6_lmdb.tar 113.7M

Usage

Train a deep network. For example, train a VGG16 network on distribution datasets.

cd prototxt && bash ./train_vgg_gs.sh

Our trained models

The models trained on the distribution datasets can be downloaded from here.

coming soon...

About

Implementation of "Joint Image Emotion Classification and Distribution Learning via Deep Convolutional Neural Network"

Resources

License

Releases

No releases published

Packages

No packages published
You can’t perform that action at this time.