Image Splicing Localization Using CNN

This repo is part of project in Asia University Machine Learning Camp 2018

This repo is part of the code in paper "Image splicing localzation via semi-global network and fully connected conditional random fields",accepted by ECCV Workshop on Objectionable Content and Misinformation 2018 [PDF]

What is Image Splicing

Spliced image is created from two authentic images. By masking the part of donor image, the selected region is pasted to the host image after some operations (translation and rescale the donor region). Sometimes, several post-processing techniques(such as Gaussian filter on the border of selected region) are used to the spliced region for the harmony of the selected region and host image.

Methods

As shown in bottom figure, We address the problem of image splicing localization: given an input image, localizing the spliced region which is cut from another image. We formulate this as a classification task but, critically, instead of classifying the spliced region by local patch, we leverage the features from whole image and local patch together to classify patch. We call this structure Semi-Global Network. Our approach exploits the observation that the spliced region should not only highly relate to local features (spliced edges), but also global features (semantic information, illumination, etc.) from the whole image. We show that our method outperforms other state-of-the-art methods in Columbia datasets.

Installation

• Python 3.6
• PyTorch 0.4

You need to install the requirements by follow command firstly:

pip install -r requirements.txt

Make Dataset

We use Columbia Dataset for training and testing. We Split all the spliced images(in subfolder 4cam_splc) as three folds. Training(65%), validation(15%), testing(25%). For training faster, we firstly made patches dataset offline:

python tools/make_dataset_columbia /path/to/dataset

This script will generate image paches and resized full image for training and testing. So we have dataset:

	training	validation	testing
patches	14k	3k	5k

Training

You need to modify the parameters in shell script in train_local.sh for training the model, the full paramters list can be found in hybird.py:

python hybird.py\
  --epochs 60\
  --lr 1e-4\
  -c checkpoint/local\
  --arch sgn\
  --train-batch 64\
  --data columbia64\
  --base-dir /Users/oishii/Dataset/columbia/ 

Watching

We use TensorboardX to watch the training process, just install it by the readme.

run the watching commond as :

tensorboard --logdir ./checkpoint

Results

Here we show some sample results of our methods, from the left to right are the output of label, the output of mask and the ground truth mask:

Here are the label loss, segmentation loss, label accuracy, segmentation accuracy on validation set:

Acknowledgements

This work is partially support by Jeju National University and JDC (Jeju Free International City Development Center).