Permalink
Browse files

initial commit

  • Loading branch information...
0 parents commit d43a104503c6c186eb599524426453bab836f222 Jay Mahadeokar committed Sep 17, 2016
Showing with 3,674 additions and 0 deletions.
  1. +11 −0 LICENSE.md
  2. +47 −0 README.md
  3. +128 −0 classify_nsfw.py
  4. +3,488 −0 nsfw_model/deploy.prototxt
  5. BIN nsfw_model/resnet_50_1by2_nsfw.caffemodel
@@ -0,0 +1,11 @@
+
+Copyright 2016, Yahoo Inc.
+
+Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
@@ -0,0 +1,47 @@
+# Open nsfw model
+This repo contains code for running Not Suitable for Work (NSFW) classification deep neural network Caffe models.
+
+#### Not suitable for work classifier
+Detecting offensive / adult images is an important problem which researchers have tackled for decades. With the evolution of computer vision and deep learning the algorithms have matured and we are now able to classify an image as not suitable for work with greater precision.
+
+Defining NSFW material is subjective and the task of identifying these images is non-trivial. Moreover, what may be objectionable in one context can be suitable in another. For this reason, the model we describe below focuses only on one type of NSFW content: pornographic images. The identification of NSFW sketches, cartoons, text, images of graphic violence, or other types of unsuitable content is not addressed with this model.
+
+Since images and user generated content dominate the internet today, filtering nudity and other not suitable for work images becomes an important problem. In this repository we opensource a caffe deep neural network for priliminary filtering of NSFW images.
+
+#### Usage
+
+* The network takes in a image and gives output a probability (score between 0-1) which can be used to filter not suitable for work images. Scores < 0.2 indicate that the image is likely to be safe with high probability. Scores > 0.8 indicate that the image is highly probable to be NSFW. Scores in middle range may be binned for different NSFW levels.
+* Depending on the dataset, usecase and types of images, we advise developers to choose suitable thresholds. Due to difficult nature of problem, there will be errors, which depend on use-cases / definition / tolerance of NSFW. Ideally developers should create a evaluation set according to the definition of what is safe for their application, then fit a [ROC](https://en.wikipedia.org/wiki/Receiver_operating_characteristic) curve to choose a suitable threshold if they are using the model as it is.
+* ***Results can be improved by [fine-tuning](http://caffe.berkeleyvision.org/gathered/examples/finetune_flickr_style.html)*** the model for your dataset/ uscase / definition of NSFW. We do not provide any guarantees of accuracy of results. Please read the disclaimer below.
+
+#### Description of model
+We trained the model on the dataset with NSFW images as positive and SFW(suitable for work) images as negative. These images were editorially labelled. We cannot release the dataset or other details due to the nature of the data.
+
+We use [CaffeOnSpark](https://github.com/yahoo/CaffeOnSpark) which is a wonderful framework for distributed learning that brings deep learning to Hadoop and Spark clusters for training models for our experiments. Big thanks to to the CaffeOnSpark team!
+
+The deep model was first pretrained on ImageNet 1000 class dataset. Then we finetuned the weights on the NSFW dataset.
+We used the thin resnet 50 1by2 architecture as the pretrained network. The model was generated using [pynetbuilder](https://github.com/jay-mahadeokar/pynetbuilder) tool and replicates the [residual network](https://arxiv.org/pdf/1512.03385v1.pdf) paper's 50 layer network (with half number of filters in each layer). You can find more details on how the model was generated and trained [here](https://github.com/jay-mahadeokar/pynetbuilder/tree/master/models/imagenet)
+
+Please note that deeper networks, or networks with more filters can improve accuracy. We train the model using a thin residual network architecture, since it provides good tradeoff in terms of accuracy, and the model is light-weight in terms of runtime (or flops) and memory (or number of parameters).
+
+#### Running the model
+To run this model, please install [Caffe](https://github.com/BVLC/caffe) and its python extension and make sure pycaffe is available in your PYTHONPATH.
+
+We can use the [classify.py](https://github.com/BVLC/caffe/blob/master/python/classify.py) script to run the NSFW model. For convenience, we have provided the script in this repo as well, and it prints the NSFW score.
+
+ ```
+ python ./classify_nsfw.py \
+ --model_def nsfw_model/deploy.prototxt \
+ --pretrained_model nsfw_model/resnet_50_1by2_nsfw.caffemodel \
+ INPUT_IMAGE_PATH
+ ```
+
+#### ***Disclaimer***
+The definition of NSFW is subjective and contextual. This model is a general purpose reference model, which can be used for the preliminary filtering of pornographic images. We do not provide guarantees of accuracy of output, rather we make this available for developers to explore and enhance as an open source project. Results can be improved by [fine-tuning](http://caffe.berkeleyvision.org/gathered/examples/finetune_flickr_style.html) the model for your dataset.
+
+#### License
+Code licensed under the [BSD 2 clause license] (https://github.com/BVLC/caffe/blob/master/LICENSE). See LICENSE file for terms.
+
+#### Contact
+The model was trained by [Jay Mahadeokar](https://github.com/jay-mahadeokar/), in collaboration with [Sachin Farfade](https://github.com/sachinfarfade/) , [Amar Ramesh Kamat](https://github.com/amar-kamat), [Armin Kappeler](https://github.com/akappeler) and others. Special thanks to Gerry Pesavento for taking the initiative for open-sourcing this model. If you have any queries, please raise a issue and we will get back ASAP.
+
@@ -0,0 +1,128 @@
+#!/usr/bin/env python
+"""
+Copyright 2016 Yahoo Inc.
+Licensed under the terms of the 2 clause BSD license.
+Please see LICENSE file in the project root for terms.
+"""
+
+import numpy as np
+import os
+import sys
+import argparse
+import glob
+import time
+from PIL import Image
+from StringIO import StringIO
+import caffe
+
+
+def resize_image(data, sz=(256, 256)):
+ """
+ Resize image. Please use this resize logic for best results instead of the
+ caffe, since it was used to generate training dataset
+ :param str data:
+ The image data
+ :param sz tuple:
+ The resized image dimensions
+ :returns bytearray:
+ A byte array with the resized image
+ """
+ img_data = str(data)
+ im = Image.open(StringIO(img_data))
+ if im.mode != "RGB":
+ im = im.convert('RGB')
+ imr = im.resize(sz, resample=Image.BILINEAR)
+ fh_im = StringIO()
+ imr.save(fh_im, format='JPEG')
+ fh_im.seek(0)
+ return bytearray(fh_im.read())
+
+def caffe_preprocess_and_compute(pimg, caffe_transformer=None, caffe_net=None,
+ output_layers=None):
+ """
+ Run a Caffe network on an input image after preprocessing it to prepare
+ it for Caffe.
+ :param PIL.Image pimg:
+ PIL image to be input into Caffe.
+ :param caffe.Net caffe_net:
+ A Caffe network with which to process pimg afrer preprocessing.
+ :param list output_layers:
+ A list of the names of the layers from caffe_net whose outputs are to
+ to be returned. If this is None, the default outputs for the network
+ are returned.
+ :return:
+ Returns the requested outputs from the Caffe net.
+ """
+ if caffe_net is not None:
+
+ # Grab the default output names if none were requested specifically.
+ if output_layers is None:
+ output_layers = caffe_net.outputs
+
+ img_data_rs = resize_image(pimg, sz=(256, 256))
+ image = caffe.io.load_image(StringIO(img_data_rs))
+
+ H, W, _ = image.shape
+ _, _, h, w = caffe_net.blobs['data'].data.shape
+ h_off = max((H - h) / 2, 0)
+ w_off = max((W - w) / 2, 0)
+ crop = image[h_off:h_off + h, w_off:w_off + w, :]
+ transformed_image = caffe_transformer.preprocess('data', crop)
+ transformed_image.shape = (1,) + transformed_image.shape
+
+ input_name = caffe_net.inputs[0]
+ all_outputs = caffe_net.forward_all(blobs=output_layers,
+ **{input_name: transformed_image})
+
+ outputs = all_outputs[output_layers[0]][0].astype(float)
+ return outputs
+ else:
+ return []
+
+
+def main(argv):
+ pycaffe_dir = os.path.dirname(__file__)
+
+ parser = argparse.ArgumentParser()
+ # Required arguments: input file.
+ parser.add_argument(
+ "input_file",
+ help="Path to the input image file"
+ )
+
+ # Optional arguments.
+ parser.add_argument(
+ "--model_def",
+ help="Model definition file."
+ )
+ parser.add_argument(
+ "--pretrained_model",
+ help="Trained model weights file."
+ )
+
+ args = parser.parse_args()
+ image_data = open(args.input_file).read()
+
+ # Pre-load caffe model.
+ nsfw_net = caffe.Net(args.model_def, # pylint: disable=invalid-name
+ args.pretrained_model, caffe.TEST)
+
+ # Load transformer
+ # Note that the parameters are hard-coded for best results
+ caffe_transformer = caffe.io.Transformer({'data': nsfw_net.blobs['data'].data.shape})
+ caffe_transformer.set_transpose('data', (2, 0, 1)) # move image channels to outermost
+ caffe_transformer.set_mean('data', np.array([104, 117, 123])) # subtract the dataset-mean value in each channel
+ caffe_transformer.set_raw_scale('data', 255) # rescale from [0, 1] to [0, 255]
+ caffe_transformer.set_channel_swap('data', (2, 1, 0)) # swap channels from RGB to BGR
+
+ # Classify.
+ scores = caffe_preprocess_and_compute(image_data, caffe_transformer=caffe_transformer, caffe_net=nsfw_net, output_layers=['prob'])
+
+ # Scores is the array containing SFW / NSFW image probabilities
+ # scores[1] indicates the NSFW probability
+ print "NSFW score: " , scores[1]
+
+
+
+if __name__ == '__main__':
+ main(sys.argv)
Oops, something went wrong.

0 comments on commit d43a104

Please sign in to comment.