Inner Outlier Generator

Inner Outlier Generator (IO-GEN) is designed to better solve One-class Classification (OC) problems by "replacing" the central vector c in Deep Support Vector Data Description (DSVDD) with synthetic data from a GAN as shown below — i.e., (b) vs (d). As a result, classifiers deployed on the space can utilize the multi-dimensional features for anomaly detection in complex data, whereas DSVDD only relies on a simplistic distance metric.

This repository provides the official Tensorflow implementations of IO-GEN, which was first proposed in the paper:

"Identification of Abnormal States in Videos of Ants Undergoing Social Phase Change", Taeyeong Choi, Benjamin Pyenson, Juergen Liebig, and Theodore P. Pavlic, published in the Proceedings of AAAI-21.

Although theoretically, IO-GEN is applicable to any type of OC problems, here we focus on the exemplar scenario discussed in the above paper, where the classifier is trained only with observational samples from stable colony but has to distinguish unstable samples. Instructions below start with a quick introduction to the pipeline of involved networks during training and test, followed by technical manuals to reproduce similar results to the paper. Ant motional data are also available at https://github.com/ctyeong/OpticalFlows_HsAnts.

Contents

1. Model Pipeline
2. Installation
3. Training
4. Test
5. Benchmark Performance
6. Exporting Fake Ant Motions
7. Citation
8. Contact

Model Pipeline

To better understand the code, we first review the pipeline of network model and data flows during training and test.

Training Steps

1. Deep Convolutional Autoencoder (DCAE) is trained.

2. Encoder part of DCAE is fine-tuned as DSVDD.

3. IO-GEN is trained in an adversarial manner involving the feature space of DSVDD, which is frozen this time.

   

4. Classifier is trained on top of frozen (IO-GEN, DSVDD).

   

Structure for Test

(DSVDD, CLS) are the only components used after training to classify input optical flows as either stable or unstable.

Installation

1. Clone the repository

   $ git clone https://github.com/ctyeong/IO-GEN.git
   

2. Install the required Python packages

   $ pip install -r requirements.txt
   

   • Python 3.6 is assumed to be installed already

3. Download ant motional data

   $ cd IO-GEN
   $ git clone https://github.com/ctyeong/OpticalFlows_HsAnts.git
   $ mv split1 split2 split3 Stable Unstable ../ && cd ..
   $ rm -rf OpticalFlows_HsAnts
   

Training

train.py is the core python script to train DCAE, DSVDD, IO-GEN, and Classifier in order. Each model is evaluated per epoch, and it is saved into a specified directory whenever the best performance has been achieved during epochs. Note that the numbers of epochs are set to 750, 160, 20K, and 40, respectively to follow the original protocol of the paper.

With a particular data split

Here is an example for training with 'split1' in the current directory.

$ python train.py -s ./split1

'-s' argument must be provided although all other arguments, explained below, can be ignored just to run with default values.

Specifying the number of optical flow pairs per input

The number of x,y optical flow pairs per input can be specified by '-m' (default=2):

$ python train.py -s ./split1 -m 1

Saving best models and training logs

The directories to store best trained models and Tensorboard logs can also be specified by '-d' and '-t':

$ python train.py -s ./split1 -d ./saved_models -t ./tb_logs

Without specification, the directories are automatically set to './saved_models' and './tb_logs', respectively. The best models are saved with names like 'DCAE.h5', 'DSVDD.h5', 'IO-GEN.h5', and 'Classifier.h5' under the corresponding directory.

Test

test.py is the python script to test the trained models: 'DCAE', 'DSVDD', and 'IO-GEN'.

Testing a particular model

Any argument for training can be used for test as well. In addition, the model to test must be given with '-n' option. For example:

$ python test.py -s ./split1 -d ./saved_models -n IO-GEN -m 4

This command will use 'split1' to run the saved 'IO-GEN' model located under './saved_models' folder, which was trained with 4 optical flow pairs per input.

AUC outputs

Each test execution prints out Area Under the Curve (AUC) scores of the Receiver Operating Characteristics (ROC) by applying the model in different time windows. For instance:

D+1: .933
D+2: .943
D+3 - D+6: .909
D+7 - D+10: .792
D+11 - D+14: .688
D+15 - D+18: .678
All: .786

Similar to the report protocol in the paper, the first six rows indicate AUC scores at unique time bins while the colony is stabilized. In contrast, the last row is the performance measurement considering all samples from the entire observation period (D+1~D+18)

Benchmark Performance

Here are the AUC scores for IO-GEN and other baseline approaches reported in the IAAI-21 paper. Each score is the average over three unique splits with the standard deviation. More details are provided in the paper.

Overall Performance

METHOD	AUC
OFW	.506
DCAE	.506 (.002)
OC-SVM	.523 (.004)
DSVDD	.762 (.013)
GEN	.587 (.032)
N-GEN	.699 (.006)
IO-GEN	.786 (.009)

Temporal Performance

Models were tested with test samples within unique time bins to see how they respond to dynamic ant behaviors over time during colonial stabilization.

METHOD	D+1	D+2	D+3-D+6	D+7-D+10	D+11-D+14	D+15-D+18
OFW	.873	.736	.620	.474	.390	.369
DCAE	.880 (.005)	.760 (.002)	.637 (.002)	.480 (.002)	.390 (.003)	.360 (.003)
DSVDD	.916 (.004)	.886 (.008)	.861 (.011)	.759 (.016)	.677 (.014)	.680 (.016)
GEN	.713 (.036)	.677 (.065)	.639 (.044)	.578 (.055)	.542 (.037)	.535 (.041)
N-GEN	.835 (.091)	.847 (.062)	.819 (.030)	.739 (.023)	.596 (.020)	.588 (.031)
IO-GEN	.933 (.027)	.943 (.014)	.909 (.014)	.792 (.013)	.688 (.017)	.678 (.022)

Exporting Fake Ant Motions

'synthesize.py' is the python script to export synthetic optical flows of ants from trained 'IO-GEN'. If you have saved the model at './save_models/IO-GEN.h5' after training with 2 optical flows pairs per input of 'split1', the following command generates 5 fake optical flow pairs under './fake_imgs':

$ python synthesize.py -s split1 -m 2 -p saved_models/IO-GEN.h5 -f ./fake_imgs -b 5 -c Spectral -i 1

'-c' option allows to specify a particular colormap (among available choices), and '-i' inverses the order of pixel values in terms of magnitude, when set to 1, for better visualization with some colormaps.

Once the above has been executed, './fake_imgs' contains:

0.jpg, 1.jpg, 2.jpg, 3.jpg, 4.jpg

Note that each image file visualizes the horizontal and the vertical optical flows side-by-side.

Examples with different colormaps

Synthetic outputs from four different colormaps are displayed below. For each colormap, a pair of optical flows are exported side-by-side in an image file.

Spectral	gist_heat

coolwarm	jet

Citation

If you use this code or reference IO-GEN for your research, please cite our paper:

@inproceedings{CPLP21,
  title={Identification of Abnormal States in Videos of Ants Undergoing Social Phase Change},
  author={Choi, Taeyeong and Pyenson, Benjamin and Liebig, Juergen and Pavlic, Theodore P},
  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
  year={2021}
}

Contact

If there is any question or suggestion, please do not hesitate to shoot an email to tchoi@lincoln.ac.uk, or DM to @ssuty on Twitter! Thanks!