TWO-GENERATOR-GAN

This project is completed in CVPR lab, Department of Electrical Engineering and Technology, IIT Ropar under the guidence of Dr. Subrahmanyam Murala. Aim of this project is to segment objects within videos with background estimation.

Contributors

1. Omkar Thawakar
2. Prashant W Patil , PhD Scholar, IIT Ropar.

Prerequisite

1. Tensorflow
2. Pillow
3. Matplotlib

Datasets

1. DAVIS dataset
2. LASIESTA dataset
3. SegTreck dataset

Model Flow

Training

python RBFS_model.py --mode train --input_dir PATH_CONTAINING_TRAINING_IMAGES --output_dir PATH_TO_SAVE_TRAINED_MODEL

Testing

python RBFS_model.py --mode test --input_dir PATH_CONTAINING_TESTING_IMAGES --output_dir PATH_TO_SAVE_RESULTS --checkpoints TRAINED_MODEL_PATH

Dataset

Images in our training sets are generated in a same way as described in origibal pix2pix format. Following is the demonstration of our image format along with sample image in our dataset.

You can create training dataset by running following script

python create_dataset.py --input INPUT_IMG_DIR --background BACKGROUND_IMG_DIR --target TARGET_IMG_DIR --out_dir OUTRPUT_DIR

Our Results

Following are the results obtained with our model. Format of our results is given below.

input image >> groundtruth background >> pred background >> pred seg of moving object >> target moving object seg

==============================================================================

Quantitative Comparison

For Quantitative comparison following files provide videowise results for respective datasets.

Quantitative_Analysis_Without_Erossion.py     ######## without applying dilation and erossion on output
Quantative_Analysis_with_erossion.py          ######## with applying dilation and erossion on output

Structure of folder containing result is as follows

  ├── DAVIS                     # Dataset Name
    ├── video1                  # video name
    │   ├── *input.png          # input image
    │   ├── *background.png     # background image
    │   └── *output_1.png       # generator 1 output
    |   └── *output_2.png       # generator 2 output
    |   └── *target.png         # groundtruth image
    └── video2
    | .......
    .......
    .......

For Quantitative comparison run following codes

Quantitative_Analysis_Without_Erossion.py --dataset_name dataset_name
Quantative_Analysis_with_erossion.py --dataset_name dataset_name

Results

	SSIM	MAE	F_Measure	Precision	Recall
video1	0.938925	0.013869	0.741054	0.938016	0.616825
video2	0.915701	0.023757	0.819515	0.884632	0.773631
video3	0.900783	0.018898	0.730071	0.877356	0.626688
video4	0.873954	0.048541	0.683289	0.851387	0.577624
video5	0.921787	0.015483	0.545057	0.506826	0.593607