Skip to content
No description, website, or topics provided.
Python Jupyter Notebook Shell
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
data
general added faster_rcnn model; py3 compatible May 30, 2019
tf_plus added faster_rcnn model; py3 compatible May 30, 2019
CoordConv.py formatted Feb 20, 2019
LICENSE license Feb 28, 2019
NOTICE license Feb 26, 2019
README.md update Feb 28, 2019
experiment_logs.sh experiment logs Feb 28, 2019
model_builder_faster_rcnn.py updates May 30, 2019
model_builders.py added faster_rcnn model; py3 compatible May 30, 2019
params.py added faster_rcnn model; py3 compatible May 30, 2019
train.py added faster_rcnn model; py3 compatible May 30, 2019
train_faster_rcnn.py updates May 30, 2019
train_gan.py added faster_rcnn model; py3 compatible May 30, 2019
util.py
utils_faster_rcnn.py added faster_rcnn model; py3 compatible May 30, 2019

README.md

CoordConv

This repository contains source code necessary to reproduce the results presented in the paper An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solution (NeurIPS 2018):

@inproceedings{liu2018coordconv,
  title={An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solution},
  author={Liu, Rosanne and Lehman, Joel and Molino, Piero and Petroski Such, Felipe and Frank, Eric and Sergeev, Alex and Yosinski, Jason},
  booktitle={Advances in Neural Information Processing Systems},
  year={2018}
}

For more on this project, including a 8-min video explanation, see the Uber AI Labs blog post.

CoordConv layer, as a drop-in replacement for convolution

The standalone CoordConv layer, wrapped as a tf.layers object, can be found in CoordConv.py. Models constructed in model_builders.py show usage of it.

Data

To generate Not-so-Clevr dataset, which consists of squares randomly positioned on a canvas, and with uniform and quarant splits:

python ./data/not_so_clevr_generator.py

To generate two-object Sort-of-Clevr images, run a modification of the Sort-of-Clevr source code:

python ./data/sort_of_clevr_generator.py

Supervised Coordinate Tasks

The train.py script executes the training of all supervised coordinate tasks as described in the paper. Use --arch to toggle among different tasks.

The file experiment_logs.sh records the entire series of experiments enumerating different hyperparameters for each task, as exactly used to produce results in the paper. Note that we generate random experiment ids for job tracking in the Uber internal cluster, which can be ignored. We also use resman to keep results organized, which is highly recommended!

Examples to run Supervised Coordinate Classification:

# coordconv version
python train.py --arch coordconv_classification -mb 16 -E 100 -L 0.005 --opt adam --l2 0.001 -mul 1  
# deconv version
python train.py --arch deconv_classification -mb 16 -E 2000 -L 0.01 --opt adam --l2 0.001 -mul 2 -fs 3

Use --data_h5 data/rectangle_4_uniform.h5 and --data_h5 data/rectangle_4_quadrant.h5 to observe the performances on two types of splits.

Examples to run Supervised Rendering:

# coordconv version
python train.py --arch coordconv_rendering -mb 16 -E 100 -L 0.005 --opt adam --l2 0.001 -mul 1
# deconv version
python train.py --arch deconv_rendering -mb 16 -E 2000 -L 0.01 --opt adam --l2 0.001 -mul 2 -fs 3

Use --data_h5 data/rectangle_4_uniform.h5 and --data_h5 data/rectangle_4_quadrant.h5 to observe the performances on two types of splits.

Examples to run Supervised Coordinate Regression:

# coordconv version
python train.py --arch conv_regressor -E 100 --lr 0.01 --opt adam --l2 0.00001
# deconv version
python train.py --arch coordconv_regressor -E 100 --lr 0.01 --opt adam --l2 0.00001

Use --data_h5 data/rectangle_4_uniform.h5 and --data_h5 data/rectangle_4_quadrant.h5 to observe the performances on two types of splits.

Generative Tasks

python train_gan.py --arch simple_gan -mb 16 -E 50 -L 0.0001 --lr2 .0005 --opt adam --z_dim 25 --snapshot-every 1
python train_gan.py --arch clevr_gan -mb 16 -E 50 -L 0.0001 --lr2 .0005 --opt adam --z_dim 25 --snapshot-every 1

TODO

Add RL, and VAE and LSUN GAN models

You can’t perform that action at this time.