Skip to content
A PyTorch implementation of the paper: Specifying Object Attributes and Relations in Interactive Scene Generation
Python HTML JavaScript CSS Shell
Branch: master
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
images init Aug 11, 2019
scene_generation Fixing and improving the GUI Nov 10, 2019
scripts Update train_accuracy_net script Nov 12, 2019
README.md Fixing and improving the GUI Nov 10, 2019
requirements.txt Bump pillow from 6.1.0 to 6.2.0 Oct 30, 2019
train.py Refactor the dataset, and few fixes Oct 28, 2019

README.md

Specifying Object Attributes and Relations in Interactive Scene Generation

A PyTorch implementation of the paper Specifying Object Attributes and Relations in Interactive Scene Generation

Paper

Specifying Object Attributes and Relations in Interactive Scene Generation
Oron Ashual1, Lior Wolf1,2
1 Tel-Aviv University, 2 Facebook AI Research
The IEEE International Conference on Computer Vision (ICCV), 2019, (Oral)

Network Architechture

Youtube

paper_video

Usage

1. Create virtual environment (optional)

All code was developed and tested on Ubuntu 18.04 with Python 3.6 (Anaconda) and PyTorch 1.0.

conda create -n scene_generation python=3.7
conda activate scene_generation

2. Clone the repository

cd ~
git clone git@github.com:ashual/scene_generation.git
cd scene_generation

3. Install dependencies

conda install --file requirements.txt -c conda-forge -c pytorch
  • install pytorch which will fit your CUDA TOOLKIT

4. Install COCO API

Note: we didn't trained our models with COCO panoptic dataset, the coco_panoptic.py code is for the sake of the community only.

cd ~
git clone https://github.com/cocodataset/cocoapi.git
cd cocoapi/PythonAPI/
python setup.py install
cd ~/scene_generation

5. Train

$ python train.py

6. Encode the Appearance attributes

python scripts/encode_features --checkpoint TRAINED_MODEL_CHECKPOINT

7. Sample Images

python scripts/sample_images.py --checkpoint TRAINED_MODEL_CHECKPOINT --sample_features 1 --batch_size 32 --output_dir OUTPUT_DIR 

8. or Download trained models

Download these files into models/

7. Play with the GUI

The GUI was built as POC. Use it at your own risk:

python scripts/gui/simple-server.py --checkpoint YOUR_MODEL_CHECKPOINT --output_dir [DIR_NAME] --draw_scene_graphs 1

Citation

If you find this code useful in your research then please cite

@InProceedings{Ashual_2019_ICCV,
    author = {Ashual, Oron and Wolf, Lior},
    title = {Specifying Object Attributes and Relations in Interactive Scene Generation},
    booktitle = {The IEEE International Conference on Computer Vision (ICCV)},
    month = {October},
    year = {2019}
}

Acknowledgement

Our project borrows some source files from sg2im. We thank the authors.

You can’t perform that action at this time.