Skip to content
SCAN: Learning Abstract Hierarchical Compositional Visual Concepts
Branch: master
Clone or download
Pull request Compare This branch is even with miyosuda:master.
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
doc
.gitignore
LICENSE
README.md
board.sh
data.tar.gz
data_manager.py
data_manager_test.py
main.py
model.py
model_test.py
options.py
utils.py
utils_test.py

README.md

SCAN

About

Replicating SCAN algorithm described in Google DeepMind's paper "SCAN: Learning Abstract Hierarchical Compositional Visual Concepts"

Image datasets are created with Rodent envrironment.

Requirements

  • Tensorflow 1.2 or later
  • Python2 or 3

How to train

First extract dataset, and then run main.py

$ tar xvf data.tar.gz
$ python main.py

Result

Sym2img

Symbol to image conversion result.

  1. Generated images when wall_color=white is specified.

  1. Generated images when wall_color=white, floor_color=white are specified.

  1. Generated images when wall_color=white, floor_color=white, obj_color=white are specified.

  1. Generated images when wall_color=white, floor_color=white, obj_color=white, obj_id=ice_lolly are specified.

Img2sym

Input Output
obj_color=white, wall_color=white, floor_color=white, obj_id=ice_lolly

(All of the outputs are correct.)

Input Output
obj_color=purple, wall_color=dark_yellow, obj_id=hat

(Correct obj_color was red, but confused as purple. floor_color was not specifiled in the output.)

Concept recombination

Concept reombination result as image outputs.

  1. Recombination result of (obj_color=white) AND (wall_color=white). (Correct result should have white object and white wall.)

  1. Recombination result of (obj_color=white, obj_id=ice_lolly) IN_COMMON (obj_color=white, wall_color=white. (Correct result should have white object.)

  1. Recombination result of (obj_color=white, wall_color=white) IGNORE (obj_color=white). (Correct result should have white wall.)

βVAE disentanglement

Disentanglement result for latent variables for object parameters. (Wall color, Object color, Floor Color, Object Type, Object position).

Wall color

Obj color

Floor color

Obj Type

Obj pos (and Obj Type)

You can’t perform that action at this time.