TensorFlow impementation of: Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
LICENSE
README.md
__init__.py
dataset.py
e2c_plane.py
e2c_seq.py
env0.png
env1.png
env_blank.png
plane_data2.py
tests.py
vae.py
viz_results.py

README.md

e2c

TensorFlow impementation of: Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images, with code optimized for clarity and simplicity.

latent

Only 160 lines of code, and only uses Python modules that come installed with TensorFlow. Proper writeup explaining the paper plus improved model code to soon follow.

Results

Left column are x_t, x_{t+1}, and right column are the E2C reconstructions. reconstruction

Larger step sizes (magnitude of u) yield better latent space reconstruction...

unfolding latent space

but degrade image reconstruction fidelity (more on this later...). Here's a different set of obstacles:

poor reconstruction

Features:

  • Implements the standard E2C model with the factorized Gaussian KL divergence term (Eq. 14)
  • Adam Optimizer + Orthogonal weight initialization scheme by Saxe et al..
  • Learns the latent space of the planar control task (uses the same parameters described in the paper, Appendix B.6.2)

Training the Model

First, generate the synthetic training data plane2.npz by running the following script.

$ python plane_data2.py

Then, train the model

$ python e2c.py

You can then generate visualizations by executing:

$ python viz_results.py

Acknowledgements

Thanks to Manuel Watter for answering my questions about the paper.

License

Apache 2.0