World Models applied to the Open AI Sonic Retro Contest
Switch branches/tags
Nothing to show
Clone or download
dylan djian
Latest commit 8e84651 Jun 30, 2018
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
lib [ALL] Fixed training + long term sampling for display Jun 7, 2018
models
.gitignore [PLAY_BEST] Update on random results scripts Jun 30, 2018
README.md [PLAY_BEST] Update on random results scripts Jun 30, 2018
const.py
play.py [ALL] Added latent interpolation, heavy code cleaning, fix some issues Jun 7, 2018
play_best.py [PLAY_BEST] Update on random results scripts Jun 30, 2018
requirements.txt [CODE] Cleaning code Jun 6, 2018
test_cuda.py [ALL] First public push Jun 6, 2018
test_sonic.py [ALL] First public push Jun 6, 2018
train_controller.py
train_lstm.py [ALL] Fixed training + long term sampling for display Jun 7, 2018
train_vae.py [ALL] Fixed training + long term sampling for display Jun 7, 2018

README.md

retro-contest-sonic

A student implementation of the World Models paper with documentation.

Ongoing project.

TODO

CURRENTLY DOING

DONE

  • β-VAE for the Visual model
  • MDN-LSTM for the Memory model
  • CMA-ES for the Controller model
  • Training pipelines for the 3 models
  • Human recordings to generate data
  • MongoDB to store data
  • LSTM and VAE trained "successfully"
  • Multiprocessing of the evaluation of a set of parameters given by the CMA-ES
  • Submit learnt agents

LONG TERM PLAN ?

  • Cleaner code, more optimized and documented
  • Game agnostic
  • Continue training / testing better architectures
  • Online training instead of using a database

How to launch the scripts

  • Install the modules in the requirements.txt, pytorch 0.4 and mongoDB
  • Buy or find the ROMs of Sonic The Hedgehog and install them with retro-gym.

Once you've done that, you will need to train the 3 components :
python train_vae.py
python train_lstm.py --folder=xxx
python train_controller.py --folder=xxx where xxx is the folder number created in saved_models/

While training the VAE and the LSTM, pictures will be saved in a folder results/

Once you're done, you can use your best trained controller to play a random level using : python play_best --folder=xxx
Dont forget to change the RENDER_TICK in const.py to 1, so you can see what's happening.

Resources

Differences with the official paper

  • No temperature
  • No flipping of the loss sign during training (to encourage exploration)
  • β-VAE instead of VAE