Skip to content

scottemmons/sgm

master
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
sgm
 
 
 
 
 
 
 
 

Sparse Graphical Memory (SGM)

Sparse Graphical Memory (SGM) is a data structure for reinforcement-learning agents to solve long-horizon, sparse-reward navigation tasks. This codebase is a TensorFlow implementation of SGM accompanying the paper Sparse Graphical Memory for Robust Planning.

To install:

  1. Create a new conda environment: conda create -n sgm python=3.6
  2. Activate the conda environment: conda activate sgm
  3. Install the requirements: pip install -r requirements.txt
  4. Install the package: pip install -e .

To launch the experiments:

  1. To launch SGM experiments, run all the cells in notebooks/SGM in Thinned FourRooms.ipynb
  2. To launch SoRB experiments, run all the cells in notebooks/SoRB in Thinned FourRooms.ipynb

To visualize the results:

  1. Call sgm/plot.py with the directories of the logs produced by the experiments followed by the --returns_v_cleanup_steps flag, e.g., python sgm/plot.py logs/thinned_fourrooms_sgm logs/thinned_fourrooms_sorb --returns_v_cleanup_steps
  2. Find the visualized results in the plots directory, e.g., in plots/returns_v_cleanup_steps

Commands you may find useful

Enable rendering of environments when there is no display, e.g., on a server: xvfb-run -s "-screen 0 1400x900x24" jupyter notebook https://stackoverflow.com/questions/40195740/how-to-run-openai-gym-render-over-a-server

Create tunnel from local machine to remote machine: ssh -N -f -L localhost:8888:localhost:8888 user@remote.hostname.edu

Run Jupyter notebook with no time limit on cell execution and replace the notebook's contents with the new output: jupyter nbconvert --ExecutePreprocessor.timeout=-1 --execute --to notebook --inplace <notebook.ipynb> https://stackoverflow.com/questions/35545402/how-to-run-an-ipynb-jupyter-notebook-from-terminal

Credits

This code was built upon the code released by Eysenbach et al. under the Apache License, Version 2.0.

Reference

@inproceedings{emmons2020sparse,
  Author = {Emmons, Scott and Jain, Ajay and
  Laskin, Michael and Kurutach, Thanard
  and Abbeel, Pieter and Pathak, Deepak},
  Title = {Sparse Graphical Memory
  for Robust Planning},
  Booktitle = {Neural Information Processing
  Systems},
  Year = {2020}
}

About

Sparse Graphical Memory for Robust Planning

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published