This GitHub repository contains code and data for our paper A Bayesian-Symbolic Approach to Reasoning and Learning in Intuitive Physics at NeurIPS 2021.
If you use the code in your research, please cite us using the following BibTeX entry:
@inproceedings{xu2021bsp,
title={A {B}ayesian-Symbolic Approach to Reasoning and Learning for Intuitive Physics},
author={Kai Xu and Srivastava, Akash and Gutfreund, Dan and Sosa, Felix and Ullman Tomer and Tenenbaumm, Joshua B. and Sutton, Charles},
booktitle={The 35th Conference on Neural Information Processing Systems (NeurIPS)},
year={2021},
}
/
data/
: generated data and processed dataphys101
: selected scenes from Physics 101 (Wu et al., 2016); originally from http://phys101.csail.mit.edu/synth
: 3 types of synthetic scenesullman
: visual stimulus used in Ullman et al. (2019); originally from https://github.com/tomeru/LPDS/
notebooks/
: interactive notebooks to extract or analyse experimental resultsdemo-world.ipynb
: example notebook to demonstrate how to run simulation withBayesianSymbolic.jl
monly.ipynb
: notebook to extract results for M-step only resultsem.ipynb
: notebook to extract results for complete EM resultsphys101.ipynb
: notebook to extract results for PHYS101 resultsullman.ipynb
: notebook to extract results for ULLMAN results
paper/
: interactive notebooks to produce artifects for the paperraw/
: extracted resultsfigures.ipynb
: notebook to make figurestables.ipynb
: notebook to make tables
scripts/
: executable scripts to generate or pre-process data or to run experimentsevalexpr_efficiency.jl
: script to compare different implementations for expression evaluations- This does not affect the results but only performance.
generate-synth.jl
: script to generate SYNTHhelper.jl
: common helper functions used by scriptsneural.jl
: concrete architecture and hyper-parameters for neural baselinespreprocess-ullman-master.jl
: master script to process a scene from ULLMAN, callingscripts/preprocess-ullman.py
preprocess-ullman.py
: script to process scenes from ULLMANrunexp-ullman.jl
: script to run an experiment on ULLMAN; for other datasets, userunexp.jl
runexp.jl
: script to run an experiment on a given dataset and specific hyper-parameters- All datasets but ULLMAN are supported; for ULLMAN, use
runexp-ullman.jl
- All datasets but ULLMAN are supported; for ULLMAN, use
ullman_hacks.jl
: ULLMAN specific grammar- See the comments on hacks in the beginning of the file
src/
: source codes for simulation and the BSP algorithmBayesianSymbolic.jl/
: world construction and simulationsdata/
: data processing functionsphys101.jl
: PHYS101 specific functionspreprocessing.jl
: generic data preparation functionsullman.jl
: ULLMAN specific functionsullman.txt
: ground truth information for ULLMAN
scenarios/
: scenarios implemented in Turing.jlbounce.jl
: the BOUNCE scenario from SYNTHfall.jl
: the FALL scenario from PHYS101magnet.jl
: the MAGNET scenario- This is not used in the paper
mat.jl
: the MAT scenario from SYNTHnbody.jl
: the NBDOY scenario from SYNTHspring.jl
the SPRING scenario from PHYS101ullman.jl
: the scenario from ULLMAN
analyse.jl
: quantitative and visual analysisapp_inf.jl
: functions for approximate inferencedataset.jl
: functions for loading datasets (SYNTH, PHYS101 and ULLMAN)exp_max.jl
: types and interfaces for the EM algorithmneural.jl
: generic implementations of neural baselinessym_reg.jl
: functions for symbolic regressionutility.jl
utility functions for simulation and loss computation
- =suupl/=: supplementary materials
bounce_inspection
: visualisation for results discussed in appendix C.3.1generalization
: visualisation for results discussed in appendix C.3.2
Manifest.toml
: the exact package version of this environmentmaster.jl
: master scripts to run a batch of experiments, calling scripts inscripts/
Project.toml
: the dependency of this environmentREADME.md
: this file you are reading
BSP is implemented with Julia and some of the dependencies or scripts also rely on Python.
Please follow https://julialang.org/downloads/ to download and install Julia.
Make sure julia
is available in your executable path.
Then from the root of this repo, you can do julia -e "import Pkg; Pkg.instantiate()"
to instantiate the environment.
There are a few more steps to have Julia properly linked with Python, which is explained next.
You will need to have Python installed and a virtual environment set up. The virtual environment should have the following packages
matplotlib
pandas
wandb
To properly link this virtual environment with Julia, please follow https://github.com/JuliaPy/PyCall.jl.
You will also have all necessary Python dependencies to run scripts/preprocess-ullman.py
.
Please see the libraries imported in the script.
Once these steps are done, you are ready to run the scripts.
Run the following experiments
julia master.jl efficiency synth/nbody
julia master.jl efficiency synth/bounce
julia master.jl efficiency synth/mat
Collect the results using notebooks/monly.ipynb
and make the figure using paper/figures.ipynb
Run the following experiments
julia master.jl ablation synth/nbody
julia master.jl ablation synth/bounce
julia master.jl ablation synth/mat
Collect the results using notebooks/monly.ipynb
and make the figure using paper/figures.ipynb
Run the following experiments
julia master.jl em synth/nbody
Collect the results using notebooks/em.ipynb
and make the figure using paper/tables.ipynb
Run the following experiments
julia master.jl phys101 fall
julia master.jl phys101 spring
Collect the results using notebooks/phys101.ipynb
and make the figure using paper/tables.ipynb
Run the following experiments
julia master.jl ullman
Collect the results using notebooks/ullman.ipynb
and make the table using paper/tables.ipynb
& figure using paper/figures.ipynb
Set the environment variable JULIA_NUM_THREADS=10
before running any scripts will enable multiple-threading (e.g. 10 threads in this example) whenever it’s programmed to do so.
For example, master.jl
executes a batch of experiments and it is programmed to run them in a multi-threading manner which can benefit from setting this environment variable.