This supplementary material contains the code and video for the IROS'23 paper "Generating Scenarios from High-Level Specifications for Object Rearrangement Tasks".
Abstract
Rearranging objects is an essential skill for robots. To quickly teach robots new rearrangements tasks, we would like to generate training scenarios from high-level specifications that define the relative placement of objects for the task at hand. Ideally, to guide the robot's learning we also want to be able to rank these scenarios according to their difficulty. Prior work has shown how diverse scenario generation from specifications and providing the robot with easy-to-difficult samples can improve the learning. Yet, existing scenario generation methods typically cannot generate diverse scenarios while controlling their difficulty. We address this challenge by conditioning generative models on spatial logic specifications to generate spatially-structured scenarios that meet the specification and desired difficulty level. Our experiments showed that generative models are more effective and data-efficient than rejection sampling and that the spatially-structured scenarios can drastically improve training of downstream tasks by orders of magnitude.
The Supplementary Video shows generated scenarios for each of the three experiments.
The code contains three interactive Jupyter notebooks in which users can choose desired satisfaction values and generate/render scenarios. The code has been tested on macOS 12.3 with Python 3.8 in an Anaconda environment.
Running this code requires Python 3.8. The experiments use Python Poetry for packages and dependency management. Please make sure you have Poetry installed.
To clone the repository and install the dependencies:
git clone https://github.com/sannevw/scenario_generation.git
poetry installThe notebooks are named "IROS23-Experiment-X" where X indicates the experiment number. The notebooks automatically load our backend and the trained models. You should now be able to run the Jupyter Notebooks (opening in your web browser):
poetry run jupyter notebookThis repository contains the following notebooks: