[Prototype] - Rearrange Scene Evaluation and Collision Proxy Optimizer (CPO)

[aclegg3]

Motivation and Context

This branch contains a set of utilities designed for evaluation and optimization of synthetic scenes for object rearrangement.

Primary components (high level):

• collision_shape_automation.py: CollisionProxyOptimizer(CPO) class and supporting utilities for evaluating the quality of collision proxy shapes and receptacles for individual objects via a set of crafted scalar metrics. Also includes optimization of new collision objects via these metrics with COACD.
• examples/viewer.py : A modified scene viewer with additional UI for evaluating scene quality. Includes:
  • Visualization and classification of Receptacles via CPO metrics.
  • Visualization of collision proxies.
  • Ability to manually filter receptacles (with mouse UI) and export metadata containing the final filtered receptacle set for the scene.
• examples/obj_viewer.py : A modified viewer for investigating a single object and CPO metric ingredients allowing the specification of a custom collision proxy via CLI.
• examples/spot_viewer.py: A modified viewer for evaluating and correcting robot navigability in the scenes. Controls a spot robot to navigate the scene and provides a user interface for selecting, moving, and removing objects.

Dependencies:

• habitat-lab main or including PR 1249
• coacd and trimesh

pip install coacd trimesh

• spot_viewer.py requires spot robot:

python -m habitat_sim.utils.datasets_download --uids hab_spot_arm

Details examples/viewer.py:

Primary purpose is to evaluate a full scene for use with rearrangement and manually filter Receptacles.
CLI options:

• --init-cpo: runs the CPO metrics for all Receptacles in the scene. Necessary to visualize receptacles with metric coloring.
• --rec-filter-file provide an alternative path to receptacle filter metadata file IO.

UI:

• See the help text output when running the tool for details.

Example

python examples/viewer.py --dataset path/to/fphab/fphab/fphab.scene_dataset_config.json --scene 102816036 --init-cpo

Details examples/spot_viewer.py:

Primary purpose is to evaluate Spot robot's ability to navigate the scene and then to interactively modify the scene to correct for any issues.

Details examples/obj_viewer.py:

Primary purpose is to evaluate particular objects and get a closer look at their performance as receptacles.
CLI options:

• --target-object: (replaces --scene) - provide the name of the object to visualize (e.g. f9106be97a1c9ea06b8ac1ddf0e3aa60b1fe2a94)
• --col-obj: provide and alternative collision proxy object file.

UI:

• TODO document this

Example:

python examples/obj_viewer.py --dataset path/to/fphab/fphab/fphab.scene_dataset_config.json --target-object f9106be97a1c9ea06b8ac1ddf0e3aa60b1fe2a94

Details tools/collision_shape_automation.py:

Primary purpose is to provide a set of tools and CLI for optimizating collision shapes such they better match ground truth object shape and Receptacle performance.

CLI options:
Run with --help for details.
Provide either a list of scenes or objects (shortened names separated by spaces):

• --scenes: one or more scenes to optimize.
• --objects: one or more objects to optimize.
  Additional parameters:
• --output-dir: output directory for saved csv and images. Default = ./collision_shape_automation/.
• --debug-images: turns on debug image output.

Example:

python tools/collision_shape_automation.py --dataset path/to/fphab/fphab/fphab.scene_dataset_config.json --objects f9106be97a1c9ea06b8ac1ddf0e3aa60b1fe2a94

Note that COACD files will export to ./COACD_output/ and the best proxy shapes will be copied into your provided dataset directory in place of the current proxy shape.
For example, after running this tool on fphab/ scene dataset, run git status in the dataset directory to see modified collision shapes.

How Has This Been Tested

Locally

Example output of optimization:

python tools/collision_shape_automation.py --dataset path/to/fphab/fphab/fphab.scene_dataset_config.json --objects f9106be97a1c9ea06b8ac1ddf0e3aa60b1fe2a94

Types of changes

Prototype, no testing or intent to merge as-is.