Hanging Points Generator

Setup

pip install -e .
pip3 install -r py3_requirement.txt

not recommended
The code is for open3d 0.11 and above, so you need to modify it for 0.9
To use voxelization on ubuntu 16.04,
git clone -b hotfix/voxels https://github.com/kosuke55/Open3D.git
then refer to this page and compile Open3d from source.(related PR)

recommended
If you use ubuntu 18.04 and python3, just pip install open3d

Mesh Reconstruction demo

python donwload_sample_data.py --rgbd --urdf
cd examples/
python icp_camera_pose_estimation.py
python create_mesh_tsdf.py
python create_mesh_voxelize_marching_cubes.py

texture mapping

Use this for texture mapping

Mesh Reconstruction demo(ROS)

python donwload_sample_data.py --rosbag

inhand

roslaunch hanging_points_generator sample_create_mesh.launch

handeye

roslaunch hanging_points_generator sample_create_mesh_handeye.launch

Reconstruct mesh exmple

1. Collect rgbd images.

2. Create mesh. (Left: ICP->TSDF Right: ICP->Voxelization-> Marching cubes)

Generate fucntion points sample

Download sample data.

python donwload_sample_data.py --urdf
cd hanging_points_generator

hanging

python generate_hanging_points.py

Can be executed in parallel using eos run-many.

run-many 'generate_hanging_points.py' -n 10 -j 10

pouring

python generate_hanging_points.py

Can be executed in parallel using eos run-many.

run-many 'generate_pouring_points.py' -n 10 -j 10

How to check contact points

usage: check-hanging-pose [-h] [--input INPUT]
                          [--input-file-name INPUT_FILE_NAME] [--idx IDX]
                          [--pose POSE] [--pose-file-name POSE_FILE_NAME]
                          [--clustering CLUSTERING] [--eps EPS]
                          [--filter-penetration FILTER_PENETRATION]
                          [--inf-penetration-check] [--align] [--average]
                          [--average-pos] [--skip-list-file SKIP_LIST_FILE]
                          [--large-axis] [--just-check-num-points]

optional arguments:
  -h, --help            show this help message and exit
  --input INPUT, -i INPUT
                        input urdf (default: )
  --input-file-name INPUT_FILE_NAME, -ifn INPUT_FILE_NAME
                        input object file name. base.urdf or textured.urdf
                        (default: base.urdf)
  --idx IDX             data idx (default: 0)
  --pose POSE, -p POSE  input pose (default: )
  --pose-file-name POSE_FILE_NAME, -pfn POSE_FILE_NAME
                        input pose file name. This is needed only when input
                        is directoryex)contact_points,
                        filtered_contact_points.json (default: contact_points)
  --clustering CLUSTERING, -c CLUSTERING
                        dbscan clustering min points (default: 0)
  --eps EPS, -e EPS     dbscan eps params (default: 0.01)
  --filter-penetration, -f
                        filter penetration (default: False)
  --inf-penetration-check, -ipc
                        infinity penetration check (default: False)
  --align               align coords (default: False)
  --average             average coords rot (default: False)
  --average-pos         average coords pos (default: False)
  --skip-list-file SKIP_LIST_FILE, -slf SKIP_LIST_FILE
                        slip obect list file (default: skip-list-file)
  --large-axis, -la     use large axis as visulaizing marker (default: False)
  --just-check-num-points, -jcnp
                        just check nuber of points without visualiziong
                        (default: False)
  --keyword KEYWORD, -k KEYWORD
                        skip files that do not inculude this keyword. this
                        option works when input is a directory. (default:
                        None)

example

# If contact_points is dir, load multiple contact_points.
cd urdf/037_scissors
check-hanging-pose -i textured.urdf -p contact_points -la
check-hanging-pose -i textured.urdf -p contact_points -c -1 --ipc --align --average -la

Left:Before filtering   Right:After filtering

Gererate random shape objects with GAN

Read random_mesh/README.md and train GAN.
Here you can download models trained with the hanging and pouring categories of ShapeNet.

python donwload_sample_data.py ----gan-trained-model

Genrate random shape mesh with

python random_mesh_generator.py -p <pretrained_model> -s <save_dir>

Train CNN to infer function points

Use hanging_poins_cnn to train CNNs to infer function points.

Externals

andyzeng/tsdf-fusion-python

Citation

@inproceedings{takeuchi_icra_2021,
 author = {Takeuchi, Kosuke and Yanokura, Iori and Kakiuchi, Yohei and Okada, Kei and Inaba, Masayuki},
 booktitle = {ICRA},
 month = {May},
 title = {Automatic Hanging Point Learning from Random Shape Generation and Physical Function Validation},
 year = {2021},
}

@inproceedings{takeuchi_iros_2021,
 author = {Takeuchi, Kosuke and Yanokura, Iori and Kakiuchi, Yohei and Okada, Kei and Inaba, Masayuki},
 booktitle = {IROS},
 month = {September},
 title = {Automatic Learning System for Object Function Points from Random Shape Generation and Physical Validation},
 year = {2021},
}