w251 Spring 2023 Final Project

6DoF Multi-Object Pose Estimation

Team Casey Hsiung, Daniele Grandi, Evan Fjeld, Mon Young, Preethi Raju

Links

• Example demo video
• White paper
• Presentation slides

Purpose

This repository contains information and code used for our CASAPose 6D pose estimation project.

Edge Pipeline

• EC2 edge implmentation.

• Theoretical application of our CASAPose model implementation packaged and deployed on a robot.

Folder Structure

• dataset - Dataset creation step-by-step. More detail in this folder's README.md file
• casapose - Our training, validation, and eval environment. More detail in this folder's README.md file
• csv_outputs - Our training, validation, and eval output files. It is used to generate plots and tables in the 251_final_project_plots_tables.ipynb file.
• edge_final - Our docker and codes to place on the edge device. More detail in this folder's README.md file
• workings - Our notes, drafts, and test code for our project.

Details are in our paper and presentation file. Noticeable highlights

• We synthetically created our new headphone object
• We created our dataset containing 15 Linemod objects and our headphones object. Our dataset contains 5,000 synthetic images with associated JSON and meshes files.
• Our synthetic dataset outperforms the PBR dataset
• Our synthetic eval test dataset outperforms the LMO eval test dataset (as expected)

Photo comparisons and graphs

• baseline vs training2 evaluated with LM-O dataset. Our training2 detects glue, has better accuracy on cat, but mis-detect ape.

CASAPose Citation

Gard, Niklas, Anna Hilsmann, and Peter Eisert. "CASAPose: Class-Adaptive and Semantic-Aware Multi-Object Pose Estimation." arXiv preprint arXiv:2210.05318 (2022).