Skip to content
/ MIME Public

This is the official code for MIME: Human-Aware 3D Scene Generation (CVPR2023)

Notifications You must be signed in to change notification settings

yhw-yhw/MIME

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

MIME: Human-Aware 3D Scene Generation

drawing

This repository contains the code and dataset that accompanies our paper MIME: Human-Aware 3D Scene Generation .

You can find detailed usage instructions for training your own models, using our pretrained models as well as performing the interactive tasks described in the paper below.

If you found this work influential or helpful for your research, please consider citing

@inproceedings{yi2022mime,
title = {{MIME}: Human-Aware {3D} Scene Generation},
author = {Yi, Hongwei and Huang, Chun-Hao P. and Tripathi, Shashank and Hering, Lea and 
Thies, Justus and Black, Michael J.},
  booktitle={IEEE Conference on Computer Vision and Pattern Recognition (CVPR)}, 
month={June}, 
year={2023} 
}

Preliminaries

  • See docs/installation.md to install all the required packages and pretrained models.
  • See docs/dataset.md to download the 3D-FRONT-Human datasets and learn how to add free-space and contact humans inside a 3D room.

Download the CHECKPOINT and DATA for running the code.

Please download several files from the download webpage.

  • Downlad the MIME_CKPT and put it into data/CKPT
  • Download other needed code_data and put them into data.
  • Download 3D-FRONT-HUMAN preprocess data preprocess_3DFRONTHUMAN_input.tar.gz and unzip into data/preprocess_3DFRONTHUMAN_input
  • Download our preprocessed 3DFRONTHUMAN_relative_path_pkl.zip for 3D-FRONT-Human and unzip into data/3DFRONTHUMAN_relative_path_pkl
  • Download our preprocess body input input_finement for running scene refinement, unzip it into data/input_refinement.
  • Download several visualized samples in 3D-FRONT-HUMAN for visualization.

Change the enviornment variables.

You need to modify these variables in env.sh.

# ! need to modify.
export PYTHONPATH=${change_to_python_path}:$PYTHONPATH
export CODE_ROOT_DIR=${change_to_code_path}
export DATA_ROOT_DIR=${change_to_original_3DFRONT_path}

Scene Generation

Scene Refinement

  • See docs/refinement.md to refine the scene layout with gemetrical human-scene interaction details.

Visualize the Dataset and Results

  • See docs/visualization.md to visualize the 3D-FRONT-HUMAN dataset and the distribution of different objects locations in our MIME generated results.

Training

Evaluation

Acknowledgments

We thank Despoina Paschalidou, Wamiq Para for useful feedback about the reimplementation of ATISS, 
and Yuliang Xiu, Weiyang Liu, Yandong Wen, Yao Feng for the insightful discussions, 
and Benjamin Pellkofer for IT support. 
This work was supported by the German Federal Ministry of Education and Research (BMBF): Tübingen AI Center, FKZ: 01IS18039B.  

We build our MIME architecture based on ATISS.

Disclosure

MJB has received research gift funds from Adobe, Intel, Nvidia, Meta/Facebook, and Amazon. 
MJB has financial interests in Amazon, Datagen Technologies, and Meshcapade GmbH. 
JT has received research gift funds from Microsoft Research.

About

This is the official code for MIME: Human-Aware 3D Scene Generation (CVPR2023)

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published