Text to Mesh

The included notebook takes a text prompt and generates a 3D mesh. The steps to do so are as follows:

1. Generate image from text using Stable Diffusion pipeline
2. Generate depth map using DPTForDepthEstimation
3. Convert depth map + image to rgbd image -> point cloud -> mesh using Open3D library

Prompt

interior of a mountain chalet by frank lloyd wright

Image

Mesh

interior_of_a_mountain_chalet_by_frank_lloyd_wright-1690431419.webm