image-to-3d-pipeline

Turn a handful of images into a 3D mesh, then find out which model did it best.

Single-image-to-3D reconstruction has a dozen open-source models and no obvious winner; the honest way to choose is to run several on the same input and look hard at what comes out. This pipeline does exactly that. It takes AI-generated renders of a fictional submersible, reconstructs candidate meshes with several models, scores them against a fixed Blender inspection protocol, and serves the winner in a browser-based WebGL explorer.

For the curious, a live photogrammetry case study from this repo: 3d.thedreamers.us/about, and the explorer itself at 3d.thedreamers.us for live interaction.

Eight viewpoints from one concept. Multiview reasoning happens before any geometry: port profile, three-quarter, bow-on, stern-on. Every reconstruction model in this repo starts from this sheet, so their outputs are comparable.

Six moves, one browser object. The winning mesh in the WebGL viewer, and the pipeline it came through: mask, infer, bake, render, reject, export. The note under the strip is the honest limit of the whole method: a depth map estimates distance for visible pixels and cannot reveal what the camera never saw.

The build story, live. The public page walks from source imagery to a browser-delivered 3D world, step by step, in the open.

Built by Dreamers Inc.

All source images in this repository are AI-generated synthetic renders of a fictional vessel. No real vehicle, vessel, person, or client is depicted.

• tools/: preprocessing, inference, rendering, and evaluation scripts.
• 05-web-explorer/: local Three.js/WebGL explorer for the reconstructed meshes.
• 06-evaluation/: the written evaluation and a variant comparison screenshot.
• 07-experiments/: controlled follow-up experiments (Hunyuan3D 2MV, TRELLIS.2, MapAnything) with their protocol documents.
• examples/: three sample source renders and one reconstructed GLB (submersible-v2-stochastic.glb, about 1.5 MB), so the pipeline output can be inspected without running anything.
• setup.sh: clones the third-party reconstruction tools at pinned commits into tools/vendor/.

Large artifacts are excluded: Python environments, model checkpoints, intermediate PLY/GLB outputs, and the explorer's full runtime asset set. 05-web-explorer/README.md describes how to supply runtime assets, and examples/ provides a mesh you can use for that.

1. 06-evaluation/EVALUATION.md for the ranking and its limitations.
2. 06-evaluation/explorer-variant-comparison.jpg for the same browser scene with three candidate meshes.
3. 05-web-explorer/README.md for local run instructions.
4. examples/ for sample inputs and a reconstructed output.

Run ./setup.sh to clone the four reconstruction models at the exact commits used in this evaluation. They are cloned rather than vendored because each carries its own license, which you accept when you fetch it:

Review each license before use, particularly the Stability AI and Tencent terms, which restrict commercial use and territory respectively.

1. Preprocess: tools/preprocess_object_masks.py produces transparent object-only masks from the source renders.
2. Reconstruct: tools/run_trellis_multiview.py runs multi-image TRELLIS; 07-experiments/ holds runners for the other candidates.
3. Evaluate: tools/run_reconstruction_eval.sh renders each GLB through the same fixed Blender orbit and builds contact sheets for the 0-2 visual rubric in 07-experiments/HUNYUAN_2MV_EXPERIMENT_PROTOCOL.md.
4. Explore: 05-web-explorer/ loads the chosen GLB in a navigable underwater scene with BVH collision.

The code in this repository is licensed under the Apache License 2.0 (see LICENSE). The third-party tools fetched by setup.sh are excluded and remain under their own licenses.