V23D

Human reconstruction pipeline with:

• frame/mask preprocessing and torso-centered retargeting
• COLMAP SfM camera estimation
• SMPL/SMPLify-X initialization
• SMPL texture projection from per-part splats
• 3D Gaussian Splatting (3DGS) follow-up training

Repository Layout

• V23D/src/preprocess: frame/mask retargeting and normalization
• V23D/src/sfm: COLMAP runner and pose export
• V23D/src/recon: SMPL and 3DGS preparation/training glue
• V23D/workflows/smpl_fitting: SMPLify-X wrappers and fitting scripts
• V23D/workflows/texture_projection: per-part splat to SMPL mesh coloring
• V23D/workflows/pipeline_orchestration: end-to-end orchestration scripts

End-to-End Pipeline

1. Retarget (torso-centered normalization)

   • Reads RGB frames + masks.
   • Estimates subject bbox/anchor per frame.
   • Applies smoothed affine warp to keep the body centered and scale-stable.

2. COLMAP SfM

   • Feature extraction and matching (sequential or exhaustive).
   • Mapping + bundle adjustment.
   • Multi-attempt mode (strict/relaxed) with minimum registered-pose enforcement.
   • Outputs sparse model, camera poses, and run metadata.

3. SMPL Initialization (SMPLify-X sampled frames)

   • Runs 2D keypoint detection on sampled frames.
   • Converts to Body25 format.
   • Infers focal length from COLMAP cameras.
   • Runs SMPLify-X in SMPL mode to produce initial body parameters.

4. Texture Projection (per-part splat)

   • Projects front/back splat images into SMPL-view space.
   • Assigns nearest valid per-part colors to mesh vertices.
   • Applies configured fallback/gap-fill rules.
   • Exports colored SMPL mesh (PLY/OBJ workflow dependent).

5. 3DGS Follow-up

   • Undistorts images with COLMAP model.
   • Optionally applies strict foreground masks to undistorted images.
   • Trains Graphdeco 3DGS on prepared scene.
   • Saves checkpoint point clouds by iteration.

Main Scripts

• Full orchestration:
  • V23D/workflows/pipeline_orchestration/run_smpl_init_3dgs_pipeline.py
• 3DGS-only follow-up from existing SfM outputs:
  • V23D/workflows/pipeline_orchestration/run_3dgs_followup.py
• SMPLify-X sampled fitting:
  • V23D/workflows/smpl_fitting/run_smplifyx_torchvision_sampled_frames.py
• Per-part splat to SMPL texture:
  • V23D/workflows/texture_projection/per_part_splat_from_precomputed.py

Typical Run (Example)

$env:PATH='E:\envs\v23d_local\Library\bin;'+$env:PATH
Set-Location 'C:\V23D\V23D'

$root='E:\V23D_Data\approaches\smpl_init_3dgs_output_pack1'

& 'E:\envs\v23d_local\python.exe' workflows\pipeline_orchestration\run_3dgs_followup.py \
	--frames "$root\centered\frames" \
	--masks "$root\centered\masks" \
	--colmap "$root\colmap" \
	--gs-repo 'E:\V23D_Data\gaussian-splatting' \
	--scene-dir "$root\gs_scene" \
	--model-dir "$root\gs_model" \
	--iterations 7000 \
	--python-exe 'E:\envs\v23d_local\python.exe' \
	--log-level INFO

Outputs

Common output artifacts:

• pipeline_summary.json: top-level stage summary
• retarget_metadata.json: per-frame affine normalization details
• colmap_run.json: SfM registration and selected sparse model metadata
• scene_prep_info.json: 3DGS scene prep details
• train_3dgs_info.json: 3DGS training command and status
• point_cloud/iteration_*/point_cloud.ply: 3DGS checkpoints

Notes

• Ensure colmap is available in PATH.
• The repo uses local external dependencies under third_party and an external Graphdeco 3DGS checkout.
• For reproducibility, prefer fixed frame directories and archived output roots per experiment.