Fuzzy Metaballs+: Differentiable Renderering with 3D Gaussians, Flow and More.

Project Page

This is an expanded version of the original FM renderer with support for flow, mesh exporting, 2-parameter and zero-parameter renderers.

It primarily is useful for reconstructing CO3D Sequences. The generation operation is

Git clone unimatch for generating optical flow and XMem for propogating the first mask into the same root directory.
Fetch some CO3D sequences. For example using get_co3d.sh for the single sequence teddy bear.
Run generate_inputs.ipynb to generate flows and masks for the reconstruction
Run run_co3d_sp.ipynb or run_co3d_sp-zpfm.ipynb to run the reconstructions with either the two parameter or zero parameter models
Compile PoissonRecon and run it to generate a mesh via PoissonRecon --in tmp_out/teddybear_34_1479_4753.ply --out teddy.ply --bType 2 --depth 6

Name		Name	Last commit message	Last commit date
Latest commit History 2 Commits
data		data
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
fm_render.py		fm_render.py
generate_inputs.ipynb		generate_inputs.ipynb
get_co3d.sh		get_co3d.sh
images_to_video.sh		images_to_video.sh
pose_estimation.ipynb		pose_estimation.ipynb
requirements.txt		requirements.txt
run_co3d_sp-zpfm.ipynb		run_co3d_sp-zpfm.ipynb
run_co3d_sp.ipynb		run_co3d_sp.ipynb
util.py		util.py
util_load.py		util_load.py
util_render.py		util_render.py
utils_opt.py		utils_opt.py
zpfm_render.py		zpfm_render.py