Feature Requests

[bennyguo]

Hello, 3D enthusiasts!

I know that many of you are expecting features/models from threestudio, but since we're currently a very small team, it could be hard to meet everyone's needs. So I set up this discussion to find the most-wanted features/models.

• Feel free to reply with your requests, but remember to check if there're already similar requests to avoid duplications.
• Hit the upvote button for the features/models you want. Every once in a while we'll select some of the most upvoted requests and see if we can get them done.

I'll give an example to start this discussion:

[bennyguo]

Implementation of DreamAvatar for high-quality 3D avatar generation from text prompts.

[andorxornot]

it is interesting that they use a parameterized human model.

[yuedajiong]

The biggest one is: 3+ image(s) guided game avatar generator. interactive ui/drag,etc or text guide edit should be better.
Any plan to implements? such as sparsefusion.

[qtabdullah]

Image to 3D

[andorxornot]

Implementation of Perp-Neg-stablediffusion , this seems to solve the Janus problem

[thuliu-yt16]

Already in our plan :)

[bennyguo]

Supported! Please follow the instructions here to try it out.

[IkuStudies]

Nice work!

[XinYu-Andy]

Implementation of TextMesh , which utilizes pseudo image supervision.

[bennyguo]

Implemented! Please follow the instructions here to give it a try.

[qtabdullah]

A galley

[LiamSmyth]

Just an idea, but it would be cool to have a way to preview training live like the instantNERF viewer or NERFStudio?

Longer term, assuming this stuff gets good enough start spending time art directing, it would be cool to be able to watch it update, and if it starts "growing" in a direction you don't like, you can reset it to the last saved checkpoint and tune the training parameters or seed, or even spawn a bunch of evolutionary branches!

[alievrusik]

Can you please add this paper method (TextMesh). Results look really nice (though could be cherypicked) and authors claim that their model beats DreamFusion.

[bennyguo]

Implemented! Please follow the instructions here to give it a try.

[claforte]

Reproduce ashawkey/stable-dreamfusion#263

(My team at Stability AI might help with that. ;-) )

[bluestyle97]

Implementation of ProlificDreamer

[bennyguo]

Done. Following the instruction to have a try!

[bluestyle97]

Done. Following the instruction to have a try!

Wow, so efficient!

[huthut001]

Can you improve your model support building 3d model shape or 3d part assembly?

[tooe81948]

Implementation of Neuralangelo: High-Fidelity Neural Surface Reconstruction (https://research.nvidia.com/labs/dir/neuralangelo/) 🥺🥺

[bennyguo]

Hi! If you simply want to use tricks from Neuralangelo to train from multi-view images, you can try my instant-nsr-pl codebase. I'm also planning to try these tricks in threestudio and see if we can get any benefit from it:)

[tooe81948]

Wow , I've tried your repository and I feel it's very good
Thanks Implementation of Neuralangelo

[bennyguo]

We used techniques from Neuralangelo in our implementation of TextMesh. Welcome to give it a try following the instructions!

[ericwudocomoi]

HiFA: High-fidelity Text-to-3D with Advanced Diffusion Guidance

[qtabdullah]

Implementation of ATT3D: Amortized Text-To-3D Object Synthesis

[bennyguo]

On it :)

[beibeirory]

I'm waiting for the implementation of ATT3D sincerely. @bennyguo

[qtabdullah]

me too

[qtabdullah]

are u done yet?

[IkuStudies]

Guys u r crushing it. I would love to request a feature into threestudio- and that is to have the ability to start the training from a pre existing obj file to guide the shape a bit.

Like a sphere obj file could begin the training and become an apple

While a cylinder might start and form a banana

Or if you had a type of couch as an obj file and wanted to start from that shape to make it spaceaged or something...

Or to start with a sculpture obj file thats frowning and caricaturize it and make the 3d model wildly smiling with a prompt

Maybe this can be done but i would find it incredibly useful to tweak existing 3d models by prompt.

Btw great work guys been following dreamfusion since its release and amazed by the progress- about to write an article just about the progress

[bennyguo]

Thank you! Actually, our Fantasia3D implementation supports initializing from a custom mesh. We provide some results based on a human mesh here and you may try other custom meshes too. Looking forward to the article!

[yankeesong]

Implementation of Magic123. Most of the techniques are already implemented here so it might now be much more work.

[HL4214]

Implementation of DreamHuman. DreamHuman: Animatable 3D Avatars from Text

[ghost]

Implementation of 3DFuse : Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generation

project page: https://ku-cvlab.github.io/3DFuse/

[mskwak01]

Hello, I am one of the co-authors of 3DFuse, and we have an implementation of 3DFuse with ProlificDreamer baseline (other baselines can be easily adapted as well) with many promising results. Will this be incorporated if we push our code to Threestudio, or are there any further steps to take? Thank you!

[ninjasaid2k]

Requesting https://arxiv.org/pdf/2308.13223.pdf - EfficientDreamer: High-Fidelity and Robust 3D Creation
via Orthogonal-view Diffusion Prior

[TruthSearcher]

Implementation of
MVDream: Multi-view Diffusion for 3D Generation
https://arxiv.org/abs/2308.16512

[yuchenlichuck]

Implementation of
Metrics to evaluate the qualitative of each model

[marinatenhave]

Huggingface or Tencent demo of ProlificDreamer, please!

[ninjasaid2k]

https://github.com/dreamgaussian/dreamgaussian implementation?

[TruthSearcher]

requesting Implementation for https://github.com/gsgen3d/gsgen

[ninjasaid2k]

can something like this be implemented

like controlnet for 3d generations?
https://arxiv.org/abs/2310.05375 - IPDreamer: Appearance-Controllable 3D Object Generation with Image Prompts

[immaturehearts]

are u planning to implement SyncDreamer, please?

[zyms5244]

Implementation of RODIN Diffusion: https://3d-avatar-diffusion.microsoft.com/