Query Regarding the Integration of Custom 3D Models with LucidDreamer's Gaussian Splatting Technique

[yihong1120]

Dear LucidDreamer Contributors,

I hope this message finds you well. I am reaching out to inquire about the possibility of integrating custom 3D models within the LucidDreamer framework, specifically in relation to the Gaussian splatting scenes generation process.

Having perused your impressive repository and the accompanying paper, I am intrigued by the potential applications of your domain-free generation technique. My interest lies in exploring how LucidDreamer might accommodate bespoke 3D models, which could be used in conjunction with the Gaussian splatting method to create unique and personalised scenes.

Would you be so kind as to provide guidance on whether the current implementation supports such an integration? If so, I would be most grateful for any documentation or examples that could assist me in understanding the process. Additionally, should this functionality not be present, might you consider it a worthwhile addition for future updates?

I am keen to experiment with LucidDreamer for an upcoming project that would greatly benefit from this capability. Your insights and advice would be invaluable in helping me to realise this endeavour.

Thank you for your time and the remarkable work you have shared with the community. I look forward to your response.

Best regards,
yihong1120

[ironjr]

Thank you for your acknowledgement and your intriguing idea. Our work is basically a Gaussian splatting generator, but as you may know, current (experimental) rendering pipelines integrate GS with other types of 3D models such as meshes. You can make use of the output .ply files from LucidDreamer and edit with various other types of software:

1. Super-splat: Web-based viewer / editor https://playcanvas.com/super-splat by @playcanvas
2. WebGL-based viewer by @antimatter15
3. Unity engine plugin by @keijiro
4. Unreal engine plugin by @Akiya-Research-Institute
5. Spline Gaussian splatting support
6. three.js integrated editor from Luma Labs AI

On the other hand, if you want our LucidDreamer to receive inputs from 3D objects and existing scene, it could be extended by attaching a rendering pipeline at the input of LucidDreamer.

I would like to note that we are very open to community contributions to improve upon this project. We admit that we do not know all the possible extensions of this project nor have the ability to implement them all. Please open any PR for the project if you have any good idea and we will be graciously integrating your contributions. Thank you.