Questions About Demo Scenes & Rendering Pipeline

[pooyafayyaz]

Hi, thanks for the great repo. I had a few questions about the demo scenes and training setup:

1. In scenes like the Cyberpunk demo, are all optional inputs being used (e.g. depth, segmentation, and normals)?

2. Is each demo a single scene trained from scratch, or are multiple splats merged together to produce the final result?

3. What was the approach used to render views from the Unreal/blender Engine scenes? (e.g. capture method, camera path)

[AshadowZ]

Hi! Thanks for your interest. Here are the answers to your questions:

1. Regarding the inputs for the Cyberpunk 2077 demo:
   During training, I used MoGe's depth and normal priors for weak supervision. As for the segmentation, it was achieved after the GS scene was fully trained, using the fs-segment tool provided in the repo.

2. Regarding the training setup (single scene vs. merged):
   Almost all of them are single scenes trained from scratch. The only exception is the MatrixCity scene shown at the end of the video. Because it's quite large (around 16 million Gaussians), it was trained in chunks and then merged together.

3. Regarding the view rendering/capture method:
   It's actually pretty straightforward. Cyberpunk 2077 has a built-in Photo Mode. I just kept the camera intrinsics fixed and took screenshots manually in Photo Mode. If you want to extract more raw buffers (like depth, normals, etc.) directly from UE/games, you might want to check out ReShade. I didn't use ReShade here because the standard Photo Mode was already sufficient for my needs.

I captured about 500 images for this specific 2077 scene. I'll upload all these datasets to a cloud drive later when I update the examples section in the docs.

[pooyafayyaz]

Thanks, having the data helps a lot.