Splatshop (v0.01f)

This editor aims to enable editing of gaussian splat models, e.g., cleaning up reconstructed models, assembling cleaned-up assets into a library, composing scenes, and painting. It currently supports up to a hundred million splats on desktop and up to around ten million in VR on an RTX 4090.

There is still a lot of work to do - we are releasing it now in order to gather feedback and figure out which features and fixes to prioritize.

An example data set is available here. Unpack it, then drag&drop it into the editor. The garden splat model is from Inria 3DGS, and originates from Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields.

A precompiled windows binary is available here. However, it may still be necessary to install CUDA 12.4 or later since CUDA kernels are compiled at startup.

Features

• Undo and Redo
• Virtual Reality viewing and editing
• Select via brush, sphere, rectangle; Delete with brush or del button
• Translate, scale, rotate of selected nodes or splats.
• Painting
• No pip or conda issues! (no Python)
• CUDA Driver API - Can edit and hot-reload CUDA code like shaders.
• Includes support for perspective-correct gaussians. (Splat models need to be trained with respective method)

Known Issues

• Academic Prototype - expect a fair amount of bugs and crashes -> Consider it an "alpha" version.
• VR only on windows - No idea how to include OpenVR on linux.
• No Spherical Harmonics due to large mem req. with modest gains - we will check out more frugal SH options in the future.
• CUDA Driver API - Long startup because CUDA compiles at runtime.
• No precompiled binaries yet.
• Undo and redo may not cover all functionality yet.
• Undo of translate/scale/rotate is currently lossy. Especially scale is prone to loss of precision when scaling too far down.
• App freezing: Line rendering caused freezes on RTX 20xx series cards. That was fixed, but there is a chance that triangle models in VR mode may also freeze RTX 20xx series cards.

Installing

Windows

Dependencies:

• CUDA 12.4
• Visual Studio 2022 (version 17.10.3)
• CMake 3.27
• RTX 4070 or better recommended.

Create Visual Studio solution files in a build folder via cmake:

mkdir build
cd build
cmake ../

Open build/SplatEditor.sln and compile in Release mode.
Then run from project directory.

./build/Release/SplatEditor.exe

Running from project directory is semi-important because the editor will look for resources such as ./src/gaussians_rendering.cu and ./resources/images/symbols_32_32.png relative to the project directory. Alternatively, you can copy the resources and src folders into the binary directory.

Linux

Dependencies:

• gcc 14, g++14
• CUDA 12.4 or higher
• NVIDIA driver version 550 or higher. (Lower may cause issues when compiling GPUSorting)

mkdir build
cd build
export CUDA_PATH=/usr/local/cuda-12.4/
cmake -DCUDA_CUDART_LIBRARY=/usr/local/cuda/lib64/libcudart.so ..
make

Potential Issues:

• May or may not require TBB. Uncomment the TBB part in CMakeLists, if it does.

// set CUDA_PATH variable to wherever your CUDA Toolkit is installed
export CUDA_PATH=/usr/local/cuda-12.4/

// May or may not require setting LD_LIBRARY_PATH to gcc 14, similar to this, or whereever gcc14 is installed
export LD_LIBRARY_PATH=~/gcc-14.1.0/x86_64-linux-gnu/libstdc++-v3/src/.libs:$LD_LIBRARY_PATH

// run from workspace root
./build/SplatEditor

Getting Started

Key and Mouse (as hardcoded in inputHandlingDesktop.h)

Right Click	Cancel current action
Double Click	Move towards hovered splats
Ctrl + z, Ctrl + y	Undo, Redo
1	Brush-Selection Action
2	Brush-Deletion Action
t, r, s	Translate, Rotate, Scale
b	Painting
Ctrl + D	Duplicate selection to new layer
Ctrl + E	Merge selected layer into the one below

Delete and Duplicate

1	2	3
4	5	6

1. Select splats with spherical selection brush.
2. Delete selection (key: del).
3. Select a patch of grass with brush selection and intersection set to "center". This avoids selecting nearby large splats.
4. Duplicate selection (key: ctrl + d), then move it with translation tool (key: t).
5. Repeat with varying patches of grass until the hole is covered. Overlapping lighter and darker patches may result in near-seamless transitions.

Cleanup

1	2	3
4	5	6

1. Drag&Drop ply file
2. Select region of interest with spherical selection tool.
3. Invert selection and delete (key: del).
4. Rotate until model aligns with ground plane (key: r) and translate to origin (key: t). If the ground is sufficiently densely reconstructed, you can also try using the 3-point-alignment tool, which aligns the model by specifying three points on the ground.
5. Remove undesired splats with a combination of circular and spherical selection and deletion tools. To retain a nice circular ground, select splats with a spherical brush
6. Then select the remainder of the splats that should remain. Invert selection and delete.

Add an asset to library

1. Select splats
2. Duplicate selection (key: ctrl + d)
3. In the layers menu, right click the duplicated layer and give it a new name.
4. Right click the layer again, and select "Create asset from Layer".

Adding new actions to toolbar

• Choose a suitable existing action and copy it, e.g. BrushSelectAction.
• Rename accordingly and adapt the code to your needs. Perhaps remove the undoable parts for now.
• update() is called every frame and handles most things.
• makeToolbarSettings() allows you do specify an imgui user interface that is shown below the toolbar while your action is active.
• Add your action to toolbar.h. Include it and add a Button or ImageButton, similar to other actions. When the button is clicked, create a shared_ptr of your action and call setAction(action) to activate it. It will automatically be deactivated on right-click.
• If you need to add an icon, you can add one to symbols.svg and export it to symbols_32x32.png.

Acknowledgements

• Contributors: Markus Schütz, Christoph Peters, Florian Hahlbohm, Elmar Eisemann, Michael Wimmer
• Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, George Drettakis for 3D Gaussian Splatting for Real-Time Radiance Field Rendering
• Mark Kellog for his WebGL implementation https://github.com/mkkellogg/GaussianSplats3D which helped substantially with the 3DGS implementation.
• Thomas Smith for his GPUSorting project that is used to sort the gaussians in this project.
• Omar's Dear ImGui.
• This particular garden model is a pretrained model from Inria 3DGS and rotated to fit our editor. The garden data set originates from Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields.
• SuperSplat - A web-based gaussian splat editor, from which we learned the usefullness of the ring rendering mode for editing, especially finding and removing floaters.