Multi VAO loading

[chenqi13814529300]

I tried loading multiple Gaussian ploys and instantiating multiple VAOs. After loading, it was found that there were some issues with the rendering of the Gaussian sphere at the edges. Some angles are normal, while others are very poor

[chenqi13814529300]

[hyperlogic]

This is likely due to the rendering order of the VAOs. Gaussian splats require depth sorting in order to render correctly. If the splats from two different VAOs overlap/interpenetrate, they will likely exhibit errors like the ones shown in the images you provided (I assume you set the background color to red)

You are already making changes to the source code, so I would suggest making another modification. Specifically, combining the data from both ply files into one. Here's how you "might" achieve this.

diff --git a/src/gaussiancloud.cpp b/src/gaussiancloud.cpp
index 654d7f7..363c6b9 100644
--- a/src/gaussiancloud.cpp
+++ b/src/gaussiancloud.cpp
@@ -92,34 +92,38 @@ bool GaussianCloud::ImportPly(const std::string& plyFilename)
         }
     }

-    gaussianVec.resize(ply.GetVertexCount());
+    std::vector<Gaussian> gVec;
+    gVec.resize(ply.GetVertexCount());

     int i = 0;
     ply.ForEachVertex([this, &i, &props](const uint8_t* data, size_t size)
     {
-        gaussianVec[i].position[0] = props.x.Get<float>(data);
-        gaussianVec[i].position[1] = props.y.Get<float>(data);
-        gaussianVec[i].position[2] = props.z.Get<float>(data);
+        gVec[i].position[0] = props.x.Get<float>(data);
+        gVec[i].position[1] = props.y.Get<float>(data);
+        gVec[i].position[2] = props.z.Get<float>(data);
         for (int j = 0; j < 3; j++)
         {
-            gaussianVec[i].f_dc[j] = props.f_dc[j].Get<float>(data);
+            gVec[i].f_dc[j] = props.f_dc[j].Get<float>(data);
         }
         for (int j = 0; j < 45; j++)
         {
-            gaussianVec[i].f_rest[j] = props.f_rest[j].Get<float>(data);
+            gVec[i].f_rest[j] = props.f_rest[j].Get<float>(data);
         }
-        gaussianVec[i].opacity = props.opacity.Get<float>(data);
+        gVec[i].opacity = props.opacity.Get<float>(data);
         for (int j = 0; j < 3; j++)
         {
-            gaussianVec[i].scale[j] = props.scale[j].Get<float>(data);
+            gVec[i].scale[j] = props.scale[j].Get<float>(data);
         }
         for (int j = 0; j < 4; j++)
         {
-            gaussianVec[i].rot[j] = props.rot[j].Get<float>(data);
+            gVec[i].rot[j] = props.rot[j].Get<float>(data);
         }
         i++;
     });

+    // concatinate gVec on the end of gaussianVec.
+    gaussianVec.insert(gaussianVec.end(), gVec.begin(), gVec.end());
+
     return true;
 }

After this change you can load two ply files into a single GaussianCloud instance.

gaussianCloud->ImportPly(ply1);
gaussianCloud->ImportPly(ply2);

then pass that gaussianCloud to splatRenderer->Init(). This should "in theory" render both ply files together as a single VAO, which will eliminate the sorting errors.

Hope this helps.

[chenqi13814529300]

Thank you very much for your answer. Unfortunately, two weeks ago, I had already completed the plan you just proposed. Enter the folder path containing multiple Gaussian folders. Then, multiple ply layers are formed into one ply layer, and VAO is called for display. The rendering was successful without any issues.
In the past few days, I believe that editing and other operations may be necessary for future single Gaussian layers.
Gaussian space requires depth sorting to render correctly. I completely agree that if rendering multiple VAOs, it may be necessary to perform depth sorting on all VAOs, but I think this is still a bit difficult. I will continue to consider how to implement him, and I would be very happy if you have any good suggestions.

[hyperlogic]

Apologies for the delay. Perfect rendering of interpenetrating .ply files without full sorting isn't possible. But if the splats aren't fully interpenetrating, but are partitioned into a grid or some other pattern. You could sort by each centroid and render in back to front order. Also, if you're willing to live with some incorrect results along the edges of two grid cells, you can just turn off the depth test, this would get rid of those "red gaps". There would likely be some flickering as the sort order changed.. but it would be "mostly" correct.

Here's were you can disable the depth test.

diff --git a/src/app.cpp b/src/app.cpp
index f2d5108..cc43efb 100644
--- a/src/app.cpp
+++ b/src/app.cpp
@@ -144,7 +144,7 @@ static void Clear(glm::ivec2 windowSize, bool setViewport = true)
     glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

     // NOTE: if depth buffer has less then 24 bits, it can mess up splat rendering.
-    glEnable(GL_DEPTH_TEST);
+    //glEnable(GL_DEPTH_TEST);

 #ifndef __ANDROID__
     // AJT: ANDROID: TODO: implement this in fragment shader, for OpenGLES I guess.

[chenqi13814529300]

Before your reply, I also tried to turn off deep testing (or glEnable (GL-DEPTH-TEST) myself;
GlDepthFunc (GL-ALWAYS);). There has indeed been some improvement, but I am well aware that it is not very reasonable.
I saw your reply today. What you mean is to take the center coordinates of each tile and calculate the distance from the camera position. Then render in reverse order of distance. I have tried and there is some relief, but it is not as effective as removing the depth test. Because when the camera is above the seam of two tiles, the effect is still not very good.

        struct RendererDistancePair {
            std::shared_ptr<SplatRenderer> renderer;
            float distance;


            bool operator<(const RendererDistancePair& other) const {

                return distance > other.distance;
            }
        };


        std::vector<RendererDistancePair> pairs;

        for (int i = 0; i < vecSplatRenderer.size(); i++) {
            std::vector<glm::vec4> posVec = vecSplatRenderer[i]->posVec;
            glm::vec3 center = calculateCenter(posVec);

            glm::vec4 camPosition = glm::inverse(cameraMat) * glm::vec4(center, 1.0f);
            float distanceInViewSpace = glm::distance(glm::vec3(camPosition), objectCenter);


            pairs.push_back({ vecSplatRenderer[i], distanceInViewSpace });
        }


        std::sort(pairs.begin(), pairs.end());


        for (const auto& pair : pairs) {
            std::shared_ptr<SplatRenderer> renderer = pair.renderer;
            renderer->Sort(cameraMat, projMat, viewport, nearFar);
            renderer->Render(cameraMat, projMat, viewport, nearFar);
        }

[chenqi13814529300]

Why does the super-splat project on Github add multiple Gaussian files that render exactly right. This one is based on WebGL. Is it also just rendering a VAO？