Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Render compressed GS data #6371

Merged
merged 22 commits into from
May 20, 2024
Merged

Conversation

slimbuck
Copy link
Member

@slimbuck slimbuck commented May 15, 2024

This is a followup PR to #6357.

Up until now when loading compressed gaussian splat PLY files, we have simply decompressed the data at load time and rendered the scene as usual.

This PR adds a render path to load and render the compressed data directly, cutting memory consumption by 4x for the raw splat data and speeding up rendering by a modest 4%.

For the following bicycle scene:

Old New
Render time 13.4ms 13.0ms
Total VRAM 208MB 123MB
Load Time 6.0s 1.8s

NOTE: load time above includes the examples browser startup.

Screenshot 2024-05-15 at 14 00 55

This PR makes the following changes:

  • adds a createIter function to GSplatData, which constructs an iterator for accessing gaussian splat data. the iterator works the same whether the underlying data is compressed or not.
  • makes use of the new iterator in the rest of GSplatData, making most code smaller and simpler.
  • adds compressed versions of gsplat and gsplat-material for rendering compressed data directly.
  • adds asset.data options which control whether to decompress the data at load time (useful for tools like super-splat) and also whether to reorder data for better GPU render performance.
  • speed up gs bounding box calculation by using splat bounding sphere (i.e. overestimation) and rename the exact bound calculation calcAabbExact.

Todo:

  • it may be possible to speed up rendering by early-out based on chunk bound (to be investigated)
  • we might prefer to optimise the compressed format:
    • store v1v2 instead of rotation/scale
    • store rotation/scale/v1v2 in halfs instead of floats

@slimbuck slimbuck added area: graphics Graphics related issue enhancement labels May 15, 2024
@slimbuck slimbuck requested a review from a team May 15, 2024 13:27
@slimbuck slimbuck self-assigned this May 15, 2024
Copy link

vercel bot commented May 15, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
engine ✅ Ready (Inspect) Visit Preview 💬 Add feedback May 20, 2024 9:52am


const splatCoreVS = /* glsl */ `

uniform mat4 matrix_model;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is so much duplication between this and non-compressed material. I think we should unify them, and use ifdef inside shader to pick the right bits.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes there is some duplication, but I'd like to keep the two materials separate until the optimisations are final.

Copy link
Contributor

@willeastcott willeastcott left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔥🔥🔥

slimbuck and others added 3 commits May 20, 2024 10:47
Co-authored-by: Will Eastcott <willeastcott@gmail.com>
Co-authored-by: Will Eastcott <willeastcott@gmail.com>
@slimbuck slimbuck merged commit d09fa8c into playcanvas:main May 20, 2024
9 checks passed
@slimbuck slimbuck deleted the gs-take-two-compressed branch May 20, 2024 09:55
slimbuck added a commit to slimbuck/engine that referenced this pull request May 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: graphics Graphics related issue enhancement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants