feat: add fisheye projection for Gaussian splats#8576
Merged
mvaligursky merged 3 commits intomainfrom Apr 3, 2026
Merged
Conversation
Adds a generalized fisheye projection model (g(θ) = k·tan(θ/k)) with Jacobian-corrected covariance for both raster and compute GSplat rendering paths. Controlled via GSplatParams.fisheye (0–1 range). Based on "Optimal Projection for 3D Gaussian Splatting" (https://arxiv.org/abs/2402.00752). Made-with: Cursor
Contributor
There was a problem hiding this comment.
Pull request overview
Adds fisheye projection support to the unified Gaussian splatting pipeline (quad + compute paths), including shader variants, derived projection parameter computation on CPU, and fisheye-aware culling in the compute interval pipeline.
Changes:
- Add
GSplatParams.fisheye(0–1) and introduce aFisheyeProjectionhelper to derive projection constants (k, proj scales, maxTheta). - Add
GSPLAT_FISHEYEshader variants for both GLSL/WGSL vertex paths and WGSL compute path (projection + Jacobian-adjusted covariance). - Switch compute interval culling to cone-based culling when fisheye is enabled; update the LOD streaming example UI with fisheye + environment controls.
Reviewed changes
Copilot reviewed 20 out of 20 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| src/scene/shader-lib/wgsl/chunks/gsplat/vert/gsplatStructs.js | Adds fisheye per-splat shared values to the vertex center struct (WGSL). |
| src/scene/shader-lib/wgsl/chunks/gsplat/vert/gsplatCorner.js | Implements fisheye Jacobian path for covariance projection (WGSL). |
| src/scene/shader-lib/wgsl/chunks/gsplat/vert/gsplatCenter.js | Adds fisheye projection path and fisheye uniforms (WGSL). |
| src/scene/shader-lib/wgsl/chunks/gsplat/compute-gsplat-local-tile-count.js | Extends compute tile-count shader uniforms / call-site to support fisheye (WGSL). |
| src/scene/shader-lib/wgsl/chunks/gsplat/compute-gsplat-interval-cull.js | Adds fisheye cone-culling variant for interval cull (WGSL). |
| src/scene/shader-lib/wgsl/chunks/gsplat/compute-gsplat-common.js | Adds fisheye projection + Jacobian in shared compute covariance routine (WGSL). |
| src/scene/shader-lib/glsl/chunks/gsplat/vert/gsplatStructs.js | Adds fisheye per-splat shared values to the vertex center struct (GLSL). |
| src/scene/shader-lib/glsl/chunks/gsplat/vert/gsplatCorner.js | Implements fisheye Jacobian path for covariance projection (GLSL). |
| src/scene/shader-lib/glsl/chunks/gsplat/vert/gsplatCenter.js | Adds fisheye projection path and fisheye uniforms (GLSL). |
| src/scene/gsplat-unified/gsplat-renderer.js | Adds shared fisheyeProj helper instance to the renderer base. |
| src/scene/gsplat-unified/gsplat-quad-renderer.js | Toggles GSPLAT_FISHEYE define + binds fisheye uniforms for quad path. |
| src/scene/gsplat-unified/gsplat-params.js | Adds new GSplatParams.fisheye API property + docs. |
| src/scene/gsplat-unified/gsplat-manager.js | Updates culling pipeline to feed fisheye parameters and cone-culling state. |
| src/scene/gsplat-unified/gsplat-local-dispatch-set.js | Adds lazy caching for standard vs fisheye tile-count compute resources. |
| src/scene/gsplat-unified/gsplat-interval-compaction.js | Adds cached compute variants for frustum vs fisheye cone culling + parameter wiring. |
| src/scene/gsplat-unified/gsplat-frustum-culler.js | Stores camera cone-culling inputs (pos/forward/maxTheta) for fisheye mode. |
| src/scene/gsplat-unified/gsplat-compute-local-renderer.js | Selects/creates fisheye vs standard tile-count compute variant and binds uniforms. |
| src/scene/gsplat-unified/fisheye-projection.js | New helper computing derived fisheye projection constants from slider + camera state. |
| examples/src/examples/gaussian-splatting/lod-streaming.example.mjs | Adds fisheye UI wiring, FOV clamping, and HDRI environment presets/loader. |
| examples/src/examples/gaussian-splatting/lod-streaming.controls.mjs | Adds fisheye slider + environment selector; expands FOV slider range. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This was referenced Apr 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds fisheye projection support for 3D Gaussian splatting, based on the generalized camera model from Optimal Projection for 3D Gaussian Splatting. The projection uses g(θ) = k·tan(θ/k) with a Jacobian-corrected covariance, enabling wide-angle and "little planet" effects.
Changes:
GSplatParams.fisheyeproperty (0–1 range) to control fisheye projection strengthFisheyeProjectionhelper class computes all derived projection values from the normalized slider inputAPI Changes:
GSplatParams.fisheye: new property (number, 0–1, default 0)Examples:
lod-streamingexample with Fisheye slider and FOV clamping for invalid statesPerformance: