Initialize gradient accumulation tensors before UT projection path#608
Merged
Merged
Conversation
The Unscented Transform (UT) projection path in _do_projection returned early without lazily initializing _accumulated_gradient_step_counts and _accumulated_mean_2d_gradient_norms. This left these tensors as None when accumulate_mean_2d_gradients was True, causing downstream consumers (e.g., refinement in fvdb-reality-capture) to crash. Move the lazy initialization block before the UT early return so the tensors are always created when gradient accumulation is enabled, regardless of which projection method is used. The UT kernel does not accumulate into these tensors, so they remain zero — correct "no data" representation until full UT gradient accumulation support is added. See: openvdb/fvdb-reality-capture#279 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Mark Harris <mharris@nvidia.com>
harrism
force-pushed
the
issue-279-ut-gradient-accumulation
branch
from
b070048 to
dcf3494
Compare
harrism
added
the
Gaussian Splatting
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes a missing side effect in GaussianSplat3d._do_projection where the UNSCENTED (UT) projection early-return bypassed lazy initialization of gradient-accumulation tensors, leaving them as None even when gradient accumulation was enabled.
Changes:
- Move lazy initialization of
_accumulated_gradient_step_counts,_accumulated_mean_2d_gradient_norms, and_accumulated_max_2d_radiito occur before the UT early-return path in_do_projection. - Add a unit test asserting that the UNSCENTED projection path initializes gradient accumulation state when
accumulate_mean_2d_gradients=True.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
fvdb/gaussian_splatting.py |
Ensures UT projection path runs gradient-accumulation lazy init before returning early. |
tests/unit/test_gaussian_splat_3d.py |
Adds regression test covering UT-path initialization of accumulated gradient tensors. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
fwilliams
added a commit
that referenced
this pull request
Apr 15, 2026
Resolve conflict in fvdb/gaussian_splatting.py: accept main's structural change from PR #608 (move UT projection block after gradient accumulation init), then re-apply our ut->unscented rename and empty shN fix for degree-0 SH evaluation. Signed-off-by: Francis Williams <francis@fwilliams.info> Made-with: Cursor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
_accumulated_gradient_step_counts,_accumulated_mean_2d_gradient_norms,_accumulated_max_2d_radii) to before the Unscented Transform early return in_do_projectionaccumulate_mean_2d_gradients=TrueRoot Cause
The UT projection path in
_do_projectionreturned early (line ~1500) without lazily initializing the gradient accumulation tensors. The lazy init block (lines ~1524-1547) only ran for the ANALYTIC path. This leftaccumulated_gradient_step_countsandaccumulated_mean_2d_gradient_normsasNonefor the entire training run when using OpenCV camera models, causing downstream consumers to crash.Fix
Move the lazy init block before the
_use_ut()check. The UT kernel (_C.project_gaussians_ut_fwd) does not accumulate into these tensors, so they remain zero — a correct "no gradient data" representation. Full UT gradient accumulation support can follow as a separate enhancement.See: openvdb/fvdb-reality-capture#279
Test plan
test_ut_projection_initializes_gradient_accumulationfails before fix (asserting None), passes afterTestGaussianCameraApitests pass with zero regressions🤖 Generated with Claude Code