Problem
The 3D Gaussian Splatting paper's key training-time mechanism is densification — periodically SPLIT Gaussians whose accumulated gradient magnitude exceeds a threshold (they cover too much space) and PRUNE Gaussians whose opacity has collapsed to near-zero. This grows the initial cloud (~100k random Gaussians) into millions of specialized ones over the training run, and is what gives real GS its paper-quality output.
GaussianSplattingOptions exposes fields like DensificationInterval, SplitPositionJitter, SplitScaleFactor, SplitOpacityFactor — the config knobs exist. But nothing in the facade path (ConfigureModel → BuildAsync) invokes densification during training. The Gaussian count stays fixed at whatever the constructor produced.
Combined with #1833 (facade optimizer settings silently ignored for GS), this caps GS quality at "random-init cloud with tiny gradient updates" — which is what the Week 11 Session B demo demonstrates visually (a few blurry blobs, unchanged across camera angles, regardless of training epochs).
Fix direction (agreed in design discussion)
Model-internal densification, following DDPM's specialization pattern.
DDPM handles its specialization (noise-prediction training, timestep sampling) entirely inside its inherited NeuralNetworkBase.Train(input, target) path — callers see a uniform Train(x, y) API; the specialized behavior is internal. No callback infrastructure was needed. GS densification is the same shape of problem — an interleaved-during-training operation that mutates the model — so it applies the same pattern.
Concretely: densification lives inside GaussianSplatting.TrainOnImageBatch (per #1834's IImageTrainable<T>). GaussianSplattingOptions.DensificationInterval and friends drive the schedule; the model decides when to densify based on its options. Facade calls TrainOnImageBatch per iteration; the model runs its gradient step + interleaves densification when the schedule fires. No new callback interface, no divergence from how DDPM already works.
Why this exceeds industry standard
- Reference GS implementations (gsplat, nerfstudio, INRIA's official 3DGS repo) hardcode densification inside their training scripts. Users configure via YAML sections but the schedule is baked in — one code path per fork.
- AiDotNet with model-internal densification: densification is a first-class configurable property of
GaussianSplattingOptions — same options object exposes densification + hyperparameters + rendering all in one place. Reference impls fragment these across YAML sections (model config vs. training config vs. optimizer config).
- No new callback infrastructure to teach; no divergence from how DDPM (and future specialized models) work. Uniform mental model.
Beyond-industry excellence goals
Four goals, all agreed to during design discussion:
1. All densification config in one strongly-typed options object (already partially there)
GaussianSplattingOptions already exposes DensificationInterval, SplitPositionJitter, SplitScaleFactor, SplitOpacityFactor, SplitOpacityMax. Extend to cover every industry-standard knob:
DensificationStartIteration — when to begin densifying (skip first N iters while cloud stabilizes)DensificationEndIteration — when to stop (near end of training, freeze the cloud)GradientNormThreshold — the actual split trigger (paper: τ_pos = 0.0002)OpacityPruneThreshold — opacity below which a Gaussian is culled (paper: 0.005)MaxGaussianCount — hard ceiling to prevent OOMGradientAccumulationWindow — how many iterations of gradient norm to average for the split decision
All nullable, all industry-standard defaults (from the 3DGS paper), user can override any.
2. Adaptive densification schedule (data-driven, not fixed interval)
Reference impls use a fixed every N iterations schedule. AiDotNet exceeds by exposing an AdaptiveSchedule option (default off, matches industry when off) that triggers densification when observed signals warrant it:
- Gradient norm variance drops (cloud has stabilized enough that split noise won't destabilize)
- Loss plateaus (cloud can't improve without more capacity)
Adaptive schedule is fully customizable via a strategy interface; industry-standard fixed-interval schedule remains the default.
3. Post-training densification pruning + compression pass
After training, run a compression pass:
- Prune all Gaussians below
OpacityPruneThreshold
- Merge nearby Gaussians whose bounding ellipses overlap by > threshold
- Quantize spherical harmonics (float32 → int8 with a per-Gaussian scale)
Reference impls do this manually as post-processing scripts. AiDotNet exposes it as a first-class option: CompressOnBuildComplete = true (default false — industry-standard: user runs their own post-processing). When enabled, BuildAsync returns with the compressed cloud. Fully customizable via CompressionOptions (per-technique on/off).
4. Per-attribute learning rate schedules for split children
When a Gaussian is split, the two children inherit modified hyperparameters:
- Position LR decreases (they're now local — small movements matter more, big jumps destabilize)
- Scale LR increases (they're smaller — need more updates per step to converge)
- Opacity LR unchanged
Paper implements this manually. AiDotNet exposes via SplitChildLearningRateScales — nullable per-attribute multipliers, industry-standard defaults from the paper, user can override.
Wiring surface
Related
Problem
The 3D Gaussian Splatting paper's key training-time mechanism is densification — periodically SPLIT Gaussians whose accumulated gradient magnitude exceeds a threshold (they cover too much space) and PRUNE Gaussians whose opacity has collapsed to near-zero. This grows the initial cloud (~100k random Gaussians) into millions of specialized ones over the training run, and is what gives real GS its paper-quality output.
GaussianSplattingOptionsexposes fields likeDensificationInterval,SplitPositionJitter,SplitScaleFactor,SplitOpacityFactor— the config knobs exist. But nothing in the facade path (ConfigureModel→BuildAsync) invokes densification during training. The Gaussian count stays fixed at whatever the constructor produced.Combined with #1833 (facade optimizer settings silently ignored for GS), this caps GS quality at "random-init cloud with tiny gradient updates" — which is what the Week 11 Session B demo demonstrates visually (a few blurry blobs, unchanged across camera angles, regardless of training epochs).
Fix direction (agreed in design discussion)
Model-internal densification, following DDPM's specialization pattern.
DDPM handles its specialization (noise-prediction training, timestep sampling) entirely inside its inherited
NeuralNetworkBase.Train(input, target)path — callers see a uniformTrain(x, y)API; the specialized behavior is internal. No callback infrastructure was needed. GS densification is the same shape of problem — an interleaved-during-training operation that mutates the model — so it applies the same pattern.Concretely: densification lives inside
GaussianSplatting.TrainOnImageBatch(per #1834'sIImageTrainable<T>).GaussianSplattingOptions.DensificationIntervaland friends drive the schedule; the model decides when to densify based on its options. Facade callsTrainOnImageBatchper iteration; the model runs its gradient step + interleaves densification when the schedule fires. No new callback interface, no divergence from how DDPM already works.Why this exceeds industry standard
GaussianSplattingOptions— same options object exposes densification + hyperparameters + rendering all in one place. Reference impls fragment these across YAML sections (model config vs. training config vs. optimizer config).Beyond-industry excellence goals
Four goals, all agreed to during design discussion:
1. All densification config in one strongly-typed options object (already partially there)
GaussianSplattingOptionsalready exposesDensificationInterval,SplitPositionJitter,SplitScaleFactor,SplitOpacityFactor,SplitOpacityMax. Extend to cover every industry-standard knob:DensificationStartIteration— when to begin densifying (skip first N iters while cloud stabilizes)DensificationEndIteration— when to stop (near end of training, freeze the cloud)GradientNormThreshold— the actual split trigger (paper:τ_pos = 0.0002)OpacityPruneThreshold— opacity below which a Gaussian is culled (paper:0.005)MaxGaussianCount— hard ceiling to prevent OOMGradientAccumulationWindow— how many iterations of gradient norm to average for the split decisionAll nullable, all industry-standard defaults (from the 3DGS paper), user can override any.
2. Adaptive densification schedule (data-driven, not fixed interval)
Reference impls use a fixed
every N iterationsschedule. AiDotNet exceeds by exposing anAdaptiveScheduleoption (default off, matches industry when off) that triggers densification when observed signals warrant it:Adaptive schedule is fully customizable via a strategy interface; industry-standard fixed-interval schedule remains the default.
3. Post-training densification pruning + compression pass
After training, run a compression pass:
OpacityPruneThresholdReference impls do this manually as post-processing scripts. AiDotNet exposes it as a first-class option:
CompressOnBuildComplete = true(defaultfalse— industry-standard: user runs their own post-processing). When enabled,BuildAsyncreturns with the compressed cloud. Fully customizable viaCompressionOptions(per-technique on/off).4. Per-attribute learning rate schedules for split children
When a Gaussian is split, the two children inherit modified hyperparameters:
Paper implements this manually. AiDotNet exposes via
SplitChildLearningRateScales— nullable per-attribute multipliers, industry-standard defaults from the paper, user can override.Wiring surface
ConfigureModel(existing) — no change;GaussianSplattingOptionsextended.ConfigureOptimizer(existing) — feeds into the per-attribute schedules (per GaussianSplatting facade training silently ignores ConfigureOptimizer settings (LR, betas, weight decay) — uses internal updates instead #1833's hyperparameter routing).Configure*methods.Related
IImageTrainable<T>is where GS'sTrainOnImageBatchlives; densification runs inside it)