Skip to content

Support Nemotron 3.5 Lightning - #837

Merged
FurtherAI merged 7 commits into
mainfrom
austin/nemotron3_5_lightning
Sep 3, 2026
Merged

Support Nemotron 3.5 Lightning#837
FurtherAI merged 7 commits into
mainfrom
austin/nemotron3_5_lightning

Conversation

@FurtherAI

Copy link
Copy Markdown
Collaborator

Summary

  • register the exact nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 checkpoint at revision b3caaabed0263651a17dc1f2d4ce97e794f76c44
  • reuse the existing NemotronHHandler and already-qualified Nano Mamba-2, MoE, attention, context-parallel, LoRA, export, and native-vLLM paths because Lightning's 52-layer backbone geometry and schedule are identical
  • deliberately disable Lightning's MTP configuration and exclude MTP tensors from HF validation; MTP training, adapters, export, and serving remain unsupported by design
  • support Lightning's standard-Transformers config/checkpoint layout, built-in HF oracle, and model.layers LoRA namespace without changing Nano's remote-code backbone.layers contract
  • make shared reduced fixtures understand serialized layers_block_type schedules and derive a missing scalar depth from that canonical schedule

Scope

This is an exact-checkpoint, same-architecture qualification. It does not change attention, FlexAttention, recurrent/Mamba-2, MoE, CP, or serving math. It does not alter train/inference thresholds, throughput floors, timeouts, lifecycle behavior, or compile settings. The throughput entry binds Lightning's exact workload fingerprint to the existing Nemotron-H profile without changing any floor.

Supported here: the BF16 backbone of NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 only. Lightning MTP, FP8/NVFP4 variants, Base/Super/Ultra variants, and multimodal checkpoints are not enabled.

Validation

  • all-files prek hooks pass after rebasing onto current main
  • focused provider, HF parity, padding, and workflow invariants: 96 passed, 1 intentional skip
  • reduced HF parity: 61/61 checks; output MAPE 0.0017235%, worst gradient MAPE 0.321838% (FP32 0.5% gate)
  • exact LoRA coverage/export: all 9 requested module families, no missing or unexpected families
  • CP2/EP2 train/inference with native vLLM LoRA: 0.4566266% logprob MAPE (4% gate), 0.00068274 top-20 KL (0.002 gate), 98.4375% top-1 match
  • FP32 packing invariance: all 4 fork/truncate/deep/repeated cases pass; worst logits MAPE 0.0057305%
  • production-width six-layer CP2/EP2 trainability: one completed update, finite nonzero 22.933194 gradient norm, checkpoint publication, and generation from the exact updated native-vLLM adapter
  • reduced throughput stage: 43,389.264 isolated physical tok/s, 43,311.030 matched E2E trainer-core tok/s, 0.998556 matched/isolated ratio, 41,555.020 E2E tok/s, 985.860 accepted tok/s, and 9.379706% useful MFU; W&B disabled

The complete default model-support workflow was intentionally not run for this same-architecture alias qualification. The optional correctness-sensitivity suite was also intentionally not run.

@FurtherAI
FurtherAI merged commit 03f5ffb into main Sep 3, 2026
7 checks passed
@FurtherAI
FurtherAI deleted the austin/nemotron3_5_lightning branch September 3, 2026 00:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant