NeMo Framework 26.08 — Megatron Bridge Release Notes

Public roadmap: issue #5144

Featured Blog

Accelerating GLM-5.2 Long-Context Training with cuDNN Fused DSA Kernels and THD-Packed Context Parallelism using Megatron Bridge

Learn how cuDNN fused DSA kernels, GLM-5.2 IndexShare, context parallelism, and THD-packed sequences enable efficient 128K-context GLM-5.2 training with Megatron Bridge—delivering up to ~3.4× higher throughput than the TileLang reference on NVIDIA GB200 and up to ~2× on NVIDIA H100 while preserving training convergence.

Highlights

Model Collection Support

LLM

GLM-5.2 — bridge and recipes (GB200/H100) with the cuDNN DSA path; pure text LLM, no vision encoder (PR#4520, issue #4490, community @HollowMan6)
EXAONE 4.0 — HF ↔ Megatron checkpoint conversion for LG AI Research EXAONE 4.0 (PR#2532, community @Bias92)
K-EXAONE and K-EXAONE 2.0 — EXAONE MoE causal LM bridge (PR#4730, PR#5191, community @sangHa0411)
Gemma-4 E4B — Bridge-native dense text provider, HF ↔ Megatron conversion, parity checks, and launch scripts (PR#4148, community @DOGEUNNKIM)
HunYuan V3 — HYV3Bridge for the Hy V3 MoE causal LM (PR#4680, community @xhx1022)
Nemotron 3.5 Lightning — bridge, recipes, and MTP support (PR#5035)
DeepSeek V4 — enhanced support, verification in progress (PR#5360, issue #4633)
- Offline-packed SFT with THD and contiguous context parallelism, required by the DSv4 CSA compressor that exchanges boundary hidden states between adjacent CP ranks (PR#5126)

VLM / Multimodal

MiniMax M3 — full multimodal support: language-model bridge and recipes (PR#4652, community @khazic) plus VLM import/export (PR#5058, issue #4585)
EXAONE 4.5 — LG's first VLM: EXAONE 4.0 32B base with a 1.2B vision encoder (PR#4730, community @sangHa0411)

Diffusion

LLaDA 1.5 — masked-diffusion language-model bridge with HF ↔ Megatron conversion and a block-diffusion generation path (PR#4019)

Training & Functionality

Unified VLM dataset architecture — the Hugging Face and Energon paths now share one configuration contract, one processing layer, and one set of collators
- Serializable Config + Builder contract (PR#4685, PR#4774)
- Single model-aware collator for Direct-HF and Energon (PR#4776)
- Shared VLM processing helpers and model-owned collators (PR#4109, PR#4166, issue #4041 Step 1)
- Enforced collator import boundaries (PR#4528, PR#4571)
- Finetuning batch helpers split out (PR#4602)
- Stable tokenizer fingerprinting for packed paths (PR#4817)
- Explicit fixed padding preserved through validation (PR#4842)
- Model-owned processor defaults preserved (PR#5218)
Reliable assistant-only loss masking — fragile text-search masking replaced with a layered fallback strategy (issue #4041 Step 3)
- Template-anchored masks with a documented fallback chain (PR#4169)
- Same mask helpers reused for text chat collate (PR#4307)
- Per-model boundaries for Ministral3 (PR#4477), Kimi (PR#4478), and GLM-4.5V (PR#5001)
- VLM contract fixes for GLM content shape (PR#5216) and Qwen token type IDs (PR#5215)
- Schema normalization for ShareGPT aliases (PR#5078) and custom chat JSONL (PR#5012)
- Truncation without surviving supervision now rejected (PR#5217)
More reliable in-batch packing — in-microbatch concatenation behind a pad-then-strip round trip replaced with direct packed-batch construction (issue #4041 Step 2)
- Packing moved from the forward step into collate (PR#4315, PR#4255)
- Direct packed THD batches with current MCore/TE metadata (PR#4507)
- Packed-metadata fixes for Omni cu-seqlens argmin (PR#4400) and ragged offsets (PR#5400)
- Long-context THD + CP packed SFT (PR#4598)
- Seeded, reproducible offline packing (PR#4891)
- Padding sentinels no longer supervised (PR#4943, PR#4675, PR#4926, PR#4056)
Checkpoint, resume, and export usability
- Save HF weights alongside native Megatron checkpoints (PR#3825, community @dancingpipi)
- Energon dataloader save/resume restores the same data stream, keeping the loss curve continuous (PR#4521, community @zsombor-kaiko)
- Removed a duplicate TE FusedAdam FP32 state allocation in distributed-optimizer resume, eliminating a memory spike (PR#4936)
PEFT / LoRA
- Multi-GPU adapter export for large PEFT checkpoints (PR#4221)
- Reduced LoRA activation memory by re-gathering inputs from sequence-parallel regions (PR#4581)
- Shared outer LoRA so MoE models can be served by SGLang with a matching weight layout (PR#3958, community @nanjiangwill)
Training-loop upstreaming — Megatron Core now owns the serializable config + builder contract Bridge pioneered
- ModelBuilder fully replaces the legacy model_builder function pointer and get_model() call sites (MCore PR #5516)
- Completes the HybridModel (MCore PR #4550) and GPTModel (MCore PR #4741) builder migrations; Bridge dropped its forked copies (PR#3988)
Epoch-based training — supported in addition to iteration-based infinite sampling (PR#4147, community @muyihao)
Masked-LM support — PreTrainedMaskedLM wrapper for *ForMaskedLM architectures (PR#4735, community @bbuschkaemper)
MegatronMIMO expert parallelism — expert-parallel language modules (PR#4485)

Checkpoint Conversion

NVFP4 HF weight export — native ModelOpt export path (PR#4102, community @HollowMan6)
Grouped-MoE quantized export — W4A16 and W4A4 NVFP4 for grouped-MoE weights (PR#4566, community @HollowMan6)
ModelOpt KV-cache amax mapping — enables simulated KV-cache quantization in NeMo-RL (PR#4591)

Performance

Refactored performance recipes — one flat recipe surface across all model families (PR#2803)
- Device-specific library recipe namespace (PR#4622)
- Legacy perf config tree removed (PR#4623)
- Recipe environment-variable support (PR#4682)
- Flat perf-recipe names routed through train.sh --recipe (PR#4945)
DeepSeek V4 Flash MXFP8 benchmark recipes — 128-GPU configs at ~716 TFLOP/s/GPU on GB200 and ~763 TFLOP/s/GPU on GB300 (PR#5258)
VR200 performance configs — platform and environment behavior carried into the flat recipe layout (PR#5196)
Nemotron 3.5 Nano H100 benchmark tuning — retuned benchmark recipe (PR#5266)

Examples & Tutorials

Unified, Slurm-enabled CLI entry points for all models — one scripts/training/train.sh (PR#4856, PR#4945) and scripts/conversion/convert.sh with distributed roundtrip support (PR#4873, PR#4886), replacing per-model launchers
- The model-specific scripts under examples/models/ are superseded and will be phased out gradually by the next release
- Each unified script is accompanied by a model verification card recording per-model verification status
  - Cards published this release: GLM-5.2 (PR#5034), Nemotron 3.5 Nano (PR#5124, PR#5199), Gemma 3 VL (PR#5084), and DeepSeek V4 Flash (PR#5360)
Nemotron 3 Ultra recipes and examples
- Conversion, inference, DCLM pretraining, packed OpenMath SFT and PEFT, plus packing scripts and docs (PR#4224)
- Performance recipes (PR#4911, PR#5287)
MoE training tutorial — notebook walkthrough of MoE recipe usage (PR#3890)
Pure evaluation example — returns all averaged test losses from a checkpoint (PR#4125)

Community Contributions

@HollowMan6 — GLM-5.2 bridge and recipes; ModelOpt NVFP4 and grouped-MoE quantized HF export
@sangHa0411 — EXAONE 4.5 (VLM) and K-EXAONE (MoE LLM) bridges
@HowardZorn — Qwen3.5 text model bridges (dense + MoE)
@Bias92 — EXAONE 4.0 model bridge
@DOGEUNNKIM — Gemma-4 E4B support
@xhx1022 — HunYuan V3 MoE bridge
@khazic — MiniMax M3 language-model bridge and recipes
@bbuschkaemper — Masked-LM HF-pretrained wrapper support
@dancingpipi — save HF weights alongside native Megatron checkpoints
@zsombor-kaiko — Energon dataloader checkpointing (save/resume)
@nanjiangwill — shared outer LoRA support
@muyihao — epoch-based training duration for finetuning

A big thank you to our community contributors for their valuable support!

Deprecated Models

The following legacy model families are deprecated as of 26.08 (0.6.0) and scheduled for removal in 0.7.0. They still load and run in this release but emit a FutureWarning on checkpoint load, and are no longer actively maintained or tested against current upstream checkpoints. If you depend on any of these, pin megatron-bridge<0.7.0 or migrate to the current generation of the family before upgrading (PR#5025, PR#5125).

Llama 2 — superseded by Llama 3.x
Mistral 7B and Mistral Small 3 24B
Gemma 1 (2B, 7B) and Gemma 2 (2B, 9B, 27B) — superseded by Gemma 3 / Gemma 4
DeepSeek V2 and DeepSeek V2 Lite — superseded by DeepSeek V3 / V4
Llama Nemotron (70B, Nano 8B/4B, Super, Ultra)
Nemotron H v1 (4B, 8B, 47B, 56B) and Nemotron Nano v2 (9B, 12B) — superseded by the HybridModel Nemotron 3.x line
Nemotron Nano v2 VL 12B
Legacy Nemotron (NemotronForCausalLM, Nemotron-4 340B)

Known Issues

TE fused cross-entropy may degrade stability: setting cross_entropy_fusion_impl to te causes convergence issues on some models. Use native instead, or upgrade Transformer Engine to 2.18, which carries the fix (TE PR #3193).
SPCX payload corruption: the NCCL 2.30.5/SPCX stack can corrupt data in two observed cases:
- Grouped all-to-all at high expert parallelism: variable-split BF16 sends can be misclassified in MCore's MoE alltoall dispatcher. HybridEP is an alternative.
- P2P: shared tagged-multireceive can report completion before all data arrives, reproduced with raw NCCL. Disable shared comms with NCCL_NET_SHARED_COMMS=0.
- Workaround: use NCCL's internal IB transport (NCCL_NET_PLUGIN=none NCCL_NET=IB). Tracking: issue #5462.
Step-3.7-Flash checkpoint parity is not yet verified.
Large-model round-trip validation can exceed the NCCL process-group timeout: every rank participates in export, but rank 0 lazily loads and exhaustively compares the original Hugging Face tensors while peers may reach the synchronization collective first. This expected rank skew can exceed the default 600-second timeout even though rank 0 is progressing. Set --distributed-timeout-minutes above the worst-case verification time; for checkpoints too large for practical exhaustive validation, use separate import and export --distributed-save workflows instead of roundtrip (PR #5492).

Changelog Details

[recipe][perf] feat: unified flat performance recipes for all model families by @yaoyu-33 :: PR: #2803
refactor(data): adopt config and builder dataset architecture by @yaoyu-33 :: PR: #4685
fix(training): align GDN MTP FLOPs with decoder layout by @yaoyu-33 :: PR: #4834
refactor(data)!: unify multimodal dataset builders by @yaoyu-33 :: PR: #4774
feat(conversion): add shared checkpoint launcher by @yaoyu-33 :: PR: #4873
fix(conversion): resolve tied output aliases from parameter mappings by @yaoyu-33 :: PR: #4884
fix(training): restore SFT cache and MoE overlap compatibility by @yaoyu-33 :: PR: #4882
test: consolidate redundant unit coverage by @yaoyu-33 :: PR: #4887
refactor(misc): minimize private Transformer Engine integrations by @cuichenx :: PR: #4772
chore(skills): add Regent Open Plugin manifest by @ko3n1g :: PR: #4892
chore(skills): remove Open Plugin manifest (superseded) by @ko3n1g :: PR: #4894
[ckpt] fix: restore RNG state when future saves disable it by @yaoyu-33 :: PR: #4893
[inference] fix: respect prompt file truncation limit by @yaoyu-33 :: PR: #4896
[training] fix: resume global non-persistent checkpoints by @yaoyu-33 :: PR: #4900
fix(recipe): add FC LoRA targets for Nemotron Omni by @cuichenx :: PR: #4853
[data] feat: Add Tulu 3 dataset preset and current validation evidence by @yaoyu-33 :: PR: #4888
[data] fix: isolate and unify Nemotron Omni collation by @cuichenx :: PR: #4776
fix(qwen3-asr): preserve native auto registrations by @aoshen02 :: PR: #4876
fix(training): reset model callbacks during in-process restart by @yaoyu-33 :: PR: #4889
[data] fix: honor seed during offline packing by @yaoyu-33 :: PR: #4891
test(recipes): validate all recipe factories by @yaoyu-33 :: PR: #4912
build(docker): make nvshmem reinstall optional in Dockerfile.ci by @thomasdhc :: PR: #4901
test(model): stabilize Gemma-1 activation roundtrip by @yaoyu-33 :: PR: #4927
ci(coverage): exclude examples and scripts by @yaoyu-33 :: PR: #4915
[docs] fix: Align async inference examples with Bridge CLI by @yaoyu-33 :: PR: #4919
[model] fix: Preserve config-only Gemma 3 provider creation by @yaoyu-33 :: PR: #4918
[doc] fix: Update GLM-4.5 recipe examples by @yaoyu-33 :: PR: #4921
[data] fix: respect drop_last for iteration training by @yaoyu-33 :: PR: #4926
feat(ckpt): add roundtrip conversion launcher by @yaoyu-33 :: PR: #4886
ci: AUT-897 raise UV_HTTP_TIMEOUT to 120s in CI container build by @svcnemo-autobot :: PR: #4930
fix(model): align Gemma 4 MoE global RoPE with Hugging Face by @yaoyu-33 :: PR: #4916
[docs] docs: Add LongStraw ecosystem news by @yaoyu-33 :: PR: #4933
chore: Set fastapi < 0.139.2 by @chtruong814 :: PR: #4935
Adding FSDP/HSDP example for nemotron3 nano by @gautham-kollu :: PR: #4745
[misc] fix: Make standalone top-p sampling usable by @yaoyu-33 :: PR: #4937
fix(recipe): propagate GLM-4.5V Slurm failures by @yaoyu-33 :: PR: #4938
[training] fix: avoid duplicate MIMO evaluation timer by @yaoyu-33 :: PR: #4939
[data] fix: reject unsafe custom batch padding by @yaoyu-33 :: PR: #4943
[training, ckpt] fix: Resolve default in-process restart world size by @yaoyu-33 :: PR: #4944
feat(training): route performance recipes by name by @yaoyu-33 :: PR: #4945
chore(deps): AUT-901 bump Megatron-LM to mcore main b4ad280d by @svcnemo-autobot :: PR: #4948
[training] fix: Close TensorBoard writer on restart by @yaoyu-33 :: PR: #4951
[docs] docs: Mention Macaron-V1 in LongStraw news by @yaoyu-33 :: PR: #4958
chore: AUT-905 bump Megatron-Core (main) to 3219f38 by @svcnemo-autobot :: PR: #4956
[misc] fix: reject unsupported legacy stopping controls by @yaoyu-33 :: PR: #4954
Hsdp Config by @gautham-kollu :: PR: #4746
build(deps): upgrade Transformers to 5.12.1 by @cuichenx :: PR: #4928
feat(recipes): support recipe environment variables by @yaoyu-33 :: PR: #4682
fix: preserve Qwen3 routing and packed loss masks by @yaoyu-33 :: PR: #4946
fix(ckpt): reduce distributed optimizer resume memory by @cuichenx :: PR: #4936
[recipes] fix: Restore DeepSeek HSDP recipe environment by @yaoyu-33 :: PR: #4972
fix(perf): read legacy SQuAD cache columns by @yaoyu-33 :: PR: #4970
Add CLI configurability of num_moe_experts by @scsudhakaran :: PR: #4968
revert: AUT-927 "Migrate GPT-OSS to HybridModel (#4476)" by @svcnemo-autobot :: PR: #4982
[recipe] fix: propagate GPT-OSS pretraining failures by @yaoyu-33 :: PR: #4984
[training] fix: Create MIMO embedding groups in global order by @yaoyu-33 :: PR: #4985
[ckpt] fix: Dispatch MIMO loads through checkpoint manager by @yaoyu-33 :: PR: #4981
[ckpt] fix: Clear saved pipeline layout for single-rank load by @yaoyu-33 :: PR: #4992
[inference] fix: Seed VLM sampling generator by @yaoyu-33 :: PR: #4993
[ckpt, data] feat: complete Energon dataloader checkpointing save/res… by @zsombor-kaiko :: PR: #4521
ci: AUT-895 gate GB200 tests on DISABLE_GB200_TESTS variable by @svcnemo-autobot :: PR: #4929
docs: fix broken links in Megatron-Bridge docs (NVBug 6421302) by @chenopis :: PR: #4969
fix(data): retry packed-parquet resolution with NFS directory-metadata refresh (#4207) by @janbernloehr :: PR: #4865
fix(model): correct Qwen3-VL vision_dp_when_cp CP gradient and avoid 0-image-rank hang by @going-song :: PR: #4784
build: AUT-944 bump Megatron-Core main pin to 58bf14e by @svcnemo-autobot :: PR: #4991
ci: AUT-945 seed a PR's container build from its own buildcache by @svcnemo-autobot :: PR: #4995
feat(training): launch canonical performance recipes by @yaoyu-33 :: PR: #4620
26.06.01 perf summary by @malay-nagda :: PR: #4989
[recipe, model] fix: resolve 26.08 model workflow regressions by @yaoyu-33 :: PR: #4994
feat(model): add model support verification cards by @yaoyu-33 :: PR: #4909
fix(training): align comm-overlap validation with Megatron-Core by @cuichenx :: PR: #4971
test(ckpt): isolate conversion launcher utility imports by @yaoyu-33 :: PR: #5003
build(deps): provide transformer-engine metadata so uv lock skips its source build by @ko3n1g :: PR: #5018
docs(model): verify H100 checkpoint resumes by @yaoyu-33 :: PR: #5009
[docs] fix: Include model verification skill in docs index by @yaoyu-33 :: PR: #5026
[training] fix: align ModelOpt restore with resume source by @yaoyu-33 :: PR: #5014
[training] fix: Clean up failed pretrain sessions by @yaoyu-33 :: PR: #5013
[data] fix: Handle Qwen3-Next custom chat data by @yaoyu-33 :: PR: #5012
[recipe] fix: Resync Kimi pipeline layout after overrides by @yaoyu-33 :: PR: #5008
feat(conversion): add HF embedding row repair script by @yaoyu-33 :: PR: #5005
docs(model): list verification index items explicitly by @yaoyu-33 :: PR: #5029
fix(perf): keep recipe imports off login nodes by @yaoyu-33 :: PR: #5032
fix(model): use TP=1 for GLM-5 inference by @yaoyu-33 :: PR: #4999
[perf] feat: add Qwen3 and DeepSeek-V3 MoE recipes by @dimapihtar :: PR: #4850
remove fine-tuning references by @malay-nagda :: PR: #5024
fix(perf): reinstate aws-ofi-nccl EFA plugin for EKS by @ko3n1g :: PR: #5033
Update DeepSeek-V3 B300 recipe by @scsudhakaran :: PR: #5015
build(deps): AUT-960 bump Megatron-Core main to 6cd6ea53 by @svcnemo-autobot :: PR: #5044
add fixed mem clk rubin by @malay-nagda :: PR: #4920
feat(training): add VR200 memory clock lock by @yaoyu-33 :: PR: #5049
fix(training): include Comet in logger presence gate by @yaoyu-33 :: PR: #5041
fix(recipe): stop enabling cuDNN norm by default by @yaoyu-33 :: PR: #5040
[model] fix: preserve Nemotron runtime config by @cuichenx :: PR: #4974
fix(recipe): restore Nemotron VL SFT batches by @cuichenx :: PR: #5000
fix(data): support GLM-4.5V assistant masking by @cuichenx :: PR: #5001
perf(recipe): optimize Moonlight training throughput by @yaoyu-33 :: PR: #5021
feat(recipes): add Qwen3.5 and Nemotron 3 Nano GB200 pretraining by @cuichenx :: PR: #4854
feat: add Slurm inference launcher by @cuichenx :: PR: #4931
chore(deps): AUT-952 bump Megatron-Core dev ref by @svcnemo-autobot :: PR: #5019
chore(model): deprecate legacy model support by @yaoyu-33 :: PR: #5025
feat(model): add MiniMax-M3 text-only verification by @yaoyu-33 :: PR: #5002
test(recipes): drop brittle count asserts from recipe env invariants by @cuichenx :: PR: #5056
test(recipes): stop pinning exact recipe count in env invariant by @cuichenx :: PR: #5057
fix(data): shuffle direct HF SFT with cyclic sampler by @yaoyu-33 :: PR: #5048
[training] feat: add scheduler max steps for shortened tests by @cuichenx :: PR: #4979
fix(model): improve conversion and inference examples by @yaoyu-33 :: PR: #4805
feat(tokenizers): support loading a chat template from a file by @omri-manolo :: PR: #4899
fix(model): forward Qwen3-VL output processor hooks by @yaoyu-33 :: PR: #5060
fix(skill): require one inference run by @yaoyu-33 :: PR: #5063
[test] fix: remove recipe inventory count assertions by @yaoyu-33 :: PR: #5064
feat (recipe): add Nemotron 3 Ultra pretraining perf-recipe by @ghadiaravi13 :: PR: #4911
fix(docker): MB-937 guard Torch op cleanup during shutdown by @svcnemo-autobot :: PR: #5046
fix(perf): MB-923 support recipes without environment maps by @svcnemo-autobot :: PR: #5043
fix(data): normalize ShareGPT role aliases by @yaoyu-33 :: PR: #5078
[training] fix: account for uneven PP in memory estimate by @yaoyu-33 :: PR: #5077
[inference] fix: Handle variable-length VLM batches by @yaoyu-33 :: PR: #5072
fix(model): honor plain export dtype for Kimi K2.5 by @yaoyu-33 :: PR: #5069
feat(recipe): add Qwen3 30B GB200 MXFP8 verification by @yaoyu-33 :: PR: #5066
build(deps): AUT-960 bump Megatron-Core dev to 0afb41b9 by @svcnemo-autobot :: PR: #5045
fix(perf): MB-923 align mounted scripts with bridge source by @svcnemo-autobot :: PR: #5079
chore(misc): remove legacy evaluation examples by @cuichenx :: PR: #5059
Add notes and instructions for dynamic/non-dynamic resolution path for Nemotron-Omni by @huvunvidia :: PR: #5053
[recipe, perf] feat: port r0.5.0 performance recipes by @yaoyu-33 :: PR: #5061
feat(model): accept pre-sharded packed Qwen3-VL inputs by @yaoyu-33 :: PR: #4917
fix(models): verify Gemma 4 VL BF16 conversion by @yaoyu-33 :: PR: #5036
feat(model): add MiniMax-M3 VLM import and export by @yaoyu-33 :: PR: #5058
fix(flops): use parquet packed dataset for LoRA SQuAD FLOP accounting by @rsalagame-nvidia :: PR: #5010
fix(data): use Nemotron Ultra recipe defaults by @cuichenx :: PR: #5082
[training] fix: Respect disabled exit signal handling by @yaoyu-33 :: PR: #5068
perf(recipe): overlap Qwen3 expert communication on H100 by @yaoyu-33 :: PR: #5094
[misc] fix: Preserve chat templates in inference tokenizer by @yaoyu-33 :: PR: #5091
[recipe] fix: Correct Ministral 3 SFT learning rates by @yaoyu-33 :: PR: #5092
[ckpt] fix: select newest global recovery root by @yaoyu-33 :: PR: #5090
Add EXAONE 4.5 & K-EXAONE bridge by @sangHa0411 :: PR: #4730
[data] fix: build memmap indexes on every node by @yaoyu-33 :: PR: #5107
[ckpt] fix: ignore FSDP RNG after layout changes by @yaoyu-33 :: PR: #5100
[inference] fix: Honor configured Qwen VLM visual token IDs by @yaoyu-33 :: PR: #5101
[doc] fix: update Moonlight recipe examples by @yaoyu-33 :: PR: #5103
[training] fix: Preserve flat Hugging Face configs by @yaoyu-33 :: PR: #5106
chore(deps): AUT-1076 bump MCore main by @svcnemo-autobot :: PR: #5114
[model] fix: Restore callable Mamba stack specs by @yaoyu-33 :: PR: #5116
[doc] fix: update OLMoE recipe examples by @yaoyu-33 :: PR: #5118
fix(training): abort async checkpoints after pretrain failure by @yaoyu-33 :: PR: #5117
[data] fix: accept decoded Nemotron video inputs by @yaoyu-33 :: PR: #5121
build(deps): AUT-969 update Transformer Engine to 2.18.0 by @svcnemo-autobot :: PR: #5054
[training] fix: Honor logging environment override by @yaoyu-33 :: PR: #5120
[model] fix: honor provider precision overrides by @yaoyu-33 :: PR: #5122
[model] fix: preserve linear Llama RoPE scaling by @yaoyu-33 :: PR: #5097
docs(model-card): verify MiniMax-M3 VLM by @yaoyu-33 :: PR: #5065
[recipes] feat: add MTP support to Nemotron 3 Nano by @cuichenx :: PR: #5035
feat(recipe): add DSv4-Flash GB200 pretrain recipes (BF16/MXFP8/Muon) by @weijiac0619 :: PR: #5127
chore(model): deprecate additional legacy model support by @yaoyu-33 :: PR: #5125
test(mimo): restore bias-free MoE training smoke by @liding-nv :: PR: #5115
Add SequentialMLP mappings for MTP heads in nemotron_h_bridge.py (5140) by @svcnvidia-nemo-ci :: PR: #5152
[test] fix: repair Nemotron-H CI regressions (5143) by @svcnvidia-nemo-ci :: PR: #5146
cp: Nemotron 3.5 Nano verification card and final-checkpoint refresh (#5124, #5199) into r0.6.0 by @svcnvidia-nemo-ci :: PR: #5178
fix(docker): update dependencies in Dockerfile and fw_pyproject.toml for compatibility and security (5202) by @svcnvidia-nemo-ci :: PR: #5208
fix(model): map GLM-5 routed expert count (5220) by @svcnvidia-nemo-ci :: PR: #5248
feat(sft): DSv4 Flash offline-packed SFT with THD + CP (5126) by @svcnvidia-nemo-ci :: PR: #5247
[Doc] Document latest Mcore toggle moe_hybridep_pad_uneven_dispatch_inputs for HybridEP THD training (4213) by @svcnvidia-nemo-ci :: PR: #5159
fix(perf): port r0.5.0 VR200 performance configs (5196) by @svcnvidia-nemo-ci :: PR: #5197
feat(misc): add memory snapshot analysis skill (5150) by @svcnvidia-nemo-ci :: PR: #5198
fix(model): add Gemma 3 VL verification coverage (5084) by @svcnvidia-nemo-ci :: PR: #5204
fix(training): preserve alltoall flex fallback (5207) by @svcnvidia-nemo-ci :: PR: #5214
fix(model): align MiniMax evidence and inference task routing (5194) by @svcnvidia-nemo-ci :: PR: #5235
fix(peft): sync grouped expert LoRA over expert DP (5231) by @svcnvidia-nemo-ci :: PR: #5269
chore(deps): update mlflow version to 3.15.0 (5267) by @svcnvidia-nemo-ci :: PR: #5275
chore: bump mcore dev commit for DSv4 support by @weijiac0619 :: PR: #5284
fix(recipes): derive pretraining vocab from tokenizer (4996) by @cuichenx :: PR: #5280
fix(data): preserve VLM pixel defaults (5218) by @svcnvidia-nemo-ci :: PR: #5271
fix(data): reject truncation without supervision (5217) by @svcnvidia-nemo-ci :: PR: #5272
fix(data): normalize GLM VLM assistant content (5216) by @svcnvidia-nemo-ci :: PR: #5273
fix(data): pad Qwen VLM token type IDs (5215) by @svcnvidia-nemo-ci :: PR: #5274
perf(recipe): tune Nemotron 3.5 Nano H100 benchmark (5266) by @svcnvidia-nemo-ci :: PR: #5279
fix(training): restore weak scaling for benchmark recipes (5222) by @svcnvidia-nemo-ci :: PR: #5282
feat (recipe): add Nemotron 3 Ultra MXFP8 GB200 pretraining perf-recipe (5287) by @svcnvidia-nemo-ci :: PR: #5309
fix(training): count expert parameter norms with TP and EP (4998) by @svcnvidia-nemo-ci :: PR: #5177
fix(model): restore DeepSeek V3 sequence auxiliary loss (5165) by @svcnvidia-nemo-ci :: PR: #5172
cp: Nemotron 3.5 Lightning checkpoint and naming updates (#5293, #5303) into r0.6.0 by @svcnvidia-nemo-ci :: PR: #5300
fix(training): backport HybridEP uneven-dispatch compatibility to r0.6.0 by @cuichenx :: PR: #5288
perf(recipe): add DeepSeek V4 Flash GB300 FP8MX recipes by @malay-nagda :: PR: #5258
build: bump nvidia-modelopt to 0.46.0rc1 (5331) by @svcnvidia-nemo-ci :: PR: #5338
chore(docker): update Dockerfile and install script to remove unnecessary packages by @balasaajay :: PR: #5330
build: AUT-1353 bump transformer-engine to 2.17.1 by @svcnemo-autobot :: PR: #5349
build: AUT-1346 pin wandb to 0.27.2 by @svcnemo-autobot :: PR: #5348
fix(ci): AUT-1359 skip humming module in import check by @svcnemo-autobot :: PR: #5358
fix(model): AUT-1326 backport Gemma 3 VL fused attention to r0.6.0 by @svcnemo-autobot :: PR: #5336
fix(recipe): backport #5386 batch-size alignment to r0.6.0 by @cuichenx :: PR: #5393
fix(perf): mount common performance scripts (5381) by @svcnvidia-nemo-ci :: PR: #5404
chore(deps): update mlflow to 3.15.1 by @balasaajay :: PR: #5419
feat(model): add GLM-5.2 verification card (5034) by @svcnvidia-nemo-ci :: PR: #5301
fix(ckpt): validate conversion world size against TPPP, not TPPP*EP (5302) by @svcnvidia-nemo-ci :: PR: #5312
[docs] fix: point Gemma 4 VL inference to verification card (5311) by @svcnvidia-nemo-ci :: PR: #5319
fix(model): enable RADIO recompute for Nemotron Omni SFT by @cuichenx :: PR: #5320
[model] fix: honor Gemma 4 image attention mode (5333) by @svcnvidia-nemo-ci :: PR: #5350
[ckpt] fix: support vocab padding with Megatron-FSDP TP (5353) by @svcnvidia-nemo-ci :: PR: #5359
fix(ckpt): avoid roundtrip export barrier timeout (5377) by @svcnvidia-nemo-ci :: PR: #5412
fix(peft): preserve grouped expert SwiGLU LoRA order (5376) by @svcnvidia-nemo-ci :: PR: #5414
fix(training): include MTP stages in embedding groups (5380) by @yaoyu-33 :: PR: #5422
fix: repair release QA regressions (5351) by @yaoyu-33 :: PR: #5423
[models, conversion] fix: verify Step-3.5 model workflows (5334) by @yaoyu-33 :: PR: #5424
fix(training): surface distributed failures without cleanup hangs (5283) by @svcnvidia-nemo-ci :: PR: #5327
fix(modelopt): handle sparse export task lists (5374) by @svcnvidia-nemo-ci :: PR: #5446
fix(ckpt): restrict dataloader state deserialization (5418) by @svcnvidia-nemo-ci :: PR: #5437
fix(recipes): use max-pool FSDP for Qwen3.5-VL (5402) by @svcnvidia-nemo-ci :: PR: #5430
fix(recipe): verify GLM-5.2 RP2 pretraining and packed SFT (5344) by @yaoyu-33 :: PR: #5425
[examples] refactor: direct GLM inference to model card (5346) by @yaoyu-33 :: PR: #5427
docs(models): link MiMo and StepFun examples (5432) by @svcnvidia-nemo-ci :: PR: #5439
docs(recipe): retire stale Gemma 4 VL SFT launchers (5433) by @svcnvidia-nemo-ci :: PR: #5435
fix(peft): synchronize shared expert LoRA across EP (5396) by @svcnvidia-nemo-ci :: PR: #5428
fix(model): preserve Step3.7 text config fields (5417) by @svcnvidia-nemo-ci :: PR: #5429
[ckpt] fix: Deinterleave low-memory GLU checkpoints (5366) by @svcnvidia-nemo-ci :: PR: #5370
Update wandb dependency version to 0.27.2 by @balasaajay :: PR: #5461
fix(model): backport Gemma4 PLE sequence parallel sharding to r0.6.0 by @cuichenx :: PR: #5450
fix(data): collate ragged packed sequence metadata (5400) by @yaoyu-33 :: PR: #5426
build: AUT-1430 upgrade GitPython to 3.1.58 by @svcnemo-autobot :: PR: #5480
fix(models): use HybridEP for MiniMax-M3 expert parallelism (5448) by @svcnvidia-nemo-ci :: PR: #5473
fix(diffusion): calculate WAN FLOPs from runtime geometry (5249) by @svcnvidia-nemo-ci :: PR: #5477
feat(data): infer assistant loss mask for gpt-oss Harmony chat template (5389) by @svcnvidia-nemo-ci :: PR: #5447
fix(scripts): preserve FHT and fetch missing MCore commits (5478) by @svcnvidia-nemo-ci :: PR: #5481
fix(models): use HybridEP for Kimi-K2.5 inference (5421) by @cuichenx :: PR: #5485
fix(model): handle disabled GLM-5 MTP export (5445) by @svcnvidia-nemo-ci :: PR: #5488
docs(ckpt): document roundtrip scaling limits (5491) by @svcnvidia-nemo-ci :: PR: #5492
docs(model-card): add Nemotron 3 Super verification card (5062) by @svcnvidia-nemo-ci :: PR: #5494
docs: update performance summary for 26.08 (5496) by @svcnvidia-nemo-ci :: PR: #5507
fix(deepseek-v4): use full-prefix inference (5509) by @svcnvidia-nemo-ci :: PR: #5511
fix(recipe): prevent Qwen3.5-VL FSDP SFT OOM (5508) by @svcnvidia-nemo-ci :: PR: #5515
fix(recipe): recompute Gemma 4 VL SFT layers by @yaoyu-33 :: PR: #5506
docs(dsv4): preserve container packages during MCore sync (5514) by @svcnvidia-nemo-ci :: PR: #5516
Update nemo-evaluator path in fw_pyproject.toml by @balasaajay :: PR: #5571
docs: Update 0.6.0 version entries for docs by @balasaajay :: PR: #5567
fix(deepseek): default DeepSeek V4 to HybridEP (5543) by @svcnvidia-nemo-ci :: PR: #5582
docs: fix preferred version and add 0.5.1 to the version picker by @ko3n1g :: PR: #5623
beep boop 🤖: Bumping Megatron-Bridge to v0.6.1 by @nemo-automation-bot[bot] :: PR: #5697
[model] fix: Support direct Mamba conv params (#4025) by @wujingyue
[model/training] fix: handle config edge cases (#4104) by @yaoyu-33
Add ModelOpt NVFP4 HF weight export support (#4102) by @HollowMan6
perf(llama): auto-disable PCT binding on B300 for llama pretrain (#4081) by @rsalagame-nvidia
chore(beep boop 🤖): Bump (main, mcore-main) (2026-06-02) by @github-actions[bot]
feat(distillation): add student_pre_conversion_hook to DistillationProvider (#4114) by @kevalmorabia97
Bump MINOR version from 5 to 6 (#4106) by @balasaajay
fix(mimo): restore optimizer state and enable RNG save on MIMO + GLOBAL checkpoint resume (#3832) by @kamran-nvidia
Changes to support EP reshardable per-expert LoRA (#3951) by @jstjohn
chore(skills): add eval datasets part 1 (#4116) by @cuichenx
chore(skills): add eval datasets part 2 (#4117) by @cuichenx
Add Qwen3.5-VL MegatronMIMO examples (#4071) by @liding-nv
Forward dense grouped GEMM spec kwarg (#4122) by @sraman-rgb
chore(beep boop 🤖): Bump (main, mcore-main) (2026-06-03) by @github-actions[bot]
[training] fix: Update DeepSeek-V4 FLOPs calculation (#4128) by @cuichenx
[feature] Add dense CP support for Qwen3-Omni (#3915) by @hbhflw2000
[training, ckpt] feat: Save HF weights alongside native Megatron checkpoints (#3825) by @dancingpipi
rename the architecture class of NemotronLabsDiffusion. (#4119) by @sajadn
build: bump pillow to 12.2.0 (#4151) by @ko3n1g
Sync overlap_param_gather between DDP and optimizer (#4141) by @WanZzzzzz
docs: announce NVIDIA Nemotron 3 Ultra public release (#4159) by @cuichenx
chore: bump _code_freeze workflow to v1.4.2 (#4132) by @ko3n1g
ci: bump _release_library.yml to v1.4.3 (#4137) by @ko3n1g
[ci, perf] feat: add lease-aware kubeflow executor wiring + drop dgxc executor (#4061) by @ko3n1g
fix(perf): re-land canonical 'rajpurkar/squad' HF dataset id (#4150) by @ko3n1g
[docs] feat: Add PR review skill (#4142) by @yaoyu-33
Fix cudagraph for gptoss 20b fp8 recipes (#4140) by @nv-mollys
perf: Allow TE cudnn norm with gpt_oss_20b (#4175) by @jepio
refactor(data): unify VLM processing helpers (#4109) by @yaoyu-33
build: add managed = true to [tool.uv] (#4176) by @kajalj22
feat(data): add c4 dataset option to perf launcher (#4118) by @dingqingy-nv
chore(beep boop 🤖): Bump (main, mcore-main) (2026-06-06) by @github-actions[bot]
[perf, recipe] feat: DSV3 GB300 MXFP8 full-iter CG recipe (#3983) by @dingqingy-nv
docs(skills): include review-pr skill in toctree (#4186) by @yaoyu-33
refactor(data): move VLM collators to model modules (#4166) by @yaoyu-33
chore(beep boop 🤖): Bump (main, mcore-main) (2026-06-07) by @github-actions[bot]
Move examples/diffusion into examples/models (#4145) by @cuichenx
docs(diffusion): fix stale paths after diffusion->models move (#4194) by @cuichenx
add llada1.5 (#4019) by @linnanwang
chore(beep boop 🤖): Bump (main, mcore-main) (2026-06-08) by @github-actions[bot]
fix(model): normalize Step35 MTP layer types (#4167) by @yaoyu-33
fix(nemotronh): export unified MTP hybrid patterns (#4129) by @yaoyu-33
[training] fix: Route batches to standalone MTP stages (#4208) by @yaoyu-33
chore(beep boop 🤖): Bump (main, mcore-main) (2026-06-09) by @github-actions[bot]
ci: smoke-import installed packages in docker build stages (#4233) by @ko3n1g
Allow mtp_num_layers to be None (#4216) by @yfw
[training] fix: install OS signal trap by entering DistributedSignalHandler (#3823) by @rob-luke
fix(training): guard optional nvrx_straggler import against non-import errors (#4124) by @cuichenx
full-iter CG for GPT-OSS 120B (#4173) by @malay-nagda
fix: Fix Gemma3-VL HF↔Megatron conversion for transformers ≥ 5.8 (#4220) by @suiyoubi
ci: move venv import check from Dockerfile RUN to GH Actions job (#4240) by @ko3n1g
[model] fix: guard GLM fused expert detection (#4212) by @yaoyu-33
Add Qwen3.5-VL MegatronMIMO non-colocated SFT tutorial (#4239) by @liding-nv
fix NemotronLabsDiffusionBridge inference bug (#4179) by @sajadn
[model] fix: Repair Nemotron conversion regressions (#4049) by @yaoyu-33
build(deps): exclude CVE codecs (av/imageio/imageio-ffmpeg) from the image (#4229) by @ko3n1g
[test] chore: Re-enable Nemotron-H functional launchers (#4252) by @yaoyu-33
[docs] docs: Clarify Quick Start model source workflows (#4254) by @yaoyu-33
docs(training): clarify workflow and checkpoint guidance (#4259) by @yaoyu-33
Add shared outer lora (#3958) by @nanjiangwill
feat(recipe): DSV3 GB200 MXFP8 full-iter CG recipe (#4226) by @dingqingy-nv
[build] chore: uninstall nvidia-modelopt from fw_base (#4267) by @ko3n1g
build: bump GitPython to 3.1.50 to address CVE-2026-42215 bypass (#4280) by @ko3n1g
fix(training): route MTP inputs from model chunk (#4261) by @yaoyu-33
docs(skills): fix agentic readiness paths (#4260) by @yaoyu-33
docs(data): use energon prepare for valor32k-avqa dataset build (#4219) by @cuichenx
fix(model): keep full rotary_percent for DeepSeek-V4 MLA rope (#4271) by @Meirtz
[model] fix: use flash attention for Kimi vision tower (#4257) by @yaoyu-33
[model] fix: Support partial Nemotron-VL specs (#4070) by @cuichenx
[model] fix: Default Bailing MoE dispatcher to alltoall (#4256) by @cuichenx
Enable Qwen3 GB200/GB300 MXFP8 CuTeDSL Fusion and Full-Iter CUDA Graphs (#4211) by @rhmukundan
fix(checkpointing): include optimizer scaffold while loading (#4222) by @cuichenx
feat(scripts): add Ultra script prerequisites (#4223) by @cuichenx
fix(data): route HF VLM sequence packing by collate support (#4255) by @yaoyu-33
build: bump go-git/v5 to 5.19.0 in dist-packages wandb-core (#4289) by @ko3n1g
fix(docker): rename NEMO_EVAL_COMMIT to NEMO_EVALUATOR_COMMIT (#4291) by @ko3n1g
[model] fix: Align Gemma3 VL vision tower mapping (#4278) by @yaoyu-33
[training] fix: Stabilize NVRx straggler tests (#4279) by @yaoyu-33
FSDP GB300 DS-V3 Config with (#3703) by @gautham-kollu
perf(llama): enable TE CUDA Graphs (mlp,attn) for 70B GB200 NVFP4 V2 (#4080) by @rsalagame-nvidia
feat(recipe): add DeepSeek-V4-Flash SFT recipes, launcher and tests (#4131) by @Meirtz
fix(gemma4): map dense MLP pre-FFN norm to pre_feedforward_layernorm (#4201) by @rayandasoriya
[misc] feat: Add NVTX ranges for train_step and optimizer_step (#3198) by @minitu
chore(beep boop 🤖): Bump uv.lock (main, mcore-main) (2026-06-11) (#4302) by @svcnvidia-nemo-ci
[test] perf: build a tiny toy checkpoint for the nemotron_omni conversion test (#4287) by @cuichenx
[tutorial] feat: add MoE notebook example (#3890) by @karinseve
[data] fix: pad loss_mask and handle tensors in packed-sequence chat … (#4056) by @JRD971000
[data] fix: Handle empty PP-rank finetuning data (#4258) by @cuichenx
fix(recipe): add missing kimi package init.py and top-level import (#4293) by @nvegesna-netizen
[examples] fix: Correct GLM-4.5V fine-tuning scripts (#4310) by @cuichenx
Update Qwen3-Next recipe (#4283) by @maanug-nv
feat(recipe): enable rope fusion in DeepSeek-V4-Flash SFT recipes (#4305) by @Meirtz
chore(beep boop 🤖): Bump (main, mcore-main) (2026-06-12) by @github-actions[bot]
Disable PCT binding for Kimi in B300 (#4316) by @bhargaveede
[examples] fix: Make Qwen2-Audio SFT runnable by default (#4313) by @cuichenx
docs(deepseek): update H100 Flash parallelism (#4334) by @cuichenx
fix(ckpt): auto-generate DSv4 pipeline layout on export when run_config lacks it (#4306) by @Meirtz
Fix nemotronLabsDiffusion checkpoint conversion (#4317) by @sajadn
Add functional support matrix to README (#4333) by @snowmanwwg
chore(beep boop 🤖): Bump (main, mcore-main) (2026-06-13) by @github-actions[bot]
fix(deepseek): gate H100 fused kernel defaults (#4338) by @cuichenx
fix(gemma4-vl): avoid checkpoint save timeout (#4355) by @cuichenx
[training, perf] fix: THD-aware FLOPS via cu_seqlens (Σᵢ sᵢ²) (#3839) by @cuichenx
Revert "[training, perf] fix: THD-aware FLOPS via cu_seqlens (Σᵢ sᵢ²) (#3839)" (#4363) by @cuichenx
26.06 perf summary (#4318) by @malay-nagda
Clarify Qwen3.5-VL MIMO tutorial terminology and baseline choice (#4364) by @liding-nv
feat(conversion): support distributed adapter export (#4221) by @cuichenx
docs: trigger NVSkills CI for perf skills (#4360) by @cuichenx
fix(model): fix Falcon H1 static inference (#4367) by @cuichenx
docs: fix performance archive footnote reference (#4372) by @cuichenx
[test] fix: repair CP packed SFT functional setup (#4374) by @yaoyu-33
fix: honor attention backend for checkpoint inference (#4370) by @cuichenx
fix(ckpt): repair DeepSeek V4 conversion script (#4369) by @cuichenx
fix(recipe): use current Nemotron Nano VL finetune configs (#4362) by @cuichenx
[model] fix: pass Step35 layer names through MTP (#4368) by @yaoyu-33
[model] fix: pass TP group to VLM sequence scatter (#4371) by @yaoyu-33
fix(qwen3-asr): preserve audio config for checkpoint export (#4361) by @yaoyu-33
[test] fix: shrink CP packed SFT functional model (#4383) by @yaoyu-33
[test] fix: register internal pytest marker (#4359) by @yaoyu-33
feat(training): expose distributed_timeout_minutes in perf launcher (#4385) by @ko3n1g
chore(beep boop 🤖): Bump (main, mcore-main) (2026-06-16) by @github-actions[bot]
[docs] docs: Add MLPerf Training 6.0 news (#4393) by @yaoyu-33
[docs] docs: Link MLPerf news to DeepSeek performance recipes (#4394) by @yaoyu-33
26.06 perf summary updates (#4384) by @malay-nagda
[training, perf] fix: THD-aware FLOPS via cu_seqlens (Σᵢ sᵢ²) (re-land of #3839) (#4366) by @cuichenx
chore(beep boop 🤖): Bump (main, mcore-main) (2026-06-17) by @github-actions[bot]
feat(model): add Gemma-4 E4B support (layer spec, checkpoint loader, parity check) (#4148) by @DOGEUNNKIM
chore(beep boop 🤖): Bump uv.lock (main, mcore-main) (2026-06-19) (#4416) by @svcnvidia-nemo-ci
feat: Add EXAONE 4.0 model bridge (LG AI Research) (#2532) by @Bias92
fix: Detach the params to avoid slow down training (#1777) by @moehanabi
[ckpt, model] fix: Harden config and shard deserialization (#4008) by @yaoyu-33
fix(kimi-vl): require trusted remote code for vision modules (#4422) by @yaoyu-33
chore(beep boop 🤖): Bump (main, mcore-main) (2026-06-20) by @github-actions[bot]
[model] fix: apply attention_value_scale to V in MiMo-V2-Flash (#4155) by @Eisenhower
[diffusion] fix: make select_samples_to_pack shuffle deterministic across resume (#4237) by @nayopu
fix: save_hf_weights drops boundary shards for MTP-less GLM-4.x glm4_moe_lite export (#4189) by @dinhxuanvu
chore(beep boop 🤖): Bump (main, mcore-main) (2026-06-21) by @github-actions[bot]
feat(model): thread weight_dtype through HF export for plain-dtype DeepSeek-V4 output (#4301) by @Meirtz
b300 regression fix (#4436) by @malay-nagda
perf summary b300 26.06 (#4435) by @malay-nagda
chore(beep boop 🤖): Bump uv.lock (main, mcore-main) (2026-06-22) (#4438) by @svcnvidia-nemo-ci
docs: Add initial fern migration (#4325) by @chtruong814
feat(training): return eval losses and add pure eval example (#4125) by @yaoyu-33
test(functional): split training launch scripts (#4444) by @yaoyu-33
docs: Add 26.06 software components and known issues (#4453) by @chtruong814
[ci] test: Move functional model tests under test groups (#4443) by @yaoyu-33
docs(readme): announce Megatron Bridge 0.5.0 (#4455) by @yaoyu-33
docs: Fix nemo run docs link (#4456) by @chtruong814
refactor(docker): Dockerfile configurations and enhance environment variable management (#4288) by @balasaajay
Revert "[training, perf] fix: THD-aware FLOPS via cu_seqlens (Σᵢ sᵢ²) (re-land of #3839)" (#4464) by @malay-nagda
chore(beep boop 🤖): Bump (main, mcore-main) (2026-06-23) by @github-actions[bot]
ci: downsample persisted golden values for long-convergence runs (#4465) by @ko3n1g
docs: fix broken README model links breaking docs build (#4468) by @ko3n1g
feat(nemotronh): add Nemotron 3 Ultra recipes and examples (#4224) by @cuichenx
test(functional): recategorize launch scripts (#4470) by @yaoyu-33
fix: harden dynamic config targets (#4451) by @chtruong814
fix(data): anchor VLM assistant loss masks to chat templates (#4169) by @yaoyu-33
Refactor: rename Mamba model API to Hybrid (#4449) by @Phlip79
[ci, perf] fix: retry on NCCL collective-timeout hangs and scan all rank logs (#4480) by @ko3n1g
fix(data): add Ministral3 assistant mask boundaries (#4477) by @yaoyu-33
[ci] fix: resolve test matrix from the PR merge commit, not the head (#4486) by @ko3n1g
feat(data): add text chat collate for unified HF datasets (#4307) by @yaoyu-33
build: bump nvidia-cutlass-dsl to 4.5.0 and nvidia-cudnn-frontend to 1.24.0 (#4466) by @ko3n1g
docs: revise Qwen3.5 MegatronMIMO tutorial (#4448) by @liding-nv
Import upstreamed GPT config+builder (#4472) by @maanug-nv
fix(tests): increase LoRA finetune data rows (#4493) by @yaoyu-33
[model] fix: use hf_config to check whether model is dense (#4414) by @pavelgein
fix(docs): remove duplicate VLM processing helpers (#4494) by @yaoyu-33
Update DSV3 B300 perf recipe (#4458) by @scsudhakaran
Support MegatronMIMO encoder DP slicing for packed visual input (#4441) by @liding-nv
Add GLM5 MTP mapping support (#4197) by @HollowMan6
Add weight and bias property for LoRALinear (#4198) by @HollowMan6
refactor(inference): decouple inference scripts from the Megatron-LM reference layer (#4475) by @cuichenx
fix(data): preserve Nemotron Omni packed sequence metadata (#4400) by @cuichenx
[Training] Fix packed THD CP slicing (#4489) by @cuichenx
refactor(data): move VLM sequence batching to collate (#4315) by @yaoyu-33
[tests] fix: Size LoRA resume SFT fixture (#4505) by @yaoyu-33
[model, test] fix: handle MCore main bump (#4503) by @yaoyu-33
fix+feat(gemma2): fix SWA correctness bugs and add FlexAttention fused softcap+SWA path (#4308) by @nvegesna-netizen
fix(data): preserve Kimi thinking in VLM collate (#4478) by @yaoyu-33
[training, ckpt] fix: Sync HybridDeviceOptimizer FP32 master copies after checkpoint load (#4082) by @cuichenx
[recipe][perf] feat: unified flat performance recipes for all model families (#2803) by @yaoyu-33
[data, tests] fix: stabilize LoRA packed finetune (#4516) by @yaoyu-33
[data] test: Stabilize HF VLM masking check (#4526) by @yaoyu-33
feat(model): add sequential (non-grouped) MoE expert mappings (#4499) by @kevalmorabia97
fix(training): handle packed SFT CP masks (#4519) by @cuichenx
[build] chore: bump transformer-engine to release_v2.16_post (#4527) by @ko3n1g
fix: Remove dali from docker container (#4500) by @chtruong814
fix(config): require explicit trust for executable config paths (#4502) by @chtruong814
ci: cache-from a single coherent buildcache donor (#4529) by @ko3n1g
fix(data): Avoid VLM package collate import cycle (#4528) by @yaoyu-33
improve Qwen MegatronMIMO data loading (#4442) by @liding-nv
chore(beep boop 🤖): Bump (main, mcore-main) (2026-06-27) by @github-actions[bot]
[test] fix: avoid false skip in Nemotron Omni conversion (#4517) by @cuichenx
chore(beep boop 🤖): Bump (main, mcore-main) (2026-06-28) by @github-actions[bot]
ci: fix install check masking failed jobs in CI summary (#4554) by @svcnemo-autobot
ci: fix install-test summary masking failed jobs (#4559) by @svcnemo-autobot
ci: fix invalid gh run view --jq --arg usage in CI summary (#4568) by @svcnemo-autobot
chore(beep boop 🤖): Bump (main, mcore-main) (2026-06-29) by @github-actions[bot]
Set default for share_embeddings_and_output_weights (#4569) by @chochowski
[build] chore: bump transformer-engine to release_v2.16.post (#4536) by @ko3n1g
[data] fix: pack VLM batches directly (#4507) by @yaoyu-33
vr200 cfgs match w/ gb300 (#4574) by @malay-nagda
[training, perf] fix: reland THD FLOPS accounting with VPP fix (#4511) by @cuichenx
chore(beep boop 🤖): Bump uv.lock (main, mcore-main) (2026-06-30) (#4588) by @svcnvidia-nemo-ci
Correct Hybrid FLOP Calculation (#4508) by @Phlip79
test(models): Add AutoBridge registration matrix (#4542) by @yaoyu-33
test(data): cover collate import boundaries (#4571) by @yaoyu-33
refactor(data): split finetuning batch utilities (#4602) by @yaoyu-33
chore(beep boop 🤖): Bump uv.lock (main, mcore-main) (2026-07-01) (#4612) by @svcnvidia-nemo-ci
fix(training): default MoE expert tensor parallelism to one (#4579) by @yaoyu-33
fix(perf): repair llama packing and golden evaluation (#4584) by @yaoyu-33
[PEFT] feat: reduce LoRA sequence-parallel activation memory (#4581) by @cuichenx
feat(training): support epoch-based training duration (#4147) by @muyihao
[data] fix: Propagate seq_length override to Energon task encoder in build_datasets() (#4342) by @nayopu
add check for unsaved_keys and strict mode (#4513) by @rohitrango
ci: use NVIDIA inference for Claude review (#4618) by @chtruong814
fix(recipe): correct Qwen3.5-VL 122B H100 VPP (#4604) by @cuichenx
test(recipes): enforce NemotronH recipe invariants (#4572) by @yaoyu-33
[model] fix: adapt Gemma4 PLE recompute to MCore dev (#4445) by @yaoyu-33
chore(beep boop 🤖): Bump uv.lock (main, mcore-main) (2026-07-02) (#4625) by @svcnvidia-nemo-ci
[build] chore: revert "bump transformer-engine to release_v2.16.post (#4536)" (#4600) by @ko3n1g
fix(ci): make run_ci_tests usable with current launch scripts (#4624) by @yaoyu-33
docs: fix README drift + add consistency test (NVBug 6366190) (#4488) by @pruprakash
chore(beep boop 🤖): Bump uv.lock (main, mcore-dev) (2026-07-02) (#4626) by @svcnvidia-nemo-ci
[training] fix Qwen3-VL packed vlm_step MRoPE (#4532) by @cuichenx
Fix packed-sequence SFT prep edge cases for long-context THD+CP (#4593 §2.3-2.5) (#4598) by @sen-ppl
feat(quant): Add modelopt KV cache amax mapping. (#4591) by @mxinO
refactor(recipes): add h100 recipe namespace (#4622) by @yaoyu-33
fix(misc): preserve nested Hugging Face config mappings (#4631) by @cuichenx
[training] fix: skip attention_mask in packed THD context-parallel partitioning (#4228) (#4354) by @lonexreb
fix(data): make finetuning batch sampler epoch-aware on checkpoint resume (#4601) by @Achyuthan-S
build(deps): pin flash-mla to FlashMLA nv_dev ToT (b7643bd) (#4670) by @ko3n1g
build: remove jackson-core CVE patch layer from fw_final image (#4677) by @ko3n1g
chore(beep boop 🤖): Bump uv.lock (main, mcore-main) (2026-07-06) (#4672) by @svcnvidia-nemo-ci
chore(beep boop 🤖): Bump uv.lock (main, mcore-dev) (2026-07-06) (#4673) by @svcnvidia-nemo-ci
Generalized fix for mxfp8 param gather (#3980) by @zhongbozhu
chore(docker): update base image to nvcr.io/nvidia/pytorch:26.06 (#4649) by @balasaajay
[training] fix: require checkpoint source for finetuning (#4667) by @yaoyu-33
fix(data): preserve loss masking for mapped padding (#4675) by @yaoyu-33
fix(model): adapt MegatronMIMO grid to MCore expert-view API (#4651) by @ko3n1g
[ckpt] fix: defer async retention until finalization (#4678) by @yaoyu-33
[models] fix: preserve Ministral 3 export dimensions (#4676) by @yaoyu-33
docs(skills): repair sequence-packing guidance and refresh signature (#4636) by @cuichenx
fix(ckpt): remove all non-persistent checkpoints when most_recent_k is 0 (#4658) by @khazic
chore: Update vllm patch for 0.24.0 (#4638) by @chtruong814
AUT-712 ci/implement-281610164f9d-megatron-bridge (#4693) by @svcnemo-autobot
Remove system rdma-core/libibverbs-dev upgrade in INSTALL_DEEPEP (AUT-519) (#4695) by @svcnemo-autobot
fix(diffusion): honor local checkpoint dirs and task-derived model id in WAN inference (#4408) by @huvunvidia
[training] fix: prevent crash on side-threads (#4375) by @tdene
Migrate GPT-OSS to HybridModel (#4476) by @Phlip79
fix(recipe): export Nemotron VL recipes at package root (#4697) by @yaoyu-33
[inference] fix: preserve batched Qwen VL visual inputs (#4692) by @yaoyu-33
[data] fix: Preserve Qwen2-Audio source sampling rates (#4699) by @yaoyu-33
[model] fix: restore quantized HF export streaming (#4701) by @yaoyu-33
fix(training): pass training_log optional args by keyword to avoid misbinding (#4656) by @khazic
refactor(model)!: remove unused HF pretrained APIs (#4719) by @yaoyu-33
fix(diffusion): use generic fast tokenizer for FLUX CLIP (#4720) by @cuichenx
ci: add weekly verl e2e test suite (#4705) by @ko3n1g
fix(model): apply final logit softcapping to Gemma 4 dense models (#4645) by @cuichenx
fix(misc): pass trust_remote_code when loading tokenizer from checkpoint in compare_text_generation (#4691) by @pruprakash
docs(recipe): fix DeepSeek V2 pretraining examples (#4731) by @yaoyu-33
fix(checkpoint): preserve quantization recipe config (#4688) by @cuichenx
fix(model): map router expert_bias in DeepSeek-family common mapping list (#4657) by @khazic
fix(data): return None for Energon test split instead of aliasing validation (#4659) by @khazic
fix(model): use tanh-approximate GELU for Gemma-1 bridge (#4661) by @khazic
fix(inference): reject pipeline parallelism in QwenVLInferenceWrapper (#4662) by @khazic
docs(data): clarify shared fine-tuning dataset storage (#4715) by @cuichenx
[training] fix: Run eval callbacks in evaluation mode (#4736) by @yaoyu-33
feat(distill): defer KD conversion to a pre-wrap hook and support submodule distillation for VLMs (#4707) by @kevalmorabia97
Fix output layer not found with VPP > 1 when tie_word_embeddings (#4694) by @HollowMan6
test(checkpoint): isolate local checkpoint scenarios (#4744) by @yaoyu-33
perf(ckpt): avoid quadratic HF shard completion scans (#4714) by @cuichenx
Support GLM5.2 and recipe for GLM 5.1 & 5.2 on GB200 and H100 (#4520) by @HollowMan6
refactor(performance): remove legacy perf configs (#4623) by @yaoyu-33
fix(training): prevent VPP FLOPS metadata overcounting (#4750) by @cuichenx
[test] fix: Remove online Kimi perf recipe test (#4757) by @yaoyu-33
fix(perf): wire PreemptionPlugin for long-convergence runs (#4765) by @ko3n1g
Add verl ci that covers tie word embedding critic modes + VPP (#4755) by @HollowMan6
qwen35_vl_vr200_intro (#4762) by @malay-nagda
[recipe, docs] fix: Update DeepSeek V3 pretraining example (#4761) by @yaoyu-33
[ckpt] fix: Refresh tracker state across in-process restarts (#4758) by @yaoyu-33
[training] fix: allow W&B logging without checkpointing (#4763) by @yaoyu-33
fix(model): shard MiMo-V2 conversion experts with ETP (#4754) by @yaoyu-33
[diffusion] fix: preserve LLaDA batch positions (#4760) by @yaoyu-33
[ckpt] fix: preserve custom pipeline layouts in checkpoints (#4769) by @yaoyu-33
feat(conversion): preserve weights in toy model generator (#4751) by @yaoyu-33
[model] fix: preserve Step KV cache amax mappings (#4756) by @yaoyu-33
refactor(data): adopt config and builder dataset architecture (#4685) by @yaoyu-33
build(deps): constrain tokenizers below 0.23 so CLIP tokenizers construct (#4727) by @pruprakash
fix(ckpt): preserve artifacts in config-only exports (#4687) by @cuichenx
fix(model): enable attention output-gate fallback for step35 head-wis… (#4686) by @pruprakash
fix(data): mask MegatronMIMO padding by position, not pad_token_id value (#4660) by @khazic
docs(skills): add repository review principles (#4778) by @yaoyu-33
refactor(performance): remove run-with-env entrypoint (#4777) by @yaoyu-33
fix(model): resolve Gemma 4 architecture bugs (#4683) by @cuichenx
fix(model): make step35 layer-spec builder public so converted checkpoints load (#4690) by @pruprakash
[feature, model]Fix Qwen3-Omni Megatron HF corr checks (#4567) by @hbhflw2000
fix(training): reject unsupported local CUDA graph scopes (#4607) by @yaoyu-33
[training, data] fix: align eval CP data ownership (#4785) by @yaoyu-33
[diffusion] fix: Encode Wan T5 prompts on CPU (#4793) by @yaoyu-33
[ckpt] fix: align ModelOpt checkpoint selection (#4791) by @yaoyu-33
fix(performance): synchronize sequence length override (#4779) by @yaoyu-33
[model] fix: Gather Nemotron diffusion TP logits (#4780) by @yaoyu-33
[data] fix: make CORD target selection deterministic (#4789) by @yaoyu-33
DSV4: Making compress_rope_theta priotitzed than rope_scaling->rope_theta (#4802) by @HollowMan6
fix(ckpt): load fused MoE experts into sequential (non-grouped) layout (#4706) by @kevalmorabia97
support modelopt W4A16 & W4A4 NVFP4 export for grouped MoE weights (#4566) by @HollowMan6
feat(model): add HY V3 MoE bridge (#4680) by @xhx1022
[ckpt, model] fix: restore MCore main bump compatibility (#4770) by @yaoyu-33
[ci] test: rebalance model functional test tiers (#4803) by @yaoyu-33
[model] fix: preserve Qwen hybrid attention schedules (#4790) by @yaoyu-33
fix(perf): copy PreemptionPlugin into launcher to drop megatron import (#4814) by @ko3n1g
test(model): fix AutoBridge and Qwen VL regressions (#4821) by @yaoyu-33
[model] fix: add HY V3 AutoBridge registration contract (#4822) by @yaoyu-33
docs(recipe): fix Gemma 2 checkpoint overrides (#4811) by @yaoyu-33
[data] fix: Stabilize packed tokenizer paths (#4817) by @yaoyu-33
[model] fix: restore Bailing MTP weight conversion (#4819) by @yaoyu-33
fix(ckpt): preserve durable async non-persistent checkpoints (#4820) by @yaoyu-33
fix(diffusion): construct Wan pipeline stages (#4823) by @yaoyu-33
[ckpt] fix: load legacy checkpoints without content metadata (#4828) by @yaoyu-33
fix(training): align GDN MTP FLOPs with decoder layout (#4834) by @yaoyu-33
fix checkpointing in respect to mcore changes (#4818) by @dimapihtar
[model] fix: preserve Falcon H1 input scaling (#4845) by @yaoyu-33
[data] fix: preserve explicit fixed padding (#4842) by @yaoyu-33
Enable expert-parallel language modules in MegatronMIMO (#4485) by @liding-nv
add opt-in NUMA binding for ci testing (#4630) by @cuichenx
feat(model): add MiniMax M3 language-model bridge and recipes (#4652) by @khazic
feat(model): add masked lm support. (#4735) by @bbuschkaemper
chore: update transformer-engine dependency to version 2.17.0 (#4807) by @balasaajay
fix(training): allow local CUDA graphs for inference scopes (#4851) by @HollowMan6
Enable optimizer CUDA graph (#2411) by @vasunvidia
[diffusion, data] fix: pad WAN context for CP (#4858) by @yaoyu-33
[doc, recipe] fix: correct Gemma3-VL recipe names (#4860) by @yaoyu-33
test(functional): quarantine MegatronMIMO training smoke (#4867) by @yaoyu-33
[training, ckpt] fix: Reset MCore async queue during restart (#4839) by @yaoyu-33
fix(model): preserve sliding-window config export (#4800) by @cuichenx
test(recipe): re-enable Qwen VL functional tests (#4866) by @balasaajay
refactor(data)!: unify multimodal dataset builders (#4774) by @yaoyu-33
Pin cuDNN FE to 1.26.0 and cutlass-dsl to 4.5.0 (#4868) by @balasaajay
[model] fix: adapt Gemma 4 MoE to MCore packed sequences (#4846) by @yaoyu-33
[training] fix: support packed Nemotron-VL batches (#4874) by @yaoyu-33
[model] fix: align Gemma 2 sliding attention layers (#4875) by @yaoyu-33
feat(training): add shared recipe launcher (#4856) by @yaoyu-33
feat(conversion): add shared checkpoint launcher (#4873) by @yaoyu-33
fix(conversion): resolve tied output aliases from parameter mappings (#4884) by @yaoyu-33
fix(training): restore SFT cache and MoE overlap compatibility (#4882) by @yaoyu-33
test: consolidate redundant unit coverage (#4887) by @yaoyu-33
refactor(misc): minimize private Transformer Engine integrations (#4772) by @cuichenx
chore(skills): add Regent Open Plugin manifest (#4892) by @ko3n1g
chore(skills): remove Open Plugin manifest (superseded) (#4894) by @ko3n1g
chore(beep boop 🤖): Bump (main, mcore-main) (2026-07-16) by @github-actions[bot]
[ckpt] fix: restore RNG state when future saves disable it (#4893) by @yaoyu-33
[inference] fix: respect prompt file truncation limit (#4896) by @yaoyu-33
[training] fix: resume global non-persistent checkpoints (#4900) by @yaoyu-33
fix(recipe): add FC LoRA targets for Nemotron Omni (#4853) by @cuichenx
[data] feat: Add Tulu 3 dataset preset and current validation evidence (#4888) by @yaoyu-33
[data] fix: isolate and unify Nemotron Omni collation (#4776) by @cuichenx
fix(qwen3-asr): preserve native auto registrations (#4876) by @aoshen02
fix(training): reset model callbacks during in-process restart (#4889) by @yaoyu-33
[data] fix: honor seed during offline packing (#4891) by @yaoyu-33
test(recipes): validate all recipe factories (#4912) by @yaoyu-33
build(docker): make nvshmem reinstall optional in Dockerfile.ci (#4901) by @thomasdhc
test(model): stabilize Gemma-1 activation roundtrip (#4927) by @yaoyu-33
ci(coverage): exclude examples and scripts (#4915) by @yaoyu-33
[docs] fix: Align async inference examples with Bridge CLI (#4919) by @yaoyu-33
[model] fix: Preserve config-only Gemma 3 provider creation (#4918) by @yaoyu-33
[doc] fix: Update GLM-4.5 recipe examples (#4921) by @yaoyu-33
[data] fix: respect drop_last for iteration training (#4926) by @yaoyu-33
feat(ckpt): add roundtrip conversion launcher (#4886) by @yaoyu-33
ci: AUT-897 raise UV_HTTP_TIMEOUT to 120s in CI container build (#4930) by @svcnemo-autobot
fix(model): align Gemma 4 MoE global RoPE with Hugging Face (#4916) by @yaoyu-33
[docs] docs: Add LongStraw ecosystem news (#4933) by @yaoyu-33
chore: Set fastapi < 0.139.2 (#4935) by @chtruong814
Adding FSDP/HSDP example for nemotron3 nano (#4745) by @gautham-kollu
[misc] fix: Make standalone top-p sampling usable (#4937) by @yaoyu-33
fix(recipe): propagate GLM-4.5V Slurm failures (#4938) by @yaoyu-33
[training] fix: avoid duplicate MIMO evaluation timer (#4939) by @yaoyu-33
[data] fix: reject unsafe custom batch padding (#4943) by @yaoyu-33
[training, ckpt] fix: Resolve default in-process restart world size (#4944) by @yaoyu-33
feat(training): route performance recipes by name (#4945) by @yaoyu-33
chore(deps): AUT-901 bump Megatron-LM to mcore main b4ad280d (#4948) by @svcnemo-autobot
[training] fix: Close TensorBoard writer on restart (#4951) by @yaoyu-33
[docs] docs: Mention Macaron-V1 in LongStraw news (#4958) by @yaoyu-33
chore: AUT-905 bump Megatron-Core (main) to 3219f38 (#4956) by @svcnemo-autobot
[misc] fix: reject unsupported legacy stopping controls (#4954) by @yaoyu-33
Hsdp Config (#4746) by @gautham-kollu
chore(beep boop 🤖): Bump (main, mcore-main) (2026-07-20) by @github-actions[bot]
build(deps): upgrade Transformers to 5.12.1 (#4928) by @cuichenx
feat(recipes): support recipe environment variables (#4682) by @yaoyu-33
fix: preserve Qwen3 routing and packed loss masks (#4946) by @yaoyu-33
fix(ckpt): reduce distributed optimizer resume memory (#4936) by @cuichenx
[recipes] fix: Restore DeepSeek HSDP recipe environment (#4972) by @yaoyu-33
fix(perf): read legacy SQuAD cache columns (#4970) by @yaoyu-33
Add CLI configurability of num_moe_experts (#4968) by @scsudhakaran
revert: AUT-927 "Migrate GPT-OSS to HybridModel (#4476)" (#4982) by @svcnemo-autobot
[recipe] fix: propagate GPT-OSS pretraining failures (#4984) by @yaoyu-33
[training] fix: Create MIMO embedding groups in global order (#4985) by @yaoyu-33
[ckpt] fix: Dispatch MIMO loads through checkpoint manager (#4981) by @yaoyu-33
[ckpt] fix: Clear saved pipeline layout for single-rank load (#4992) by @yaoyu-33
[inference] fix: Seed VLM sampling generator (#4993) by @yaoyu-33
[ckpt, data] feat: complete Energon dataloader checkpointing save/res… (#4521) by @zsombor-kaiko
ci: AUT-895 gate GB200 tests on DISABLE_GB200_TESTS variable (#4929) by @svcnemo-autobot
docs: fix broken links in Megatron-Bridge docs (NVBug 6421302) (#4969) by @chenopis
fix(data): retry packed-parquet resolution with NFS directory-metadata refresh (#4207) (#4865) by @janbernloehr
fix(model): correct Qwen3-VL vision_dp_when_cp CP gradient and avoid 0-image-rank hang (#4784) by @going-song
build: AUT-944 bump Megatron-Core main pin to 58bf14e (#4991) by @svcnemo-autobot
ci: AUT-945 seed a PR's container build from its own buildcache (#4995) by @svcnemo-autobot
feat(training): launch canonical performance recipes (#4620) by @yaoyu-33
26.06.01 perf summary (#4989) by @malay-nagda
[recipe, model] fix: resolve 26.08 model workflow regressions (#4994) by @yaoyu-33
feat(model): add model support verification cards (#4909) by @yaoyu-33
fix(training): align comm-overlap validation with Megatron-Core (#4971) by @cuichenx
test(ckpt): isolate conversion launcher utility imports (#5003) by @yaoyu-33
build(deps): provide transformer-engine metadata so uv lock skips its source build (#5018) by @ko3n1g
docs(model): verify H100 checkpoint resumes (#5009) by @yaoyu-33
[docs] fix: Include model verification skill in docs index (#5026) by @yaoyu-33
[training] fix: align ModelOpt restore with resume source (#5014) by @yaoyu-33
[training] fix: Clean up failed pretrain sessions (#5013) by @yaoyu-33
[data] fix: Handle Qwen3-Next custom chat data (#5012) by @yaoyu-33
[recipe] fix: Resync Kimi pipeline layout after overrides (#5008) by @yaoyu-33
feat(conversion): add HF embedding row repair script (#5005) by @yaoyu-33
docs(model): list verification index items explicitly (#5029) by @yaoyu-33
fix(perf): keep recipe imports off login nodes (#5032) by @yaoyu-33
fix(model): use TP=1 for GLM-5 inference (#4999) by @yaoyu-33
[perf] feat: add Qwen3 and DeepSeek-V3 MoE recipes (#4850) by @dimapihtar
remove fine-tuning references (#5024) by @malay-nagda
fix(perf): reinstate aws-ofi-nccl EFA plugin for EKS (#5033) by @ko3n1g
Update DeepSeek-V3 B300 recipe (#5015) by @scsudhakaran
build(deps): AUT-960 bump Megatron-Core main to 6cd6ea53 (#5044) by @svcnemo-autobot
add fixed mem clk rubin (#4920) by @malay-nagda
feat(training): add VR200 memory clock lock (#5049) by @yaoyu-33
fix(training): include Comet in logger presence gate (#5041) by @yaoyu-33
fix(recipe): stop enabling cuDNN norm by default (#5040) by @yaoyu-33
[model] fix: preserve Nemotron runtime config (#4974) by @cuichenx
fix(recipe): restore Nemotron VL SFT batches (#5000) by @cuichenx
fix(data): support GLM-4.5V assistant masking (#5001) by @cuichenx
perf(recipe): optimize Moonlight training throughput (#5021) by @yaoyu-33
feat(recipes): add Qwen3.5 and Nemotron 3 Nano GB200 pretraining (#4854) by @cuichenx
feat: add Slurm inference launcher (#4931) by @cuichenx
chore(deps): AUT-952 bump Megatron-Core dev ref (#5019) by @svcnemo-autobot
chore(model): deprecate legacy model support (#5025) by @yaoyu-33
feat(model): add MiniMax-M3 text-only verification (#5002) by @yaoyu-33
test(recipes): drop brittle count asserts from recipe env invariants (#5056) by @cuichenx
test(recipes): stop pinning exact recipe count in env invariant (#5057) by @cuichenx
fix(data): shuffle direct HF SFT with cyclic sampler (#5048) by @yaoyu-33
[training] feat: add scheduler max steps for shortened tests (#4979) by @cuichenx
fix(model): improve conversion and inference examples (#4805) by @yaoyu-33
feat(tokenizers): support loading a chat template from a file (#4899) by @omri-manolo
fix(model): forward Qwen3-VL output processor hooks (#5060) by @yaoyu-33
fix(skill): require one inference run (#5063) by @yaoyu-33
[test] fix: remove recipe inventory count assertions (#5064) by @yaoyu-33
feat (recipe): add Nemotron 3 Ultra pretraining perf-recipe (#4911) by @ghadiaravi13
fix(docker): MB-937 guard Torch op cleanup during shutdown (#5046) by @svcnemo-autobot
fix(perf): MB-923 support recipes without environment maps (#5043) by @svcnemo-autobot
fix(data): normalize ShareGPT role aliases (#5078) by @yaoyu-33
[training] fix: account for uneven PP in memory estimate (#5077) by @yaoyu-33
[inference] fix: Handle variable-length VLM batches (#5072) by @yaoyu-33
fix(model): honor plain export dtype for Kimi K2.5 (#5069) by @yaoyu-33
feat(recipe): add Qwen3 30B GB200 MXFP8 verification (#5066) by @yaoyu-33
build(deps): AUT-960 bump Megatron-Core dev to 0afb41b9 (#5045) by @svcnemo-autobot
fix(perf): MB-923 align mounted scripts with bridge source (#5079) by @svcnemo-autobot
chore(misc): remove legacy evaluation examples (#5059) by @cuichenx
Add notes and instructions for dynamic/non-dynamic resolution path for Nemotron-Omni (#5053) by @huvunvidia
[recipe, perf] feat: port r0.5.0 performance recipes (#5061) by @yaoyu-33
feat(model): accept pre-sharded packed Qwen3-VL inputs (#4917) by @yaoyu-33
fix(models): verify Gemma 4 VL BF16 conversion (#5036) by @yaoyu-33
feat(model): add MiniMax-M3 VLM import and export (#5058) by @yaoyu-33
fix(flops): use parquet packed dataset for LoRA SQuAD FLOP accounting (#5010) by @rsalagame-nvidia
fix(data): use Nemotron Ultra recipe defaults (#5082) by @cuichenx
[training] fix: Respect disabled exit signal handling (#5068) by @yaoyu-33
perf(recipe): overlap Qwen3 expert communication on H100 (#5094) by @yaoyu-33
[misc] fix: Preserve chat templates in inference tokenizer (#5091) by @yaoyu-33
[recipe] fix: Correct Ministral 3 SFT learning rates (#5092) by @yaoyu-33
[ckpt] fix: select newest global recovery root (#5090) by @yaoyu-33
Add EXAONE 4.5 & K-EXAONE bridge (#4730) by @sangHa0411
[data] fix: build memmap indexes on every node (#5107) by @yaoyu-33
[ckpt] fix: ignore FSDP RNG after layout changes (#5100) by @yaoyu-33
[inference] fix: Honor configured Qwen VLM visual token IDs (#5101) by @yaoyu-33
[doc] fix: update Moonlight recipe examples (#5103) by @yaoyu-33
[training] fix: Preserve flat Hugging Face configs (#5106) by @yaoyu-33
chore(deps): AUT-1076 bump MCore main (#5114) by @svcnemo-autobot
[model] fix: Restore callable Mamba stack specs (#5116) by @yaoyu-33
[doc] fix: update OLMoE recipe examples (#5118) by @yaoyu-33
fix(training): abort async checkpoints after pretrain failure (#5117) by @yaoyu-33
[data] fix: accept decoded Nemotron video inputs (#5121) by @yaoyu-33
build(deps): AUT-969 update Transformer Engine to 2.18.0 (#5054) by @svcnemo-autobot
[training] fix: Honor logging environment override (#5120) by @yaoyu-33
[model] fix: honor provider precision overrides (#5122) by @yaoyu-33
[model] fix: preserve linear Llama RoPE scaling (#5097) by @yaoyu-33
docs(model-card): verify MiniMax-M3 VLM (#5065) by @yaoyu-33
[recipes] feat: add MTP support to Nemotron 3 Nano (#5035) by @cuichenx
feat(recipe): add DSv4-Flash GB200 pretrain recipes (BF16/MXFP8/Muon) (#5127) by @weijiac0619
chore(model): deprecate additional legacy model support (#5125) by @yaoyu-33
test(mimo): restore bias-free MoE training smoke (#5115) by @liding-nv
[🤖]: Update docs-versions after code-freeze for r0.6.0 by @github-actions[bot]
Bump docs version to r0.6.0 by @github-actions[bot]
Add SequentialMLP mappings for MTP heads in nemotron_h_bridge.py (5140) (#5152) by @svcnvidia-nemo-ci
[test] fix: repair Nemotron-H CI regressions (5143) (#5146) by @svcnvidia-nemo-ci
cp: Nemotron 3.5 Nano verification card and final-checkpoint refresh (#5124, #5199) into r0.6.0 (#5178) by @svcnvidia-nemo-ci
fix(docker): update dependencies in Dockerfile and fw_pyproject.toml for compatibility and security (5202) (#5208) by @svcnvidia-nemo-ci
fix(model): map GLM-5 routed expert count (5220) (#5248) by @svcnvidia-nemo-ci
feat(sft): DSv4 Flash offline-packed SFT with THD + CP (5126) (#5247) by @svcnvidia-nemo-ci
[Doc] Document latest Mcore toggle moe_hybridep_pad_uneven_dispatch_inputs for HybridEP THD training (4213) (#5159) by @svcnvidia-nemo-ci
fix(perf): port r0.5.0 VR200 performance configs (5196) (#5197) by @svcnvidia-nemo-ci
feat(misc): add memory snapshot analysis skill (5150) (#5198) by @svcnvidia-nemo-ci
fix(model): add Gemma 3 VL verification coverage (5084) (#5204) by @svcnvidia-nemo-ci
fix(training): preserve alltoall flex fallback (5207) (#5214) by @svcnvidia-nemo-ci
fix(model): align MiniMax evidence and inference task routing (5194) (#5235) by @svcnvidia-nemo-ci
fix(peft): sync grouped expert LoRA over expert DP (5231) (#5269) by @svcnvidia-nemo-ci
chore(deps): update mlflow version to 3.15.0 (5267) (#5275) by @svcnvidia-nemo-ci
chore: bump mcore dev commit for DSv4 support (#5284) by @weijiac0619
fix(recipes): derive pretraining vocab from tokenizer (4996) (#5280) by @cuichenx
fix(data): preserve VLM pixel defaults (5218) (#5271) by @svcnvidia-nemo-ci
fix(data): reject truncation without supervision (5217) (#5272) by @svcnvidia-nemo-ci
fix(data): normalize GLM VLM assistant content (5216) (#5273) by @svcnvidia-nemo-ci
fix(data): pad Qwen VLM token type IDs (5215) (#5274) by @svcnvidia-nemo-ci
perf(recipe): tune Nemotron 3.5 Nano H100 benchmark (5266) (#5279) by @svcnvidia-nemo-ci
fix(training): restore weak scaling for benchmark recipes (5222) (#5282) by @svcnvidia-nemo-ci
chore(deps): update transformer-engine dependency to a new commit hash by @balasaajay
feat (recipe): add Nemotron 3 Ultra MXFP8 GB200 pretraining perf-recipe (5287) (#5309) by @svcnvidia-nemo-ci
fix(training): count expert parameter norms with TP and EP (4998) (#5177) by @svcnvidia-nemo-ci
fix(model): restore DeepSeek V3 sequence auxiliary loss (5165) (#5172) by @svcnvidia-nemo-ci
cp: Nemotron 3.5 Lightning checkpoint and naming updates (#5293, #5303) into r0.6.0 (#5300) by @svcnvidia-nemo-ci
fix(training): backport HybridEP uneven-dispatch compatibility to r0.6.0 (#5288) by @cuichenx
perf(recipe): add DeepSeek V4 Flash GB300 FP8MX recipes (#5258) by @malay-nagda
build: bump nvidia-modelopt to 0.46.0rc1 (5331) (#5338) by @svcnvidia-nemo-ci
chore(docker): update Dockerfile and install script to remove unnecessary packages (#5330) by @balasaajay
build: AUT-1353 bump transformer-engine to 2.17.1 (#5349) by @svcnemo-autobot
build: AUT-1346 pin wandb to 0.27.2 (#5348) by @svcnemo-autobot
fix(ci): AUT-1359 skip humming module in import check (#5358) by @svcnemo-autobot
fix(model): AUT-1326 backport Gemma 3 VL fused attention to r0.6.0 (#5336) by @svcnemo-autobot
fix(recipe): backport #5386 batch-size alignment to r0.6.0 (#5393) by @cuichenx
fix(perf): mount common performance scripts (5381) (#5404) by @svcnvidia-nemo-ci
chore(deps): update mlflow to 3.15.1 (#5419) by @balasaajay
feat(model): add GLM-5.2 verification card (5034) (#5301) by @svcnvidia-nemo-ci
fix(ckpt): validate conversion world size against TPPP, not TPPP*EP (5302) (#5312) by @svcnvidia-nemo-ci
[docs] fix: point Gemma 4 VL inference to verification card (5311) (#5319) by @svcnvidia-nemo-ci
fix(model): enable RADIO recompute for Nemotron Omni SFT (#5320) by @cuichenx
[model] fix: honor Gemma 4 image attention mode (5333) (#5350) by @svcnvidia-nemo-ci
[ckpt] fix: support vocab padding with Megatron-FSDP TP (5353) (#5359) by @svcnvidia-nemo-ci
fix(ckpt): avoid roundtrip export barrier timeout (5377) (#5412) by @svcnvidia-nemo-ci
fix(peft): preserve grouped expert SwiGLU LoRA order (5376) (#5414) by @svcnvidia-nemo-ci
fix(training): include MTP stages in embedding groups (5380) (#5422) by @yaoyu-33
fix: repair release QA regressions (5351) (#5423) by @yaoyu-33
[models, conversion] fix: verify Step-3.5 model workflows (5334) (#5424) by @yaoyu-33
fix(training): surface distributed failures without cleanup hangs (5283) (#5327) by @svcnvidia-nemo-ci
fix(modelopt): handle sparse export task lists (5374) (#5446) by @svcnvidia-nemo-ci
fix(ckpt): restrict dataloader state deserialization (5418) (#5437) by @svcnvidia-nemo-ci
fix(recipes): use max-pool FSDP for Qwen3.5-VL (5402) (#5430) by @svcnvidia-nemo-ci
fix(recipe): verify GLM-5.2 RP2 pretraining and packed SFT (5344) (#5425) by @yaoyu-33
[examples] refactor: direct GLM inference to model card (5346) (#5427) by @yaoyu-33
docs(models): link MiMo and StepFun examples (5432) (#5439) by @svcnvidia-nemo-ci
docs(recipe): retire stale Gemma 4 VL SFT launchers (5433) (#5435) by @svcnvidia-nemo-ci
fix(peft): synchronize shared expert LoRA across EP (5396) (#5428) by @svcnvidia-nemo-ci
fix(model): preserve Step3.7 text config fields (5417) (#5429) by @svcnvidia-nemo-ci
[ckpt] fix: Deinterleave low-memory GLU checkpoints (5366) (#5370) by @svcnvidia-nemo-ci
Update wandb dependency version to 0.27.2 (#5461) by @balasaajay
fix(model): backport Gemma4 PLE sequence parallel sharding to r0.6.0 (#5450) by @cuichenx
fix(data): collate ragged packed sequence metadata (5400) (#5426) by @yaoyu-33
build: AUT-1430 upgrade GitPython to 3.1.58 (#5480) by @svcnemo-autobot
fix(models): use HybridEP for MiniMax-M3 expert parallelism (5448) (#5473) by @svcnvidia-nemo-ci
fix(diffusion): calculate WAN FLOPs from runtime geometry (5249) (#5477) by @svcnvidia-nemo-ci
feat(data): infer assistant loss mask for gpt-oss Harmony chat template (5389) (#5447) by @svcnvidia-nemo-ci
fix(scripts): preserve FHT and fetch missing MCore commits (5478) (#5481) by @svcnvidia-nemo-ci
fix(models): use HybridEP for Kimi-K2.5 inference (5421) (#5485) by @cuichenx
fix(model): handle disabled GLM-5 MTP export (5445) (#5488) by @svcnvidia-nemo-ci
docs(ckpt): document roundtrip scaling limits (5491) (#5492) by @svcnvidia-nemo-ci
docs(model-card): add Nemotron 3 Super verification card (5062) (#5494) by @svcnvidia-nemo-ci
docs: update performance summary for 26.08 (5496) (#5507) by @svcnvidia-nemo-ci
fix(deepseek-v4): use full-prefix inference (5509) (#5511) by @svcnvidia-nemo-ci
fix(recipe): prevent Qwen3.5-VL FSDP SFT OOM (5508) (#5515) by @svcnvidia-nemo-ci
fix(recipe): recompute Gemma 4 VL SFT layers (#5506) by @yaoyu-33
docs(dsv4): preserve container packages during MCore sync (5514) (#5516) by @svcnvidia-nemo-ci
Update nemo-evaluator path in fw_pyproject.toml (#5571) by @balasaajay
docs: Update 0.6.0 version entries for docs (#5567) by @balasaajay
fix(deepseek): default DeepSeek V4 to HybridEP (5543) (#5582) by @svcnvidia-nemo-ci
docs: fix preferred version and add 0.5.1 to the version picker (#5623) by @ko3n1g
beep boop 🤖: Bumping Megatron-Bridge to v0.6.1 [skip ci] by @github-actions[bot]

Navigation Menu

Uh oh!

NVIDIA Megatron-Bridge 0.6.0

Choose a tag to compare

Sorry, something went wrong.

Sorry, something went wrong.

Uh oh!

No results found

NeMo Framework 26.08 — Megatron Bridge Release Notes

Featured Blog

Highlights

Model Collection Support

LLM

VLM / Multimodal

Diffusion

Training & Functionality

Checkpoint Conversion

Performance

Examples & Tutorials

Community Contributions

Deprecated Models

Known Issues

Contributors

Uh oh!