Skip to content

v1.7.0

Latest

Choose a tag to compare

@hjh0119 hjh0119 released this 11 Nov 13:39
· 3 commits to main since this release

v1.7.0

MTP Subsystem Rewrite

  • model/modules/mtp_layer.py removed and replaced by model/mtp.py + model/mtp_detect.py. (#237)
  • Instead of branching on a version number, the bridge now probes the installed upstream code (MtpCapabilities, process_mtp_loss signature sniffing), so one code path covers Megatron-Core 0.18–0.20 and the dev branch.
  • MTP depth and cross-layer weight sharing are resolved automatically from the checkpoint's safetensors keys plus the HF config (resolve_mtp_config, now exported from the package root).
  • The MTP template is resolved from the full decoder layer list rather than the local rank shard. (#256)

Megatron-Core Compatibility

  • Minimum raised to 0.18 (#247) and the upper bound widened to <0.21, covering the 0.20 release and the latest dev branch. (#252)
  • DeepSeek-V4 layer-spec patching now handles builders that are functools.partial objects (Megatron dev) as well as plain ModuleSpec (0.18–0.20), so the compressor / indexer / DSv4HybridSelfAttention replacements apply on every release instead of raising on dev. (#260)
  • The GLM-5.3 dev patch was reworked and substantially reduced; apply it with tools/apply_megatron_patch.py. (#240)

New Models

  • nemotron_h_omni — NVIDIA Nemotron-3.5 Super-VL bridge spec. (#235)
  • nemo_speechlm — NVIDIA Nemotron-3.5 Transcribe bridge spec. (#244)

Parallelism & Gradient Correctness

  • GDN sequence parallel: _force_sequence_parallel_off now clears sequence_parallel on the projections and on their parameters, so the gradient finalizer no longer sums an already-complete copy — this was the reported 4x gamma gradient at TP4. Pinned by a two-rank gloo regression. (#253, #254)
  • PLE: gradients fixed under sequence parallelism. (#234)
  • QSA: packed attention is now built on each document's own block grid instead of the global sequence. (#238)
  • LoRA / NPU: row-parallel LoRA input sizing corrected for TENPU (#255); LoRA and expert handling adapted to Megatron-Core 0.18 + TENPU. (#189)

DeepSeek-V4.1

  • Requantization on export (#259): exporting a quantized DeepSeek-V4.1 checkpoint now restores the official layout the BF16 training parameters were loaded from — fp8-e4m3 with 32x32-block e8m0 scales for linears, packed MXFP4 with per-32 scales for routed experts — instead of writing everything back as BF16. This is gated on the checkpoint's own quantization_config declaration, not on a user switch, and the numerics mirror the DeepGEMM reference (utils/quantizer.py). Exported artifacts change dtype in this release.
  • FP8-resident Engram tables (#259): model/modules/engram_fp8.py keeps the checkpoint's fp8 bytes and e8m0 scales in place and decodes only the rows a forward actually touches, instead of materializing 2x bf16 row shards; loading becomes a byte copy. Requires the Megatron EPShardedEmbeddingTable API (0.20 / dev) and falls back to the BF16 table path when it is absent; a missing engram.embed.weight_scale_inv on an fp8 store now raises instead of silently mis-loading.
  • Language-only checkpoints no longer run visual preprocessing during weight conversion, while the multimodal parent conversion and image-marker order are preserved. (#248)
  • Block32 FP8 source projections and Engram WKV weights are dequantized with the existing Fp8Dequantizer for BF16 loading (F32/UE8M0 scales, both TP split dimensions, lazy Engram rows); FP8 training parameters are rejected on this path. (#249)

Gemma 4

  • Per-layer attention dimensions read correctly for transformers>=5.15, where global_head_dim / num_global_key_value_heads moved into per_layer_config. (#239)
  • Optional ffpa-attn kernel for head_dim=512 full-attention layers on packed (THD) inputs, enabled with GEMMA4_FFPA_ATTN=1; avoids materializing an s x s score matrix on Hopper, where Transformer Engine has no fused backward. Sliding-window layers, non-packed inputs and non-SM90 GPUs keep using TE, with explicit checks for a missing backend, CP, dropout, masks, attention bias and padded packed sequences. (#243)

Testing & Tooling

  • New CPU-only regressions: DeepSeek-V4.1 block32 loading and language-only preprocessing, GDN SP projection parameters, Gemma 4 per-layer config and ffpa-attn dispatch/recompute, LoRA row-parallel sizing, PLE SP/TP gradients, QSA tile index and sparse kernels, Nemotron Omni and NemoSpeechLM.
  • megatron_utils and patcher slimmed as version gating moved behind capability probes.

New Contributors

Full Changelog: v1.6.5...v1.7.0