Repository navigation
October 31, 2026 · NeMo AutoModel compatibility and checkpoint fixes
This release adds the batching, chat tokenization, and checkpoint APIs needed for Zephon's NeMo AutoModel integration, and fixes checkpointing and process cleanup when finite pipelines finish.
- Multimodal training batches: pad variable-length packs and collate image fields and model positions with
to_training(). Return document IDs and mask labels across document boundaries. - Improved chat training options: choose BOS/EOS insertion, override special-token IDs, and control assistant-header supervision and loss masks. Support named chat templates and template selection based on tool definitions.
- Byte checkpoints: save Zephon state with
checkpoint(as_bytes=True)and pass the bytes directly torestore(). This lets PyTorch DCP preserve in-flight replay state without a separate pickle wrapper.
Upgrading: Existing checkpoint() calls continue to return dictionaries. For DCP, use the byte interface and pass the saved bytes directly to restore().