Skip to content

Support offline SpeechLM2 HF exports#15736

Open
pzelasko wants to merge 1 commit into
mainfrom
codex/speechlm2-offline-backbone
Open

Support offline SpeechLM2 HF exports#15736
pzelasko wants to merge 1 commit into
mainfrom
codex/speechlm2-offline-backbone

Conversation

@pzelasko
Copy link
Copy Markdown
Collaborator

@pzelasko pzelasko commented May 29, 2026

What changed

This is an alternative implementation for the offline-loading goal in #15695.

It keeps the root checkpoint config.json as the NeMo/SpeechLM wrapper config and saves the original LLM HF config under llm_backbone/config.json to avoid a config.json filename conflict. The tokenizer remains saved once at the checkpoint root for vLLM compatibility.

At load time, HFHubMixin detects the local export layout and redirects:

  • root tokenizer files -> cfg["tokenizer_path"]
  • llm_backbone/config.json -> cfg["pretrained_llm"] or cfg["pretrained_lm_name"]

This keeps old checkpoints backward compatible because configs without these local files are left unchanged.

Compatibility coverage

  • SALM
  • SALMAutomodel
  • SALMWithAsrDecoder
  • Duplex S2S
  • Duplex S2S speech decoder
  • Duplex STT
  • Duplex EAR-TTS

Testing

  • python -m py_compile examples/speechlm2/to_hf.py nemo/collections/speechlm2/parts/hf_hub.py tests/collections/speechlm2/test_hf_hub.py tests/collections/speechlm2/test_to_hf.py
  • python -m black --check examples/speechlm2/to_hf.py nemo/collections/speechlm2/parts/hf_hub.py tests/collections/speechlm2/test_to_hf.py tests/collections/speechlm2/test_hf_hub.py
  • python -m isort --check-only examples/speechlm2/to_hf.py nemo/collections/speechlm2/parts/hf_hub.py tests/collections/speechlm2/test_to_hf.py tests/collections/speechlm2/test_hf_hub.py
  • pytest tests/collections/speechlm2/test_to_hf.py tests/collections/speechlm2/test_hf_hub.py -q

@copy-pr-bot
Copy link
Copy Markdown

copy-pr-bot Bot commented May 29, 2026

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@pzelasko pzelasko requested review from DongjiGao and Edresson May 29, 2026 15:54
@pzelasko
Copy link
Copy Markdown
Collaborator Author

/ok to test f33caf8

Signed-off-by: Piotr Żelasko <pzelasko@nvidia.com>
@pzelasko pzelasko force-pushed the codex/speechlm2-offline-backbone branch from 8be1335 to a8d8276 Compare May 29, 2026 16:19
@pzelasko
Copy link
Copy Markdown
Collaborator Author

/ok to test a8d8276

@github-actions
Copy link
Copy Markdown
Contributor

[🤖]: Hi @pzelasko 👋,

We wanted to let you know that a CICD pipeline for this PR just finished successfully.

So it might be time to merge this PR or get some approvals.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant