fix(vlm): dispatch async GRPO from VLM entrypoint - #3740
Merged
Conversation
Signed-off-by: Youngeun Kwon <youngeunk@nvidia.com>
Contributor
Author
|
/ok to test 7f00f9e |
5 tasks
youngeunkwon0405
left a comment
Contributor
Author
There was a problem hiding this comment.
LGTM — reviewed by a coordinated agent team (rl-expert, test-agent, bug-finder, devil's advocate); zero actionable findings.
Verified:
- The async dispatch block mirrors
run_grpo.pyandrun_grpo_nemo_gym.pyexactly (same 4 guards, same messages, same kwargs). - All 16 kwargs check out against
async_grpo_train; thesetup()13-tuple unpack rename is correct. - Every shipped VLM config chain roots at a yaml that defines
data.use_multiple_dataloaderand theasync_grpoblock, so the new guard reads are safe. processoris genuinely consumed in the async path (trajectory collector + validation) with full parity to the sync trainer — no silent VLM degradation.- Both tests fail on main and pass on this branch, so they pin the fix. Lint (
ruff check+ruff format --check) passes.
Nice preventive fix — before this change an async-enabled VLM config silently ran synchronous training.
Generated by Claude Code
youngeunkwon0405
marked this pull request as ready for review
August 20, 2026 23:35
Contributor
Author
|
/ok to test 7f00f9e |
Contributor
Author
|
/ok to test 7f00f9e |
Contributor
|
yikes! how did we miss this? lgtm! |
rohitrango
approved these changes
Aug 21, 2026
youngeunkwon0405
requested review from
kajalj22
and removed request for
kajalj22
August 21, 2026 02:41
Contributor
Author
|
Hi @terrykong, can we merge this fix, please? It is a very simple, straightforward fix. |
terrykong
approved these changes
Aug 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fixes the VLM GRPO entrypoint so
grpo.async_grpo.enabled=trueselectsasync_grpo_traininstead of silently continuing through the synchronous trainer.Previously,
examples/run_vlm_grpo.pyunconditionally calledgrpo_train. An async VLM configuration could therefore reserve a non-colocated rollout pool while executing synchronous policy updates, and the run would still be labeled async in performance metrics because the trainer received the enabled config flag.The VLM entrypoint now mirrors the established async dispatch in
examples/run_grpo.pyandexamples/nemo_gym/run_grpo_nemo_gym.py:async_grpo_trainwith trajectory-age, teacher-worker, and alias mappings from setup;processorto both trainers; andThis is a framework-only follow-up to review feedback on #3737. It contains no recipe changes.
Issues
None.
Validation
ruff check examples/run_vlm_grpo.py tests/unit/test_config_validation.pyruff format --check examples/run_vlm_grpo.py tests/unit/test_config_validation.pypytest -q --noconftest tests/unit/test_config_validation.py -k 'multimodal_launchers_forward_processor_to_both_trainers or vlm_launcher_dispatches_on_async_grpo_enabled'The focused Linux container run completed successfully (
3 passed; Slurm job6375778,COMPLETED 0:0). The tests verify that the async-enabled branch callsasync_grpo_train, the fallback branch callsgrpo_train, and both VLM trainers receiveprocessor=processor.Before your PR is "Ready for review"
Pre checks: