fix: pass trust_remote_code=True to remaining AutoConfig.from_pretrained sites#2496
Open
bzantium wants to merge 1 commit into
Open
fix: pass trust_remote_code=True to remaining AutoConfig.from_pretrained sites#2496bzantium wants to merge 1 commit into
bzantium wants to merge 1 commit into
Conversation
…ned call sites Three call sites still invoked AutoConfig.from_pretrained without trust_remote_code=True, which crashes for any model that ships custom modeling code via auto_map in config.json. The rest of the codebase already passes trust_remote_code=True at every from_pretrained site, so this restores consistency. * nemo_rl/algorithms/distillation.py: check_vocab_equality probes both student and teacher configs at setup; the failure is fatal because it happens before any model weights are loaded. * nemo_rl/models/generation/vllm/quantization/fp8.py: init_fp8 probes the model config to decide FP8 plumbing. * nemo_rl/models/megatron/draft/utils.py: build_draft_model probes the draft model config for speculative decoding. Without the kwarg, transformers tries to interactively prompt "Do you wish to run the custom code?" via input(), which raises EOFError inside Ray workers (stdin is closed) and then surfaces the proper ValueError asking for trust_remote_code=True. Signed-off-by: Minho Ryu <ryumin93@gmail.com>
This was referenced May 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do ?
Three call sites in the codebase still invoked
AutoConfig.from_pretrainedwithouttrust_remote_code=True. This crashes for any model that ships custom modeling code viaauto_mapinconfig.json, sincetransformersthen tries to interactively prompt the user, which raisesEOFErrorinside a Ray worker (stdin is closed) and finally surfaces the underlyingValueError:The rest of the codebase already passes
trust_remote_code=Trueat every otherfrom_pretrainedsite (flops_tracker.py,native_checkpoint.py,sglang_worker.py,vllm_worker.py,dtensor_policy_worker.py,automodel/setup.py,huggingface/common.py,megatron/setup.py,megatron/community_import.py,algorithms/utils.py), so this PR just restores consistency at the three remaining sites.Patched sites:
nemo_rl/algorithms/distillation.pycheck_vocab_equalitynemo_rl/models/generation/vllm/quantization/fp8.pyinit_fp8nemo_rl/models/megatron/draft/utils.pybuild_draft_modelThe
distillation.pysite is the one I actually hit in practice (on-policy distillation with a custom-arch HF student that shipsmodeling_*.pyandconfiguration_*.pyviaauto_map). The other two are the same precondition violation in lower-traffic paths and were caught while looking at the surroundingAutoConfig.from_pretrainedusages.Issues
None filed; this is a small consistency fix discovered while debugging a custom-arch distillation run.
Usage
No new API or configuration. After this patch a distillation run with a custom-arch student or teacher, e.g.
proceeds past
check_vocab_equalityinstead of dying at setup.Before your PR is "Ready for review"
Pre checks:
Additional Information
AutoConfig.from_pretrainedand asserts the kwarg is forwarded if the reviewers prefer.check_vocab_equalitypasses and the driver proceeds normally.