feat: implement NEFTune noisy embeddings for instruction fine-tuning#1686
Merged
akoumpa merged 3 commits intoNVIDIA-NeMo:mainfrom Apr 6, 2026
Merged
feat: implement NEFTune noisy embeddings for instruction fine-tuning#1686akoumpa merged 3 commits intoNVIDIA-NeMo:mainfrom
akoumpa merged 3 commits intoNVIDIA-NeMo:mainfrom
Conversation
Signed-off-by: stanley1208 <stanley.mei08@gmail.com> Made-with: Cursor
stanley1208
requested review from
HuiyingLi,
ZhiyuLi-Nvidia,
adil-a,
akoumpa,
hemildesai and
pthombre
as code owners
Contributor
Author
|
@akoumpa Here's the NEFTune implementation — component + recipe config option + 11 unit tests. Ready for review! |
…oisy output Signed-off-by: stanley1208 <stanley.mei08@gmail.com> Made-with: Cursor
Closed
Contributor
|
/ok to test cffa75e |
Contributor
|
@Stanley thanks a lot, it would be great if you could share a yaml showing how to use this (it's already mentioned to the issue description, what I mean is to include a yaml file to make it "click-and-run"). |
Signed-off-by: stanley1208 <stanley.mei08@gmail.com> Made-with: Cursor
akoumpa
reviewed
Apr 6, 2026
Contributor
akoumpa
approved these changes
Apr 6, 2026
Contributor
akoumpa
left a comment
akoumpa
left a comment
There was a problem hiding this comment.
Thanks a lot @stanley1208
linnanwang
pushed a commit
that referenced
this pull request
Apr 24, 2026
…1686) * feat: implement NEFTune noisy embeddings for instruction fine-tuning Signed-off-by: stanley1208 <stanley.mei08@gmail.com> Made-with: Cursor * fix: correct test_noise_applied_during_training to compare clean vs noisy output Signed-off-by: stanley1208 <stanley.mei08@gmail.com> Made-with: Cursor * feat: add example YAML config for NEFTune fine-tuning Signed-off-by: stanley1208 <stanley.mei08@gmail.com> Made-with: Cursor --------- Signed-off-by: stanley1208 <stanley.mei08@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Implement NEFTune (Noisy Embeddings Fine-Tuning) as a training component with recipe config support.
Fixes #1221.
What is NEFTune?
From the paper: adding uniform random noise to token embeddings during fine-tuning improves instruction following quality, often by a large margin, with no additional compute or data overhead.
Implementation
New component:
nemo_automodel/components/training/neftune.pyNEFTuneclass withactivate(model)/deactivate(model)methodsregister_forward_hookon the model's embedding layeralpha / sqrt(seq_len * hidden_dim)module.training)_get_input_embeddings()helper finds embeddings via HF method or common attribute namesRecipe integration:
nemo_automodel/recipes/llm/train_ft.pyneftuneconfig section insetup()Usage in YAML config: Add
neftune.noise_alpha: 5.0to your training recipe.Tests
tests/unit_tests/training/test_neftune.py_get_input_embeddingshelperRelated
CC @akoumpa