Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize rl #2065

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

Optimize rl #2065

Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
b6aff05
Update RMS LayerNorm implementation with optimizations and testing suite
NinoRisteski Mar 11, 2025
4abc76d
perf: optimize list comprehension in get_ollama_eos_tokens
NinoRisteski Mar 11, 2025
2772236
Merge branch 'main' of https://github.com/NinoRisteski/unsloth into main
NinoRisteski Mar 17, 2025
dec740e
refactor: Improve code quality in llama.py and vision.py - Refactor e…
NinoRisteski Mar 17, 2025
9712a46
optimize: Improve performance in RL module by optimizing regex and st…
NinoRisteski Mar 17, 2025
3edfeca
Fix embeddings in for_training to check if methods exist
NinoRisteski Mar 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
optimize: Improve performance in RL module by optimizing regex and st… 
…ring operations
  • Loading branch information
@NinoRisteski
NinoRisteski committed Mar 17, 2025
commit 9712a46e44dd6ea2889039616c927b0fed742d7a
12 changes: 8 additions & 4 deletions unsloth/models/rl.py
View file
Original file line number Diff line number Diff line change
@@ -576,10 +576,14 @@ def patch_functions(RLTrainer, trainer_file, RLTrainer_name, all_imports, import
old_init = init

# Remove peft_config
init = init.replace("elif peft_config is None:", "elif False:")
init = init.replace("elif peft_config is not None:", "elif False:")
init = init.replace("if peft_config is None:", "if False:")
init = init.replace("if peft_config is not None:", "if False:")
replacements = {
"elif peft_config is None:": "elif False:",
"elif peft_config is not None:": "elif False:",
"if peft_config is None:": "if False:",
"if peft_config is not None:": "if False:",
}
for old, new in replacements.items():
init = init.replace(old, new)
init = init.replace("get_peft_model(model, peft_config)", "model")

# Set use_vllm if not set