Skip to content

[QEff.Finetuning] Added support for SFTTrainer class along with tests#660

Closed
quic-dhirajku wants to merge 60 commits intoquic:ft_experimentalfrom
quic-dhirajku:ft_exp
Closed

[QEff.Finetuning] Added support for SFTTrainer class along with tests#660
quic-dhirajku wants to merge 60 commits intoquic:ft_experimentalfrom
quic-dhirajku:ft_exp

Conversation

@quic-dhirajku
Copy link
Contributor

@quic-dhirajku quic-dhirajku commented Dec 9, 2025

No description provided.

quic-akuruvil
quic-akuruvil reviewed Dec 15, 2025
View reviewed changes
quic-meetkuma
quic-meetkuma requested changes Dec 16, 2025
View reviewed changes
Copy link
Contributor

@quic-meetkuma quic-meetkuma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel some redundant tests can be moved to test_model.py. Rest looks fine.

@quic-swatia quic-swatia mentioned this pull request Dec 24, 2025
Merged
@quic-swatia
Copy link
Contributor

quic-swatia commented Dec 24, 2025

PEFT config should be added here in the BaseTrainer class as it is part of training config. trl repo has also added it in the trainer classes. We should not deviate from that. It will make the code reuse difficult at a later stage. Please refer: https://github.com/huggingface/trl/blob/main/trl/trainer/sft_trainer.py

Please keep the flexibility to accept PEFT techniques/configuration of various kinds and not just LoRA.

@quic-swatia quic-swatia self-requested a review December 24, 2025 12:31
quic-meetkuma
quic-meetkuma requested changes Dec 31, 2025
View reviewed changes
Copy link
Contributor

@quic-meetkuma quic-meetkuma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor comments. Tests look fine and covers most cases. Just check if any validation related tests are missing. E.g. validation of params being done at SFTTrainer or Trainer level implementation which should be added as a test case here. I mean important argument's validation only. There will be extensive validation happening at their side of code but we are bothered about only important params which we are concerned about.

quic-akuruvil
quic-akuruvil reviewed Jan 2, 2026
View reviewed changes
Copy link
Contributor

@quic-akuruvil quic-akuruvil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, please rebase against main, so that ft_experimental branch is aligned with current main.

mohiso22 and others added 14 commits January 2, 2026 11:54
@mohiso22 @quic-hemagnih @quic-dhirajku
[Qwen2_5_vl] - Onboarding Qwen2_5_vl model in QEfficient (quic#560)
924326b 
Signed-off-by: Mohit Soni <mohisoni@qti.qualcomm.com>
Co-authored-by: Hem Agnihotri <hemagnih@qti.qualcomm.com>
Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
@qcdipankar @quic-dhirajku
Olmo2 Bug fix (quic#589)
8c96a4d 
Fixed -10000 with MIN_MASK

Signed-off-by: Dipankar Sarkar <quic_dipankar@quicinc.com>
Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
@smedhe @quic-dhirajku
updated notebooks (quic#543)
7ad6365 
Updated the correct code with updated syntax, removed device_group
parameter in model.compile()

Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
@mohiso22 @quic-dhirajku
Qwen2.5_VL Example Script Update (quic#598)
c9e417a 
Signed-off-by: Mohit Soni <mohisoni@qti.qualcomm.com>
Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
@quic-sanising @quic-dhirajku @quic-hemagnih
Extend On-Device Sampling Support to more Causal Language Models (qui…
d27fe98 
…c#553)

### 📢 Expanded On-Device Sampling Support in QEfficient

Excited to share that **On-Device Sampling**—previously available only
for `LlamaForCausalLM`—is now supported across a broader set of
architectures! This enhancement brings faster, more efficient inference
directly to the QAIC device.

#### ✅ Newly Supported Architectures:
1. `FalconForCausalLM`
2. `GemmaForCausalLM`
3. `GPT2LMHeadModel`
4. `GPTJForCausalLM`
5. `GraniteForCausalLM`
6. `GraniteMoeForCausalLM`
7. `LlamaForCausalLM` (existing)
8. `MptForCausalLM`
9. `Phi3ForCausalLM`
10. `Qwen2ForCausalLM`

#### ⚠️ Architectures Still Pending Support:
1. `GPTBigCodeForCausalLM`
2. `InternVLChatModel`
3. `MistralForCausalLM`
4. `MixtralForCausalLM`
5. `LlamaSwiftKVForCausalLM`
6. `Grok1ModelForCausalLM`

We’re actively working to extend support to these models. Contributions,
feedback, and testing from the community are always welcome to help
accelerate this effort!

---------

Signed-off-by: quic-sanising <quic_sanising@quicinc.com>
Signed-off-by: sanising <sanising@qti.qualcomm.com>
Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
Co-authored-by: sanising <sanising@qti.qualcomm.com>
Co-authored-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
Co-authored-by: Hem Agnihotri <hemagnih@qti.qualcomm.com>
Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
@quic-meetkuma @quic-dhirajku
[QEff. Finetune]: Added fix for pad_to_max_length in tokenization. (q…
120698f 
…uic#599)

Signed-off-by: meetkuma <meetkuma@qti.qualcomm.com>
Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
@quic-mamta @mamtsing
@asmigosw @quic-rishinr @quic-dhirajku
Enable CB for vlms with multiple images and multiple prompts (quic#583)
e8cc0f7 
Signed-off-by: Mamta Singh <mamtsing@qti.qualcomm.com>
Signed-off-by: Rishin Raj <rishinr@qti.qualcomm.com>
Signed-off-by: Asmita Goswami <asmigosw@qti.qualcomm.com>
Signed-off-by: Mohit Soni <mohisoni@qti.qualcomm.com>
Signed-off-by: vbaddi <quic_vbaddi@quicinc.com>
Co-authored-by: Mamta Singh <mamtsing@qti.qualcomm.com>
Co-authored-by: Asmita Goswami <asmigosw@qti.qualcomm.com>
Co-authored-by: Rishin Raj <rishinr@qti.qualcomm.com>
Co-authored-by: Mohit Soni <mohisoni@qti.qualcomm.com>
Co-authored-by: Vinayak Baddi <vbaddi@qti.qualcomm.com>
Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
@mohiso22 @quic-dhirajku
Modeling fix (quic#605)
848dc6e 
Signed-off-by: Mohit Soni <mohisoni@qti.qualcom.com>
Co-authored-by: Mohit Soni <mohisoni@qti.qualcom.com>
Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
@ochougul @vbaddi
@mamtsing @quic-mamta @quic-dhirajku
New PR for GPTOSS decode-only model (quic#603)
171af20 
Signed-off-by: vbaddi <quic_vbaddi@quicinc.com>
Signed-off-by: Onkar Chougule <ochougul@qti.qualcomm.com>
Signed-off-by: Mamta Singh <mamtsing@qti.qualcomm.com>
Signed-off-by: Mamta Singh <168400541+quic-mamta@users.noreply.github.com>
Co-authored-by: Vinayak Baddi <quic_vbaddi@quicinc.com>
Co-authored-by: Vinayak Baddi <vbaddi@qti.qualcomm.com>
Co-authored-by: Mamta Singh <mamtsing@qti.qualcomm.com>
Co-authored-by: Mamta Singh <168400541+quic-mamta@users.noreply.github.com>
Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
@quic-vargupt @abhishek-singh591 @quic-dhirajku
Update Qeff Documentation to indicate vLLM Support in Validated Model…
9b3164e 
…s Page (quic#588)

Signed-off-by: Varun Gupta <vargupt@qti.qualcomm.com>
Co-authored-by: Abhishek Kumar Singh <sabhis@qti.qualcomm.com>
Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
@tchawada @quic-dhirajku
Adding support to load checkpoints from epoch (quic#606)
e6ac655 
Signed-off-by: Tanisha <tchawada@qti.qualcomm.com>
Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
@tchawada @quic-dhirajku
"[QEff. Finetune]: Support for resuming checkpoints using Epoch" (qui…
1d3eebf 
…c#614)

Signed-off-by: Tanisha <tchawada@qti.qualcomm.com>
Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
@abukhoy @quic-dhirajku
[Upgradation]: onnx opset version updated from 13 to 17 (quic#587)
9d53571 
This pull request is created for updating the _onnx opset_ version to 17
from 13.

## Testing
Below are the models I have tested:
### Causal Models
- TinyLlama/TinyLlama-1.1B-Chat-v1.0
- gpt2
- Salesforce/codegen-350M-mono
- microsoft/Phi-3-mini-4k-instruct
- tiiuae/falcon-7b
- Qwen/Qwen2-0.5B
- Qwen/Qwen3-0.6B
- bigcode/starcoder2-3b
- Qwen/Qwen3-30B-A3B-Instruct-2507
- Felladrin/Minueza-32M-Base
- wtang06/mpt-125m-c4
- hakurei/gpt-j-random-tinier
- mistralai/Mixtral-8x7B-Instruct-v0.1
- meta-llama/Llama-3.2-1B
- unsloth/gemma-2b
- unsloth/gemma-2-2b
- TheBloke/TinyLlama-1.1B-Chat-v0.3-AWQ
- TheBloke/Llama-2-7B-GPTQ
- ibm-granite/granite-20b-code-base
- neuralmagic/Llama-3.2-3B-Instruct-FP8
- neuralmagic/Qwen2-0.5B-Instruct-FP8
- ibm-granite/granite-3.1-2b-instruct
- ibm-granite/granite-guardian-3.1-2b
- hpcai-tech/grok-1
- Snowflake/Llama-3.1-SwiftKV-8B-Instruct
- allenai/OLMo-2-0425-1B

### Embedding Models
- BAAI/bge-base-en-v1.5
- BAAI/bge-large-en-v1.5
- BAAI/bge-small-en-v1.5
- intfloat/e5-large-v2
- sentence-transformers/multi-qa-mpnet-base-cos-v1
- ibm-granite/granite-embedding-30m-english
- ibm-granite/granite-embedding-125m-english
- BAAI/bge-reranker-v2-m3
- ibm-granite/granite-embedding-107m-multilingual
- ibm-granite/granite-embedding-278m-multilingual

### Vision Models
- llava-hf/llava-1.5-7b-hf
- OpenGVLab/InternVL2_5-1B
- meta-llama/Llama-3.2-11B-Vision-Instruct
- ibm-granite/granite-vision-3.2-2b
- meta-llama/Llama-4-Scout-17B-16E-Instruct
- google/gemma-3-4b-it

### Audio Models
- openai/whisper-tiny
- openai/whisper-base
- openai/whisper-small
- openai/whisper-medium
- openai/whisper-large
- openai/whisper-large-v3-turbo

---------

Signed-off-by: Abukhoyer Shaik <abukhoye@qti.qualcomm.com>
Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
@abukhoy @quic-dhirajku
[Docs]: Readme Fix (quic#617)
435895f 
Signed-off-by: Abukhoyer Shaik <abukhoye@qti.qualcomm.com>
Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
@ochougul @quic-dhirajku
Disagg hotfix gpt oss (quic#689)
792063f 
Signed-off-by: Onkar Chougule <ochougul@qti.qualcomm.com>
Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
@quic-dhirajku
Copy link
Contributor Author

quic-dhirajku commented Jan 2, 2026

PEFT config should be added here in the BaseTrainer class as it is part of training config. trl repo has also added it in the trainer classes. We should not deviate from that. It will make the code reuse difficult at a later stage. Please refer: https://github.com/huggingface/trl/blob/main/trl/trainer/sft_trainer.py

Please keep the flexibility to accept PEFT techniques/configuration of various kinds and not just LoRA.

It has been addressed in the latest commit.

quic-dhirajku added a commit to quic-dhirajku/efficient-transformers that referenced this pull request Jan 2, 2026
@quic-dhirajku
Added all the changes for Trainer and its tests from PR quic#660.
7c27692 
Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
@quic-dhirajku quic-dhirajku mentioned this pull request Jan 2, 2026
Merged
@quic-dhirajku
Copy link
Contributor Author

quic-dhirajku commented Jan 2, 2026

Created Pr #697 with all the changes done in this PR, along with all the comments being addressed as well. Closing this PR due to the rebase issue.

quic-dhirajku added a commit that referenced this pull request Jan 2, 2026
@quic-dhirajku
Added Trainer classes and tests for FT (#697)
edfef59 
This PR contains all the changes of PR #660 along with all the comments
being addressed. The new PR was created due a rebase issue.

Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
tchawada pushed a commit to tchawada/QEff_tanisha that referenced this pull request Jan 16, 2026
@quic-dhirajku @tchawada
Added Trainer classes and tests for FT (quic#697)
05c9246 
This PR contains all the changes of PR quic#660 along with all the comments
being addressed. The new PR was created due a rebase issue.

Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
tchawada pushed a commit to tchawada/QEff_tanisha that referenced this pull request Feb 4, 2026
@quic-dhirajku @tchawada
Added Trainer classes and tests for FT (quic#697)
ca0c415 
This PR contains all the changes of PR quic#660 along with all the comments
being addressed. The new PR was created due a rebase issue.

Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
tchawada pushed a commit to tchawada/QEff_tanisha that referenced this pull request Feb 4, 2026
@quic-dhirajku @tchawada
Added Trainer classes and tests for FT (quic#697)
3a10a4b 
This PR contains all the changes of PR quic#660 along with all the comments
being addressed. The new PR was created due a rebase issue.

Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
tchawada pushed a commit to tchawada/QEff_tanisha that referenced this pull request Feb 4, 2026
@quic-dhirajku @tchawada
Added Trainer classes and tests for FT (quic#697)
c6755fc 
This PR contains all the changes of PR quic#660 along with all the comments
being addressed. The new PR was created due a rebase issue.

Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
tchawada pushed a commit to tchawada/QEff_tanisha that referenced this pull request Feb 4, 2026
@quic-dhirajku @tchawada
Added Trainer classes and tests for FT (quic#697)
e369475 
This PR contains all the changes of PR quic#660 along with all the comments
being addressed. The new PR was created due a rebase issue.

Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
tchawada pushed a commit to tchawada/QEff_tanisha that referenced this pull request Feb 5, 2026
@quic-dhirajku @tchawada
Added Trainer classes and tests for FT (quic#697)
62a168d 
This PR contains all the changes of PR quic#660 along with all the comments
being addressed. The new PR was created due a rebase issue.

Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
tchawada pushed a commit to tchawada/QEff_tanisha that referenced this pull request Feb 5, 2026
@quic-dhirajku @tchawada
Added Trainer classes and tests for FT (quic#697)
a7cd0d0 
This PR contains all the changes of PR quic#660 along with all the comments
being addressed. The new PR was created due a rebase issue.

Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
quic-swatia pushed a commit to quic-swatia/efficient-transformers that referenced this pull request Feb 9, 2026
@quic-dhirajku
Added Trainer classes and tests for FT (quic#697)
96e4b9d 
This PR contains all the changes of PR quic#660 along with all the comments
being addressed. The new PR was created due a rebase issue.

Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
quic-akuruvil pushed a commit to quic-akuruvil/efficient_transformers that referenced this pull request Feb 9, 2026
@quic-dhirajku @quic-akuruvil
Added Trainer classes and tests for FT (quic#697)
67c6985 
This PR contains all the changes of PR quic#660 along with all the comments
being addressed. The new PR was created due a rebase issue.

Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
quic-akuruvil pushed a commit to quic-akuruvil/efficient_transformers that referenced this pull request Feb 16, 2026
@quic-dhirajku @quic-akuruvil
Added Trainer classes and tests for FT (quic#697)
1f972fd 
This PR contains all the changes of PR quic#660 along with all the comments
being addressed. The new PR was created due a rebase issue.

Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
quic-swatia pushed a commit to quic-swatia/efficient-transformers that referenced this pull request Feb 26, 2026
@quic-dhirajku
Added Trainer classes and tests for FT (quic#697)
4c663e9 
This PR contains all the changes of PR quic#660 along with all the comments
being addressed. The new PR was created due a rebase issue.

Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
smedhe pushed a commit to smedhe/QEff_Sharvari that referenced this pull request Mar 6, 2026
@quic-dhirajku @smedhe
Added Trainer classes and tests for FT (quic#697)
fec4c17 
This PR contains all the changes of PR quic#660 along with all the comments
being addressed. The new PR was created due a rebase issue.

Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
smedhe pushed a commit to smedhe/QEff_Sharvari that referenced this pull request Mar 6, 2026
@quic-dhirajku @smedhe
Added Trainer classes and tests for FT (quic#697)
2fcc91d 
This PR contains all the changes of PR quic#660 along with all the comments
being addressed. The new PR was created due a rebase issue.

Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
smedhe pushed a commit to smedhe/QEff_Sharvari that referenced this pull request Mar 6, 2026
@quic-dhirajku @smedhe
Added Trainer classes and tests for FT (quic#697)
d45dada 
This PR contains all the changes of PR quic#660 along with all the comments
being addressed. The new PR was created due a rebase issue.

Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
smedhe pushed a commit to smedhe/QEff_Sharvari that referenced this pull request Mar 8, 2026
@quic-dhirajku @smedhe
Added Trainer classes and tests for FT (quic#697)
909b1dc 
This PR contains all the changes of PR quic#660 along with all the comments
being addressed. The new PR was created due a rebase issue.

Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
smedhe pushed a commit to smedhe/QEff_Sharvari that referenced this pull request Mar 8, 2026
@quic-dhirajku @smedhe
Added Trainer classes and tests for FT (quic#697)
4493059 
This PR contains all the changes of PR quic#660 along with all the comments
being addressed. The new PR was created due a rebase issue.

Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>

Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
smedhe pushed a commit to smedhe/QEff_Sharvari that referenced this pull request Mar 8, 2026
@quic-dhirajku
Added Trainer classes and tests for FT (quic#697)
7b591ff 
This PR contains all the changes of PR quic#660 along with all the comments
being addressed. The new PR was created due a rebase issue.

Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
smedhe pushed a commit to smedhe/QEff_Sharvari that referenced this pull request Mar 9, 2026
@quic-dhirajku @smedhe
Added Trainer classes and tests for FT (quic#697)
69ab49f 
This PR contains all the changes of PR quic#660 along with all the comments
being addressed. The new PR was created due a rebase issue.

Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
smedhe pushed a commit to smedhe/QEff_Sharvari that referenced this pull request Mar 10, 2026
@quic-dhirajku @smedhe
Added Trainer classes and tests for FT (quic#697)
4aad942 
This PR contains all the changes of PR quic#660 along with all the comments
being addressed. The new PR was created due a rebase issue.

Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
smedhe pushed a commit to smedhe/QEff_Sharvari that referenced this pull request Mar 17, 2026
@quic-dhirajku @smedhe
Added Trainer classes and tests for FT (quic#697)
f87c0a7 
This PR contains all the changes of PR quic#660 along with all the comments
being addressed. The new PR was created due a rebase issue.

Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
smedhe pushed a commit to smedhe/QEff_Sharvari that referenced this pull request Mar 18, 2026
@quic-dhirajku @smedhe
Added Trainer classes and tests for FT (quic#697)
59e93ef 
This PR contains all the changes of PR quic#660 along with all the comments
being addressed. The new PR was created due a rebase issue.

Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@quic-akuruvil quic-akuruvil quic-akuruvil left review comments

@quic-swatia quic-swatia quic-swatia left review comments

@quic-meetkuma quic-meetkuma quic-meetkuma requested changes

@quic-rishinr quic-rishinr Awaiting requested review from quic-rishinr quic-rishinr is a code owner

@ochougul ochougul Awaiting requested review from ochougul ochougul is a code owner

@quic-hemagnih quic-hemagnih Awaiting requested review from quic-hemagnih quic-hemagnih is a code owner

@quic-amitraj quic-amitraj Awaiting requested review from quic-amitraj quic-amitraj is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.