Fix: restore requires_grad after _save_converted_model to work around peft inject_adapter side effect

[kiritozc]

Problem

When using PiSSA/OLoRA/LoRA-GA with peft >= 0.18.1, the first save_checkpoint freezes the training adapter's parameters due to a side effect in peft's inject_adapter. After saving, grad_norm drops to 0 and training effectively stops.

Trigger path: _save_converted_model → model.save_pretrained(path_initial_model_for_weight_conversion=...) → save_mutated_as_lora → load_adapter → inject_adapter → freezes active adapter

Root cause: peft 0.18.1 introduced a regression where inject_adapter passes the new adapter's inference_mode=True to set_adapter(self.active_adapters, inference_mode=True), which incorrectly sets requires_grad=False on the training adapter. See huggingface/peft#3290 for the upstream fix.

Fix

Save and restore requires_grad state of all parameters around the save_pretrained call in _save_converted_model. This applies to both the pissa/olora and lora-ga branches.

This is a defensive workaround that protects against the peft side effect without depending on any specific peft version. Once the upstream peft fix is merged and adopted, this workaround becomes a no-op safety net.

Related

• Upstream peft fix: FIX: inject_adapter incorrectly propagates inference_mode to active adapters huggingface/peft#3290
• peft regression introduced in commit 13fa0aea (PR fix app-ui #2765)

[gemini-code-assist]

Code Review

This pull request modifies swift/trainers/mixin.py to save and restore the requires_grad state of model parameters during model conversion, protecting against side effects in peft >= 0.18.1 where active adapters are incorrectly frozen. There are no review comments to evaluate, so no additional feedback is provided.

[Jintao-Huang]

thanks!