Fix peft_config type hint in experimental trainers

[albertvillanova]

Fix peft_config type hint in experimental trainers.

This PR refactors the PEFT configuration handling in experimental trainer classes (BCOTrainer, CPOTrainer, KTOTrainer, and ORPOTrainer). The main improvement is replacing the previous use of a generic dict for peft_config with the more specific and type-safe PeftConfig from the peft library. This change enhances type safety, improves documentation clarity, and ensures consistency across the codebase.

Changes

PEFT configuration improvements:

• The peft_config parameter in all experimental trainer classes is now typed as PeftConfig | None instead of dict | None, and the corresponding docstrings have been updated to reflect this change. This ensures better type checking and clearer intent for users and developers.

• The import statements in all trainer files have been updated to import PeftConfig from the peft library, alongside the existing PEFT-related imports.

---

Note

Low Risk
Low risk: changes are limited to type hints/docstrings and PEFT imports for peft_config, with no functional logic changes expected beyond stricter typing.

Overview
Updates experimental trainers to treat peft_config as a peft.PeftConfig (instead of a generic dict). This adjusts the __init__ type hints and docstrings across BCO/CPO/ORPO/PRM/NashMD/XPO trainers and adds the needed PeftConfig imports where PEFT is available.

^{Reviewed by Cursor Bugbot for commit a8afd35. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 1ea5f27. Configure here.}

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[qgallouedec]

Nice!