Align KTO with DPO: Validate Liger kernel configuration before trainer initialization

[albertvillanova]

This PR moves the validation of the use_liger_kernel configuration in the experimental KTOTrainer to before super().__init__(), aligning it with DPOTrainer.

Motivation

The Liger kernel checks (kernel availability, unsupported loss type, precompute_ref_log_probs, and PEFT) depend only on the config and the input model, so there is no reason to run them after the expensive super().__init__() (model loading, dataset preparation, reference model creation, accelerator setup). DPOTrainer already validates these before super().__init__(), while KTOTrainer did it after, so an invalid configuration was rejected only at the end of a costly initialization.

Solution

The validation checks are moved ahead of super().__init__() so an invalid Liger configuration fails fast. The construction of the fused Liger loss stays after super().__init__() because it depends on self.ref_model, which is only resolved during the reference model setup that runs after the parent initialization.

Changes

• Move the Liger kernel validation checks (availability, loss type, precompute_ref_log_probs, PEFT) before super().__init__() in KTOTrainer
• Keep the LigerFusedLinearKTOLoss construction after super().__init__(), where self.ref_model is available
• Use the local model instead of self.model for the PEFT check, since self.model is not yet set before super().__init__()

---

Note

Low Risk
Init-order refactor only; validation rules are unchanged aside from using the local model for the PEFT check before self.model exists.

Overview
KTOTrainer now runs all use_liger_kernel validation (package install, loss type, compute_metrics, precompute_ref_log_probs, PEFT) before super().__init__(), matching DPOTrainer so bad Liger settings fail immediately instead of after model load, dataset prep, and accelerator setup.

LigerFusedLinearKTOLoss is still constructed only after super().__init__() and reference-model setup, with a short comment that it depends on self.ref_model. The PEFT guard uses the local model argument instead of self.model, which is not assigned until the parent init runs.

^{Reviewed by Cursor Bugbot for commit e4364bb. Bugbot is set up for automated code reviews on this repo. Configure here.}

[bot-ci-comment]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[qgallouedec]

in dpo it's

"Liger DPO loss does not support precomputing reference log probabilities. Either disable "
"`precompute_ref_log_probs` or set `use_liger_kernel` to False."

[albertvillanova]

This PR is just a move of the Liger validation checks, to before super().__init__(); no error message modification.

Individual error message alignment is handled in other PR:

• Align KTO with DPO: Align error for Liger with precompute_ref_logps #6270

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.

^{Reviewed by Cursor Bugbot for commit a6c4cb3. Configure here.}