Align KTO with DPO: Log num_tokens metric

[qgallouedec]

Part of the effort to align KTOTrainer (experimental) with DPOTrainer.

DPOTrainer logs a num_tokens metric (running count of training tokens seen); KTOTrainer did not. Added it for parity:

• Initialize self._total_train_tokens = 0 in __init__ (next to self._metrics).
• Accumulate and log num_tokens in both _compute_loss and _compute_loss_liger, matching DPO.

The counter is only incremented in train mode; the metric is logged in both modes.

No change to the loss or existing metrics.

---

Note

Low Risk
Observability-only metrics wiring with no changes to loss computation or training logic.

Overview
KTOTrainer now reports a running num_tokens training metric so it matches DPOTrainer behavior.

A _total_train_tokens counter is initialized at startup. On each forward in _compute_loss and _compute_loss_liger, training batches add the distributed sum of attention_mask token counts; num_tokens is written into the trainer metrics (train increments the counter; eval logs the current cumulative total without updating it).

No changes to KTO loss or existing reward/KL metrics.

^{Reviewed by Cursor Bugbot for commit 95cdac0. Bugbot is set up for automated code reviews on this repo. Configure here.}

[bot-ci-comment]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.