Roadmap

Roadmap

Priority 1 — Stabilize the implemented classifier

• Add a deterministic data-only smoke test and training integration test in a supported environment.
• Pin working dependency versions in a lockfile and document them.
• Add a held-out, representative support-ticket dataset with de-identification.
• Record results across multiple seeds with uncertainty estimates.

Priority 2 — Repair the Phi-2 design

• Correct syntax and indentation of DPO_SFT.py.
• Validate tokenizer formatting and LoRA target modules for Phi-2.
• Migrate SFT/DPO trainer calls to an explicitly tested TRL version.
• Add tests of DPO reference-policy configuration, save/reload and held-out preference evaluation.
• Only then report actual model outputs, memory use, and training metrics.

Priority 3 — Reproducibility and release hygiene

• Add CI, security scans, README diagrams, model cards and data provenance.
• Evaluate whether PPO or ORPO should become a real separate experiment; do not claim implementation until trained and tested.
• Document licensing for original code, pretrained checkpoints and sample datasets.