Testing

Validation and Testing

Offline validation

python qwen_alien_romulus_finetune.py --validate-data
python LLM_Training_Fixed_v2.py --mode validate
python -m compileall -q .

The clean Qwen CLI path does not need PyTorch to validate JSONL. The legacy validate mode checks embedded example counts and structure before loading its external ML stack.

Integration tests not performed

• Loading real Qwen/SmolLM/Phi model weights;
• End-to-end training and optimizer steps on CUDA;
• Saved adapter reload or accuracy/perplexity metrics;
• Google Colab compatibility;
• Distribution across multiple GPUs.

What to measure for production readiness

Validation loss on a held-out split, preference win rate, factuality, prompt/completion token masking, GPU peak memory, inference latency, reproducibility and security of outputs. Add tests for adapter reload and known generation cases.