PostTrain Lab

PostTrain Lab is a reproducible experiment harness for comparing supervised fine-tuning (SFT), direct preference optimization (DPO), and group relative policy optimization (GRPO). It treats configuration, dataset lineage, evaluation, and run artifacts as part of the result—not as notebook leftovers.

The core workflow runs without a GPU or third-party Python dependencies. Its deterministic mock backend verifies the complete experiment pipeline. An optional Hugging Face TRL backend performs real training when the training extra and suitable hardware are available.

What it demonstrates

• Algorithm-specific JSONL validation for SFT, DPO, and GRPO.
• Order-independent SHA-256 dataset fingerprints.
• Deterministic train/evaluation splits and content-addressed run IDs.
• Exact-match and token-F1 evaluation with versioned JSON artifacts.
• A static, portable comparison dashboard with no web server or JavaScript dependency.
• Optional lazy-loaded TRL, Transformers, Datasets, PEFT, and Accelerate integration.
• Unit, integration, and CLI end-to-end tests with an enforced 80% coverage floor.

Quick start: no GPU

python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"

posttrain-lab validate configs/sft-demo.toml
make demo
open comparison.html

The demo validates and runs one experiment for each method, then ranks the runs in comparison.html. Mock metrics prove pipeline behavior only; they are not claims about real model quality.

Open the generated sample comparison.

Real TRL training

Install the optional training stack:

pip install -e ".[training]"

Copy a config, change [backend].name from mock to trl, select an appropriate model, and run it on a machine with enough accelerator memory:

posttrain-lab run configs/my-sft-experiment.toml --runs runs

The backend maps methods to SFTTrainer, DPOTrainer, or GRPOTrainer, saves model output to training.output_dir, and records numeric training metrics in the run manifest. Begin with a small model and dataset; real post-training can consume substantial compute.

Dataset contracts

Every non-empty JSONL line must be an object containing non-empty strings:

Method	Required fields	Additional validation
SFT	prompt, response	—
DPO	prompt, chosen, rejected	chosen and rejected must differ
GRPO	prompt, answer	answer is used by the exact-match reward

Do not commit private conversations, credentials, or unlicensed datasets. Publish a dataset card using docs/DATASET_CARD_TEMPLATE.md when you publish results.

Architecture

flowchart LR
  C["TOML experiment config"] --> V["Schema and dataset validation"]
  D["JSONL dataset"] --> V
  V --> F["Fingerprint and deterministic split"]
  F --> B{"Backend"}
  B -->|"No GPU"| M["Deterministic mock"]
  B -->|"Optional"| T["Hugging Face TRL"]
  M --> R["Versioned run artifact"]
  T --> R
  R --> E["Exact match and token F1"]
  E --> H["Static HTML comparison"]

Loading

Commands

# Validate config and dataset without running anything
posttrain-lab validate configs/sft-demo.toml

# Execute an idempotent, content-addressed experiment
posttrain-lab run configs/sft-demo.toml --runs runs

# Compare every completed run in a directory
posttrain-lab compare runs --output comparison.html

# Full local quality gate
make verify

Each run directory contains:

runs/<experiment>-<content-hash>/
├── run.json           # config snapshot, dataset fingerprint, counts, metrics
└── predictions.jsonl  # prompt, prediction, and reference for evaluation

Repeating an unchanged experiment returns the same run ID and does not overwrite a completed artifact. Changing the model, method, dataset contents, seed, hyperparameters, or backend creates a different run ID.

Project layout

posttrain-lab/
├── configs/                 # SFT, DPO, and GRPO examples
├── data/                    # small synthetic demonstration datasets
├── docs/                    # model/dataset card templates and TDD evidence
├── src/posttrain_lab/       # validated experiment engine and CLI
├── tests/                   # unit, integration, and CLI end-to-end tests
├── Dockerfile
├── Makefile
└── pyproject.toml

Current limitations

• The built-in evaluator targets short-answer and deterministic tasks; open-ended generation needs task-specific judges and human review.
• The TRL backend is intentionally thin and does not manage distributed clusters, checkpoint resumption, model publication, or experiment trackers.
• Dataset fingerprinting proves content identity, not data quality or legal permission.
• Real SFT, DPO, and GRPO results are hardware-, package-, and model-dependent. Publish the complete environment and model card with any benchmark claim.

Responsible evaluation and review

Validation and the mock backend are fully local; real TRL runs use locally configured model and dataset paths. Comparison reports expose the exact-match, token-F1, loss, sample count, method, model, and backend used for ranking. The HTML dashboard has expandable evidence and Print/PDF styling, with sibling JSON and CSV exports. No completed runs means no report or ranking—an explicit insufficient-evidence outcome. Rankings are candidates for named human review, never an automatic model-publication decision.

• Privacy and local analysis
• Trainer/reviewer guide
• Deployment runbook
• Counterfactual and fairness checks
• Outcome metrics

License

MIT. The included demonstration datasets are synthetic and released with the project.