v0.3.0 — Clone-to-training setup and integration

Clone-to-training setup and real trainer integration

This release fixes a reproduced Qwen GRPO failure and adds a concrete setup and acceptance workflow.

Start here

git clone https://github.com/manishklach/verifiable-rl-from-base.git
cd verifiable-rl-from-base
bash scripts/setup.sh cuda
source .venv/bin/activate
countdown-smoke --device cuda
bash scripts/run_experiment.sh

bash scripts/quickstart.sh runs all stages. See docs/GETTING_STARTED.md for host prerequisites.

Changes

• Load Qwen3.5 text weights directly, avoiding multimodal position-state errors between generation and smaller GRPO scoring batches.
• Constrain the core runtime and explicitly select PyTorch CPU or CUDA 12.8 wheels.
• Test actual hybrid-Qwen training, checkpoint resume, adapter reload, evaluation, report and plot generation.
• Verify exact text-weight loading and nonzero adapter updates in the tiny CPU integration test.
• Provide a real-model GPU acceptance test using the source dataset and normal rewards.
• Prevent accidental checkpoint overwrites and stop failed CUDA preflight before network access.
• Add single-GPU orchestration, integration CI and setup documentation.

Validation

48 unit/property/regression tests pass. All five CI jobs passed for the implementation: Linux/Windows Python 3.10/3.12 unit/build jobs plus Linux fresh-setup trainer integration. The final follow-up commit corrects README punctuation only. The real 490,364-row source dataset also loaded and split successfully locally.

The CPU integration model is random and tiny. Its explicitly labeled diagnostic reward exercises nonzero gradients; it is not part of real-model training. No CUDA GPU is available for validation, so this release does not claim a tested full-model GPU run, measured VRAM requirements, or improved model accuracy. Run hardware acceptance on the target GPU before a long experiment.

Attached: wheel, source distribution and SHA-256 checksums.

v0.2.0 — Verifiable RL research framework

Verifiable RL from Base v0.2.0

A research framework for training a small Qwen base model on Countdown arithmetic using
GRPO and executable rewards, without a supervised fine-tuning warm start. This first public
release preserves the imported project history and improves experimental correctness,
CPU development, packaging and documentation.

What improved

• Strict balanced answer-tag validation closes stray/nested-tag reward loopholes.
• Reward arrays now reject mismatched completion and task metadata.
• Held-out evaluation deduplicates equivalent puzzles before sample selection.
• pass@k uses the order-independent combinatorial estimator with explicit sample requirements.
• JSONL evaluation reconstructs prompts and IDs and recomputes difficulty from puzzle inputs.
• Evaluation JSON records decoding parameters, source fingerprint and generated token counts.
• Solver inputs are validated and reachable-value cache retention is bounded.
• Config validation catches invalid sizes and profiles before expensive model initialization.
• TRL's obsolete max_prompt_length argument is removed; bf16=false is respected.
• Curriculum checkpoint resume disables skipping batches from the new difficulty stage.
• Optional dependency groups separate CPU development, training, charts and interactive demo.
• Linux/Windows CI covers Python 3.10 and 3.12, tests, package builds and generation smoke runs.

Documentation

Expanded README with CPU and GPU quick starts, dependency matrix, project map and research
limitations. New architecture, evaluation-schema, troubleshooting and validation guides cover
reward contracts, pass@k semantics, solver heuristics, curriculum caveats and migration.
Contributor, extension, GPU and model-card guides remain available.

Validation

48 tests pass locally on Windows/Python 3.11. Ruff, compile checks, pip dependency checks,
source/wheel builds and a deterministic generation smoke run pass. See the repository's CI
for the cross-platform matrix and docs/VALIDATION.md for scope.

Migration from the imported 0.1 code

Install pip install -e ".[train,plot]" for training and charts; .[dev] now supports CPU
contributions without PyTorch. Remove max_prompt_length from custom YAML. Evaluation metrics
are intentionally not directly interchangeable with 0.1 due to deduplication and pass@k fixes.
The exact solver supports one through five nonnegative integer inputs.

Assets and limitations

Attached: Python wheel, source distribution and SHA-256 checksums. Source distribution includes
configuration files, scripts and documentation. Clone the repository for the full workflow.
No trained weights or GPU performance/accuracy results are included. CUDA training, adapter
inference, distributed execution, curriculum resume, Docker and demo need integration validation.
This is a software release, not evidence that RL has improved the configured model.