Track 2 is live, frozen, and certified — a direct response to launch feedback that a single model + task risks overfitting techniques to one setup.
The two tracks
| Track | Task · Model | Target | Record to beat |
|---|---|---|---|
| 1 | GSM8K · Qwen/Qwen2.5-1.5B | ≥ 57.0% | 6m 05s (packing + completion-only masking) |
| 2 | SQuAD v1.1 · HuggingFaceTB/SmolLM2-1.7B | ≥ 75.5% EM | 11m 08s (naive baseline — wide open) |
Deliberately different model families and task types: a technique only proves general by winning on both. Same hardware (Modal L40S sandbox), caps (≤30M adapter params), and 3-fresh-seed verification everywhere. Submissions declare track: t1 or track: t2 in config.
Track 2 details
- Target frozen at tag
spec-t2-frozen, calibrated by the same protocol as Track 1 (baseline observed 77.5% EM − ~2pts). - Baseline verified 3/3 fresh seeds: 77.60 / 77.62 / 77.38% EM. One run's wall-clock was inflated by platform storage I/O — disclosed in the verification report and tracked as a protocol observation in #7.
- The tricks that took −49% on Track 1 (sequence packing, completion-only masking) are unclaimed on Track 2 — likely an easy first record for whoever ports them.
Attempting and re-verifying remain free (Modal monthly credits). Rules: TASK.md · CONTRIBUTING.md