Skip to content

Ratchet 0.5.0

Choose a tag to compare

@harshbhx harshbhx released this 25 Oct 18:57
e470cfb

Ratchet is an OpenAI-compatible gateway for license-clean distillation on open models. A small Nemotron 3 Nano 4B student with a LoRA adapter answers first; Nemotron 3 Ultra on Nebius Token Factory is the teacher. Every number below can be recomputed from the saved answers in evidence/ (see docs/VERIFY.md).

Results (measured, strict scoring)

Text-to-SQL (headline)

Untrained Teacher (Ultra) Adapter
Exam 1 (172 questions) 12.8% 25.6% 39.0%
Exam 2 (200 questions, new industries) 12.5% 15.0% 33.0%

At the 95% precision bar the student answers 1 of 86 (exam 1) and 0 of 100 (exam 2) questions by itself, so most traffic still goes to the teacher.

Tool calling (second result)

Untrained Adapter Teacher (Ultra)
Hard exam, 150 questions, 20 unseen tools 62.7% 90.7% 91.3%
BFCL, 300 questions 77.3% 80.3% 88.3% (adapter is lower than Ultra)

Adapters (SHA-256 of adapter_model.safetensors)

Adapter SHA-256
Text-to-SQL, round 1 a883bcf6bbf4152efb5a874520b8fa1a09d9bead55c172507725dc255f18236a
Text-to-SQL, round 2 38f99d441e0527b75fdf8fd9d10acbf4473dcf70db7febe3ad73fccd631a6aa7
Tool calling, round 3 fde47de3913fdf14d29328a174b5a92411c3f058989017c7709b4c572db324b9

What is real, and what is not

  • Adapters were trained on a borrowed GPU, on synthetic data labelled by Nemotron 3 Ultra on Nebius Token Factory (OpenMDW-1.1).
  • Teacher calls run on Nebius Token Factory. The live demo and the simulation run on Cloudflare.
  • The Nebius AI Cloud path (Serverless Endpoint for the student, Serverless Job for Night School, a CPU VM and Object Storage) is scripted and tested offline, ready to run with make nebius-plan and make nebius-up.
  • Simulation numbers are labelled Simulation.
  • Not an official product of Nebius or NVIDIA, and not endorsed by either.

Try it

What is new in 0.5.0

See the changelog: live and simulation demos side by side, a waking-up state for cold starts, Scout example questions and a runs-left counter, a one-command Nebius path (make nebius-plan), evidence, link and secret checks in CI, and post-deploy smoke tests.

License: Apache-2.0 for the code. Model weights and data keep their own licenses.