shellminator

Natural language → bash command, inline in your terminal. A 0.5B model on CPU.

78% on the IBM nl2bash exec benchmark (50 prompts, single-pass greedy) — trained, harvested, and polished for ~$5 of GPU. Runs anywhere.

---

$ sm "copy all jpgs to /mnt/backup"

  cp *.jpg /mnt/backup/
  [⏎ run · r refine · e edit · c cancel]

⏎ Enter runs the command in your shell (cd/export/source persist) · r refines it in a multi-turn loop · e edits it inline before running · c cancels. Obviously destructive commands (rm -rf, mkfs, dd of=/dev/…, fork bombs) trigger a second-Enter confirm.

No torch. No venv. No GPU. One 400 MB GGUF served locally by llama.cpp.

Install

curl -fsSL https://raw.githubusercontent.com/ISB333/shellminator/main/install.sh | bash

That downloads the fine-tuned 0.5B GGUF (~400 MB, one time), a llama-server runtime (~16 MB), and the sm scripts, then hooks one source line into your ~/.bashrc. Restart your shell (or source ~/.bashrc) and go:

sm "show me the 5 largest files in /var"

Requirements: bash or zsh, curl, x86_64 or aarch64 Linux, ~450 MB of disk. The installer detects your shell and hooks ~/.zshrc or ~/.bashrc automatically (override with SHELLMINATOR_RC); other shells: source ~/.shellminator/shellminator.sh from your rc file does the same job.

Benchmarks

Model	IBM nl2bash-eabench bash_1 (50 prompts, single-pass greedy)
Qwen2.5-Coder-0.5B-Instruct (stock)	44%
gemma-3-270M fine-tune (first-gen shellminator)	48%
Qwen2.5-0.5B + SFT (105K exec-verified rows)	72%
Qwen2.5-0.5B + SFT + DPO self-play (804 real pairs) → this model	78%

• Protocol: execution-verified docker sandbox, one greedy pass, no retries, no self-correction.
• The +6 point jump over SFT came from logic-error self-play: the model's own wrong answers, filtered by actually running them, as DPO rejections. Full write-up below.
• Eval docs · Model · Pair dataset

How it works

• sm is a thin shell-free wrapper (bash and zsh safe): the interactive picker runs under bash sm-pick.sh — no regex/readline shell quirks — and the chosen command is eval-ed in YOUR shell, so cd/export/source persist.
• sm-pick.sh holds the whole flow: session, generate, menu, destructive-gate (the ⚠ regexes live in bash, never parsed by the calling shell).
• sm-gen (Python, stdlib only) is the client: system prompt with your cwd/arch/shell, temperature 0, single command, no markdown. Same prompt shape the model was trained on.
• sm-server manages the llama-server daemon: auto-starts on first call, auto-stops after 10 idle minutes. The GGUF carries its own ChatML template — the exact one used in training.
• Destructive commands get a ⚠ and a second-Enter confirm (regex gate, no model involved).

The training story (why 0.5B works)

The model was not just fed more data — more data plateaued at 72%. The jump to 78% came from harvesting the model's own failures:

1. Build a 9,219-prompt pool from the 105K SFT set, weighted toward the fail classes (sed insert/replace distinctions, df mount forms, find/ls counts, comm/diff, mv arg order, uname flags…).
2. Generate the model's greedy answers on Modal A100 (~$2), then run gold and candidate side-by-side in fresh sandboxes — compare exit code, stdout, and the resulting file tree. Keep only real divergences → 804 pairs, zero self-judged.
3. One DPO epoch (26 seconds of A100): chosen = the verified gold command, rejected = what the model actually wrote.

A second round on the remaining 477 disagreements made things worse (74%) — the fail classes left over aren't fixable by preference tuning at this scale. That's the ladder's honest ceiling.

What it won't do

• Multi-command scripts or long pipelines with intermediate variables (it answers a single command by design; you can chain inside refine).
• A few classes survive every lever we threw at them: sed -i '1i …' top-insert loops (it still overwrites — run with ⚠ and check), comm/diff line counting variants, and mv dir1/* dir2/ arg-order semantics. See the R1 eval for the exact residual fail list.

Configuration (env vars)

Var	Default	What
SHELLMINATOR_URL	http://127.0.0.1:47281	server base URL
SHELLMINATOR_PORT	47281	server port
SHELLMINATOR_MODEL	~/.shellminator/model.gguf	GGUF path
SHELLMINATOR_LLAMA_SERVER	set by installer	llama-server binary
SHELLMINATOR_IDLE_TIMEOUT	600	idle seconds before daemon auto-stop
SHELLMINATOR_TEMPERATURE	0	0 keeps generation deterministic
SHELLMINATOR_NO_WARN	unset	disable the destructive-command ⚠

Daemon

sm-server status   # running / stopped
sm-server stop     # stop daemon + watchdog

Reproduce

Everything the model learned lives in public HF repos — datasets, pairs, and weights — and the full recipe is in docs/training.md with the four Modal scripts under training/: pool → harvest → DPO → GGUF. Total compute for the self-play ladder: about $5 on A100. The R1/R2 eval reports are in docs/evals/.

License

MIT — see LICENSE. The fine-tune inherits Apache-2.0 from the Qwen2.5-Coder-0.5B-Instruct base.