Skip to content

CLI Reference

Jump to bottom
youngharold edited this page Feb 18, 2026 · 18 revisions

CLI Reference

Global Options

tightwad [OPTIONS] COMMAND
Option Description
-c, --config PATH Path to cluster.yaml (default: configs/cluster.yaml or $TIGHTWAD_CONFIG)

Speculative Decoding Proxy Commands

tightwad proxy start

Start the speculative decoding proxy server. Both draft and target servers must be running.

Single-drafter:

$ tightwad proxy start
Starting speculative decoding proxy...
  Draft:  qwen3-8b @ http://192.168.86.250:8081
  Target: qwen3-32b @ http://192.168.86.36:8080
  Max draft tokens: 32
  Listening on: 0.0.0.0:8088

Multi-drafter:

$ tightwad proxy start
Starting speculative decoding proxy...
  Drafters (3):
    - qwen3-8b @ http://192.168.86.250:8081 (llamacpp)
    - qwen3-1.7b @ http://192.168.86.28:8081 (llamacpp)
    - qwen3:1.7b @ http://192.168.86.86:11434 (ollama)
  Target: qwen3-32b @ http://192.168.86.36:8080
  Max draft tokens: 32
  Listening on: 0.0.0.0:8088

The proxy runs in the foreground (uvicorn). PID is written to ~/.tightwad/proxy.pid.

tightwad proxy stop

Stop the speculative decoding proxy.

$ tightwad proxy stop
Proxy stopped.

tightwad proxy status

Show server health, drafter wins, and speculation stats.

Single-drafter:

$ tightwad proxy status
  ● Draft: qwen3-8b @ http://192.168.86.250:8081
  ● Target: qwen3-32b @ http://192.168.86.36:8080

              Speculation Stats
┏━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Metric           ┃ Value     ┃
┡━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Rounds           │ 39        │
│ Drafted          │ 1247      │
│ Accepted         │ 759       │
│ Acceptance rate  │ 60.9%     │
│ Tokens/round     │ 33.5      │
│ Uptime           │ 120s      │
└──────────────────┴───────────┘

Multi-drafter:

$ tightwad proxy status
  Drafters:
┏━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━┳━━━━━━┓
┃ Model       ┃ URL                           ┃ Backend ┃ Health ┃ Wins ┃
┡━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━╇━━━━━━┩
│ qwen3-8b    │ http://192.168.86.250:8081    │ llamacpp│ alive  │ 6    │
│ qwen3-1.7b  │ http://192.168.86.28:8081     │ llamacpp│ alive  │ 4    │
│ qwen3:1.7b  │ http://192.168.86.86:11434    │ ollama  │ alive  │ 0    │
└─────────────┴───────────────────────────────┴─────────┴────────┴──────┘
  ● Target: qwen3-32b @ http://192.168.86.36:8080

tightwad chat

Interactive chat with the proxy. Also available as a browser UI at http://localhost:8088.

$ tightwad chat
Speculative mode: proxy @ :8088 -> qwen3-32b
Type your message and press Enter. Ctrl+C to quit.

You: What is the capital of France?
AI: The capital of France is Paris.
Option Description
--direct Bypass proxy, chat directly with target (for A/B speed comparison)

The --direct flag is useful for comparing speculative decoding speed against direct target inference.

RPC Cluster Commands

tightwad status

Show full cluster status: coordinator state, worker health, VRAM totals, and tensor split.

$ tightwad status
● Coordinator PID 12345 on :8080
  Health: OK

               RPC Workers
┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━┓
┃ Address             ┃ Status┃ Latency ┃
┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━┩
│ 192.168.86.36:50052 │ alive │ 0.8ms   │
│ 192.168.86.36:50053 │ alive │ 0.9ms   │
└─────────────────────┴───────┴─────────┘

Total VRAM: 76 GB across 4 GPUs
Tensor split: [0.32, 0.32, 0.21, 0.16]
Models: qwen3-72b, deepseek-r1-70b, llama3.3-70b

tightwad start

Start the coordinator llama-server. Checks all RPC workers are healthy before launching.

tightwad start [-m MODEL]
Option Description
-m, --model NAME Model name from config (default: model with default: true)

Prerequisites: All rpc-server instances must be running on workers.

tightwad stop

Stop the coordinator llama-server. RPC workers are not affected.

tightwad stop

tightwad swap MODEL

Hot-swap to a different model. Stops the coordinator and restarts with the new model. RPC workers persist since they're stateless.

tightwad swap deepseek-r1-70b

tightwad benchmark

Run a prompt-processing and text-generation benchmark against the running coordinator.

tightwad benchmark

Measures:

  • Prompt processing speed (tok/s) with ~512 token prompt
  • Text generation speed (tok/s) with 128 token completion

Requires the coordinator to be running.

Manifest Commands

tightwad manifest create

Generate a swarm manifest for a GGUF model file. Splits the file into pieces (default 64 MB) and SHA256-hashes each one. Optionally extracts GGUF metadata (architecture, quantization, parameters).

tightwad manifest create <model.gguf> [--piece-size 64] [--no-inspect] [-o path]
Option Description
--piece-size N Piece size in MB (default: 64)
--no-inspect Skip GGUF metadata extraction
-o, --output PATH Output manifest path (default: <model>.tightwad.manifest) 
$ tightwad manifest create ~/models/Qwen3-32B-Q4_K_M.gguf
⠋ Hashing pieces... ████████████████████ 282/282 pieces

Manifest created: ~/models/Qwen3-32B-Q4_K_M.gguf.tightwad.manifest
  Model:    Qwen3-32B-Q4_K_M
  Size:     18.05 GB
  Pieces:   282 x 64 MB
  Metadata: {"arch": "qwen3", "quantization": "Q4_K_M", "params": "32.8B"}

Swarm P2P Transfer Commands

See Swarm Transfer for full architecture details.

tightwad swarm seed

Start a P2P seeder HTTP server for a model file. If no manifest exists, one is created automatically. Serves pieces to pulling peers.

tightwad swarm seed <model.gguf> [--port 9080] [--host 0.0.0.0]
Option Description
--port N Seeder port (default: 9080)
--host ADDR Bind host (default: 0.0.0.0)

The seeder exposes:

  • GET /manifest — manifest JSON
  • GET /bitfield — {"have": [0, 1, ...]}
  • GET /pieces/<index> — raw piece bytes
  • GET /health — status JSON

PID is written to ~/.tightwad/swarm-<model>.pid.

$ tightwad swarm seed ~/models/Qwen3-32B-Q4_K_M.gguf --port 9080
Starting swarm seeder...
  Model:  Qwen3-32B-Q4_K_M (Qwen3-32B-Q4_K_M.gguf)
  Pieces: 282/282 (100%)
  Listen: 0.0.0.0:9080

tightwad swarm pull

Pull a model from one or more swarm peers. Supports resume — if interrupted, re-running continues from where it left off. Uses rarest-first piece selection across all peers.

tightwad swarm pull <dest.gguf> --manifest <path-or-url> --peer <url> [--peer <url>...] [--parallel 4]
Option Description
--manifest PATH_OR_URL Path or URL to manifest (required)
--peer URL Peer URL, repeatable (required, at least one)
--parallel N Max concurrent downloads (default: 4) 
$ tightwad swarm pull /tmp/Qwen3-32B.gguf \
    --manifest http://192.168.86.36:9080/manifest \
    --peer http://192.168.86.36:9080 \
    --peer http://192.168.86.250:9080
Pulling Qwen3-32B-Q4_K_M.gguf
  Pieces: 282 total, 282 to download
  Peers:  2
  Parallel: 4
⠋ Downloading pieces... ████████████████████ 282/282

Download complete: /tmp/Qwen3-32B.gguf

tightwad swarm status

Show swarm completion status for a model. Verifies pieces on first run if bitfield is empty.

tightwad swarm status <model.gguf>

$ tightwad swarm status ~/models/Qwen3-32B-Q4_K_M.gguf
Swarm Status: Qwen3-32B-Q4_K_M.gguf
  Model:      Qwen3-32B-Q4_K_M
  Size:       18.05 GB
  Pieces:     282 x 64 MB
  Have:       282/282 (100%)
  Complete!
  Metadata:   {"arch": "qwen3", "quantization": "Q4_K_M"}

Clone this wiki locally