CLI Reference

Global Options

tightwad [OPTIONS] COMMAND

Option	Description
`-c, --config PATH`	Path to cluster.yaml (default: `configs/cluster.yaml` or `$TIGHTWAD_CONFIG`)

Setup

`tightwad init`

Auto-discover inference servers on your LAN and generate a cluster.yaml config interactively.

tightwad init [--subnet 192.168.1.0/24] [--port 8082] [-o configs/cluster.yaml]

Option	Description
`--subnet`	Manual subnet override (auto-detected from your network interface if omitted)
`--port`	Additional ports to scan, repeatable (default: 11434, 8080, 8081)
`-o, --output PATH`	Output config path (default: `configs/cluster.yaml`)

What it does:

Scans your /24 subnet for open ports (11434, 8080, 8081)
Identifies Ollama and llama-server instances
Queries each server for loaded models
Presents a table of discovered servers
Prompts you to pick target (big model) and draft (small fast model)
Generates and writes cluster.yaml

$ tightwad init
Scanning LAN for inference servers...

          Discovered Servers (192.168.86.0/24)
┏━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━┓
┃ # ┃ Host           ┃ Port  ┃ Backend ┃ Models        ┃ Status  ┃
┡━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━┩
│ 1 │ 192.168.86.36  │ 11434 │ ollama  │ qwen3:32b     │ healthy │
│ 2 │ 192.168.86.250 │ 11434 │ ollama  │ qwen3:8b      │ healthy │
│ 3 │ 192.168.86.36  │ 8080  │ llamacpp│ glm-4.7-flash │ healthy │
└───┴────────────────┴───────┴─────────┴───────────────┴─────────┘
Scan completed in 4.2s

Select TARGET server:
  1) 192.168.86.36:11434 (ollama)
  2) 192.168.86.250:11434 (ollama)
  3) 192.168.86.36:8080 (llamacpp)
  Enter number (1-3): 1

Select DRAFT server:
  1) 192.168.86.36:11434 (ollama)
  2) 192.168.86.250:11434 (ollama)
  3) 192.168.86.36:8080 (llamacpp)
  Enter number (1-3): 2

Generated config:
proxy:
  host: 0.0.0.0
  port: 8088
  ...

Write to configs/cluster.yaml? [Y/n] y

Config written to configs/cluster.yaml

Next steps:
  tightwad doctor -c configs/cluster.yaml
  tightwad proxy start -c configs/cluster.yaml

Asks for overwrite confirmation if the output file already exists.

Diagnostics

`tightwad doctor`

Run a full diagnostic check of your cluster setup: configuration, binaries, model files, network connectivity, running services, and version compatibility.

tightwad doctor [--fix] [--json]

Option	Description
`--fix`	Show suggested fix commands for each failure or warning
`--json`	Output machine-readable JSON report (for scripting)

Exit code: 0 if all checks pass, 1 if any check fails.

$ tightwad doctor --fix
──────────────────────────────── Configuration ─────────────────────────────────
  ✓ Config file  /home/user/tightwad/configs/cluster.yaml
  ✓ YAML parse
  ✓ Models defined  2 model(s): qwen3-32b, llama3.3-70b
  ✓ Workers defined  2 worker(s), 2 remote GPU(s)

─────────────────────────────────── Binaries ───────────────────────────────────
  ✓ Coordinator binary  llama-server → /usr/local/bin/llama-server
  ✗ RPC server binary  rpc-server.exe not found
      → Install llama.cpp and ensure 'rpc-server.exe' is in PATH or use an absolute path in config

──────────────────────────────────── Models ────────────────────────────────────
  ○ Model: qwen3-32b  C:/Users/jarvis/models/Qwen3-32B-Q4_K_M.gguf (remote path, not checkable locally)
  ✓ Tensor split  [0.31, 0.23, 0.15, 0.31] (52 GB total across 4 GPUs)

─────────────────────────────────── Network ────────────────────────────────────
  ✓ RPC 192.168.86.250:50052  RTX 2070 — 1.2ms
  ! RPC 192.168.86.28:50052  Apple M2 Metal — 15.7ms (>10ms, WiFi?)
      → Use wired Ethernet for best RPC performance

─────────────────────────────────── Services ───────────────────────────────────
  ✓ Coordinator process  PID 54321
  ✓ Coordinator /health  http://127.0.0.1:8090/health
  ○ Proxy process  Not started (tightwad proxy start)

─────────────────────────────────── Versions ───────────────────────────────────
  ✓ Local llama-server  version: 1.2.3 (abc1234)
  ✓ Worker 192.168.86.250  version: 1.2.3 (abc1234)
  ○ Worker 192.168.86.28  No ssh_user configured — cannot check remote version

1 check(s) failed.  (10 passed, 1 failed, 1 warnings, 3 skipped)
Run with --fix to see suggested fixes.

Check sections (in order):

Section	What it checks
Configuration	Config file exists, YAML parses, models and workers defined, structural validation (port ranges, VRAM positive, valid URLs, backend enum, max_draft_tokens range, duplicate RPC addresses)
Binaries	`coordinator_binary` and `rpc_server_binary` findable via PATH or absolute path
Models	Model files exist locally (Windows paths on Mac → SKIP), tensor split sums to ~1.0
Network	TCP ping each worker RPC port, latency >10ms warns about WiFi
Services	PID files for coordinator/proxy, HTTP health endpoints, draft/target reachability
Versions	Local `llama-server --version`, SSH to workers for remote version, mismatch warnings

JSON output (--json) returns a structured report suitable for CI or monitoring:

tightwad doctor --json | python -m json.tool

{
  "passed": false,
  "sections": [
    {
      "title": "Configuration",
      "results": [
        {"name": "Config file", "status": "pass", "detail": "...", "fix": "", "data": {}}
      ]
    }
  ]
}

Cross-platform notes:

Windows model paths (C:/...) detected on Mac/Linux are reported as SKIP, not FAIL
SSH version checks use BatchMode=yes to avoid hanging on passphrase prompts
Firewall hints are platform-specific (macOS socketfilterfw, Linux ufw)

Config validation checks (reported as WARN, not FAIL):

Check	Rule
Port ranges	`coordinator_port`, all `rpc_port`, `proxy.port` must be 1-65535
VRAM positive	All `vram_gb` values must be > 0
Coordinator binary	Must be a non-empty string
Proxy URLs	Draft and target URLs must be valid `http://` or `https://` URLs
Backend enum	Draft and target backend must be `"llamacpp"` or `"ollama"`
max_draft_tokens	Must be between 1 and 256
Duplicate RPC addresses	Warns if two workers share the same host:port

These checks run after YAML parsing succeeds. They catch "you probably didn't mean this" errors before they cause runtime failures.

Log Commands

`tightwad logs`

View coordinator or proxy logs. Logs are written to ~/.tightwad/logs/.

tightwad logs [coordinator|proxy] [-f] [--clear] [-n LINES]

Option	Description
`coordinator`	View coordinator log (default)
`proxy`	View proxy log
`-f, --follow`	Live-tail the log (like `tail -f`)
`--clear`	Truncate all log files
`-n, --lines N`	Number of lines to show (default: 50)

$ tightwad logs
llama_model_load: loaded meta data with 36 key-value pairs
llama_model_load: model type = 32B
...

$ tightwad logs -f
# (streams new output in real time, Ctrl+C to stop)

$ tightwad logs proxy
INFO:     Started server process [12345]
INFO:     Uvicorn running on http://0.0.0.0:8088

$ tightwad logs --clear
Logs cleared.

Log files:

~/.tightwad/logs/coordinator.log — llama-server stdout/stderr (append mode)
~/.tightwad/logs/proxy.log — uvicorn access and error logs (append mode)

Logs are not rotated. Use tightwad logs --clear to truncate when they get large.

Speculative Decoding Proxy Commands

`tightwad proxy start`

Start the speculative decoding proxy server. Both draft and target servers must be running.

Single-drafter:

$ tightwad proxy start
Starting speculative decoding proxy...
  Draft:  qwen3-8b @ http://192.168.86.250:8081
  Target: qwen3-32b @ http://192.168.86.36:8080
  Max draft tokens: 32
  Listening on: 0.0.0.0:8088

Multi-drafter:

$ tightwad proxy start
Starting speculative decoding proxy...
  Drafters (3):
    - qwen3-8b @ http://192.168.86.250:8081 (llamacpp)
    - qwen3-1.7b @ http://192.168.86.28:8081 (llamacpp)
    - qwen3:1.7b @ http://192.168.86.86:11434 (ollama)
  Target: qwen3-32b @ http://192.168.86.36:8080
  Max draft tokens: 32
  Listening on: 0.0.0.0:8088

The proxy runs in the foreground (uvicorn). PID is written to ~/.tightwad/proxy.pid.

`tightwad proxy stop`

Stop the speculative decoding proxy.

$ tightwad proxy stop
Proxy stopped.

`tightwad proxy status`

Show server health, drafter wins, and speculation stats.

Single-drafter:

$ tightwad proxy status
  ● Draft: qwen3-8b @ http://192.168.86.250:8081
  ● Target: qwen3-32b @ http://192.168.86.36:8080

              Speculation Stats
┏━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Metric           ┃ Value     ┃
┡━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Rounds           │ 39        │
│ Drafted          │ 1247      │
│ Accepted         │ 759       │
│ Acceptance rate  │ 60.9%     │
│ Tokens/round     │ 33.5      │
│ Uptime           │ 120s      │
└──────────────────┴───────────┘

Multi-drafter:

$ tightwad proxy status
  Drafters:
┏━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━┳━━━━━━┓
┃ Model       ┃ URL                           ┃ Backend ┃ Health ┃ Wins ┃
┡━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━╇━━━━━━┩
│ qwen3-8b    │ http://192.168.86.250:8081    │ llamacpp│ alive  │ 6    │
│ qwen3-1.7b  │ http://192.168.86.28:8081     │ llamacpp│ alive  │ 4    │
│ qwen3:1.7b  │ http://192.168.86.86:11434    │ ollama  │ alive  │ 0    │
└─────────────┴───────────────────────────────┴─────────┴────────┴──────┘
  ● Target: qwen3-32b @ http://192.168.86.36:8080

`tightwad chat`

Interactive chat with the proxy. Also available as a browser UI at http://localhost:8088.

In speculative mode, each response shows inline speculation stats — rounds, tokens drafted/accepted, acceptance rate, tokens per round, and throughput:

$ tightwad chat
Speculative mode: proxy @ :8088 -> qwen3-32b
Type your message and press Enter. Ctrl+C to quit.

You: What is the capital of France?
AI: The capital of France is Paris.
  ↳ 3 rounds, 96 drafted, 71 accepted (74.0%), 24.3 tok/round, 18.2 tok/s

You: Explain recursion briefly.
AI: Recursion is when a function calls itself to solve smaller instances of the same problem.
  ↳ 5 rounds, 160 drafted, 142 accepted (88.8%), 28.4 tok/round, 22.1 tok/s

Stats are per-response deltas (not cumulative). If the status endpoint is unreachable, stats are silently skipped.

Option	Description
`--direct`	Bypass proxy, chat directly with target (no stats shown, for A/B speed comparison)

The --direct flag is useful for comparing speculative decoding speed against direct target inference.

RPC Cluster Commands

`tightwad status`

Show full cluster status: coordinator state, worker health, VRAM totals, and tensor split.

$ tightwad status
● Coordinator PID 12345 on :8080
  Health: OK

               RPC Workers
┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━┓
┃ Address             ┃ Status┃ Latency ┃
┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━┩
│ 192.168.86.36:50052 │ alive │ 0.8ms   │
│ 192.168.86.36:50053 │ alive │ 0.9ms   │
└─────────────────────┴───────┴─────────┘

Total VRAM: 76 GB across 4 GPUs
Tensor split: [0.32, 0.32, 0.21, 0.16]
Models: qwen3-72b, deepseek-r1-70b, llama3.3-70b

`tightwad start`

Start the coordinator llama-server. Checks all RPC workers are healthy before launching.

tightwad start [-m MODEL]

Option	Description
`-m, --model NAME`	Model name from config (default: model with `default: true`)

Prerequisites: All rpc-server instances must be running on workers.

`tightwad stop`

Stop the coordinator llama-server. RPC workers are not affected.

tightwad stop

`tightwad swap MODEL`

Hot-swap to a different model. Stops the coordinator and restarts with the new model. RPC workers persist since they're stateless.

tightwad swap deepseek-r1-70b

`tightwad benchmark`

Run a prompt-processing and text-generation benchmark against the running coordinator.

tightwad benchmark

Measures:

Prompt processing speed (tok/s) with ~512 token prompt
Text generation speed (tok/s) with 128 token completion

Requires the coordinator to be running.

Manifest Commands

`tightwad manifest create`

Generate a swarm manifest for a GGUF model file. Splits the file into pieces (default 64 MB) and SHA256-hashes each one. Optionally extracts GGUF metadata (architecture, quantization, parameters).

tightwad manifest create <model.gguf> [--piece-size 64] [--no-inspect] [-o path]

Option	Description
`--piece-size N`	Piece size in MB (default: 64)
`--no-inspect`	Skip GGUF metadata extraction
`-o, --output PATH`	Output manifest path (default: `<model>.tightwad.manifest`)

$ tightwad manifest create ~/models/Qwen3-32B-Q4_K_M.gguf
⠋ Hashing pieces... ████████████████████ 282/282 pieces

Manifest created: ~/models/Qwen3-32B-Q4_K_M.gguf.tightwad.manifest
  Model:    Qwen3-32B-Q4_K_M
  Size:     18.05 GB
  Pieces:   282 x 64 MB
  Metadata: {"arch": "qwen3", "quantization": "Q4_K_M", "params": "32.8B"}

Swarm P2P Transfer Commands

See Swarm Transfer for full architecture details.

`tightwad swarm seed`

Start a P2P seeder HTTP server for a model file. If no manifest exists, one is created automatically. Serves pieces to pulling peers.

tightwad swarm seed <model.gguf> [--port 9080] [--host 0.0.0.0] [--token TOKEN] [--allowed-ips CIDR...]

Option	Description
`--port N`	Seeder port (default: 9080)
`--host ADDR`	Bind host (default: 0.0.0.0)
`--token TOKEN`	Require Bearer token auth on all endpoints
`--allowed-ips CIDR`	Restrict access to these IP ranges (repeatable, e.g. `192.168.86.0/24`)

The seeder exposes:

GET /manifest — manifest JSON
GET /bitfield — {"have": [0, 1, ...]}
GET /pieces/<index> — raw piece bytes
GET /health — status JSON

PID is written to ~/.tightwad/swarm-<model>.pid.

$ tightwad swarm seed ~/models/Qwen3-32B-Q4_K_M.gguf --port 9080
Starting swarm seeder...
  Model:  Qwen3-32B-Q4_K_M (Qwen3-32B-Q4_K_M.gguf)
  Pieces: 282/282 (100%)
  Listen: 0.0.0.0:9080

`tightwad swarm pull`

Pull a model from one or more swarm peers. Supports resume — if interrupted, re-running continues from where it left off. Uses rarest-first piece selection across all peers.

tightwad swarm pull <dest.gguf> --manifest <path-or-url> --peer <url> [--peer <url>...] [--parallel 4] [--token TOKEN]

Option	Description
`--manifest PATH_OR_URL`	Path or URL to manifest (required)
`--peer URL`	Peer URL, repeatable (required, at least one)
`--parallel N`	Max concurrent downloads (default: 4)
`--token TOKEN`	Bearer token for authenticated seeders

$ tightwad swarm pull /tmp/Qwen3-32B.gguf \
    --manifest http://192.168.86.36:9080/manifest \
    --peer http://192.168.86.36:9080 \
    --peer http://192.168.86.250:9080
Pulling Qwen3-32B-Q4_K_M.gguf
  Pieces: 282 total, 282 to download
  Peers:  2
  Parallel: 4
⠋ Downloading pieces... ████████████████████ 282/282

Download complete: /tmp/Qwen3-32B.gguf

`tightwad swarm status`

Show swarm completion status for a model. Verifies pieces on first run if bitfield is empty.

tightwad swarm status <model.gguf>

$ tightwad swarm status ~/models/Qwen3-32B-Q4_K_M.gguf
Swarm Status: Qwen3-32B-Q4_K_M.gguf
  Model:      Qwen3-32B-Q4_K_M
  Size:       18.05 GB
  Pieces:     282 x 64 MB
  Have:       282/282 (100%)
  Complete!
  Metadata:   {"arch": "qwen3", "quantization": "Q4_K_M"}

CLI Reference

CLI Reference

Global Options

Setup

tightwad init

Diagnostics

tightwad doctor

Log Commands

tightwad logs

Speculative Decoding Proxy Commands

tightwad proxy start

tightwad proxy stop

tightwad proxy status

tightwad chat

RPC Cluster Commands

tightwad status

tightwad start

tightwad stop

tightwad swap MODEL

tightwad benchmark

Manifest Commands

tightwad manifest create

Swarm P2P Transfer Commands

tightwad swarm seed

tightwad swarm pull

tightwad swarm status

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Clone this wiki locally

`tightwad init`

`tightwad doctor`

`tightwad logs`

`tightwad proxy start`

`tightwad proxy stop`

`tightwad proxy status`

`tightwad chat`

`tightwad status`

`tightwad start`

`tightwad stop`

`tightwad swap MODEL`

`tightwad benchmark`

`tightwad manifest create`

`tightwad swarm seed`

`tightwad swarm pull`

`tightwad swarm status`