Skip to content

Swarm Transfer

Jump to bottom
youngharold edited this page Feb 18, 2026 · 2 revisions

Swarm Transfer

BitTorrent-style P2P model distribution for Tightwad clusters. Instead of single-source rsync (tightwad distribute), the swarm protocol splits GGUF files into 64 MB pieces with SHA256 hashes and lets peers pull from any peer that has pieces.

Why Swarm?

tightwad distribute uses rsync — one source machine pushes to every worker sequentially. A 40 GB model to 5 workers = 200 GB of outbound transfer from one machine.

With swarm transfer, workers pull from all available peers simultaneously. As soon as a worker has some pieces, it can seed them to others. The load is distributed across the network.

rsync (single-source):                swarm (P2P):

  Source ──► Worker 1 (40 GB)           Source ──► Worker 1 ──► Worker 3
  Source ──► Worker 2 (40 GB)           Source ──► Worker 2 ──► Worker 4
  Source ──► Worker 3 (40 GB)           Worker 1 ──► Worker 5
  Source ──► Worker 4 (40 GB)           Worker 2 ──► Worker 5
  Source ──► Worker 5 (40 GB)
  Total: 200 GB from source            Total: 80 GB from source (peers share the rest)

How It Works

1. Manifest Generation

A manifest describes a model file as a sequence of pieces:

model.gguf (18 GB)
  ├── Piece 0:  offset=0,          size=64MB, sha256=a1b2c3...
  ├── Piece 1:  offset=67108864,   size=64MB, sha256=d4e5f6...
  ├── Piece 2:  offset=134217728,  size=64MB, sha256=789abc...
  ...
  └── Piece 281: offset=18052...   size=12MB, sha256=def012...

The manifest also includes GGUF metadata (architecture, quantization, parameter count) if the gguf package is available. Manifest files are saved alongside the model:

~/models/
  Qwen3-32B-Q4_K_M.gguf                    # the model
  Qwen3-32B-Q4_K_M.gguf.tightwad.manifest  # piece hashes (JSON)
  Qwen3-32B-Q4_K_M.gguf.tightwad.pieces    # bitfield (which pieces we have)

2. Seeder

A seeder serves pieces over HTTP (Starlette/uvicorn). Any machine with a complete or partial model can seed.

Endpoints:

Endpoint Method Description
/manifest GET Full manifest JSON
/bitfield GET {"have": [0, 1, 2, ...]}
/pieces/{index} GET Raw piece bytes (404 if not available)
/health GET Status: model name, completion %, uptime

Default port: 9080. PID file: ~/.tightwad/swarm-<model>.pid.

3. Puller

The puller downloads missing pieces from peers. Key behaviors:

  • Rarest-first selection: Pieces available from fewer peers are downloaded first (reduces risk of a peer going offline before rare pieces are fetched)
  • Load balancing: Among peers with equal piece availability, the peer with fewest active downloads is chosen
  • SHA256 verification: Every downloaded piece is hash-verified before writing to disk. Corrupted pieces are rejected
  • Sparse file pre-allocation: The destination file is pre-allocated on first run to avoid fragmentation
  • Resumable: The bitfield (*.tightwad.pieces) is saved after every successful piece. Interrupting and re-running continues from where it left off
  • Concurrent downloads: Up to 4 parallel piece downloads by default (configurable with --parallel)

4. Bitfield Tracking

Each machine tracks which pieces it has in a .tightwad.pieces file:

{"have": [0, 1, 2, 3, 5, 7, 10]}

This enables:

  • Resume: Re-running a pull only downloads missing pieces
  • Partial seeding: A machine with 50% of pieces can still seed those pieces to others
  • Integrity verification: tightwad swarm status verifies all pieces against the manifest

Security

Swarm transfers run over plain HTTP on your LAN. Two optional security layers protect against unauthorized access:

Bearer Token Auth (--token)

All seeder endpoints require a Bearer <token> header. Pullers must pass the same token. This prevents anyone on your LAN from pulling model files without authorization.

# Seeder
tightwad swarm seed ~/models/Qwen3-32B.gguf --token mysecrettoken

# Puller (same token)
tightwad swarm pull ~/models/Qwen3-32B.gguf \
  --manifest http://192.168.86.36:9080/manifest \
  --peer http://192.168.86.36:9080 \
  --token mysecrettoken

Without the correct token, all requests return 401 Unauthorized.

IP Allowlist (--allowed-ips)

Restrict which IPs can connect to the seeder. Accepts CIDR notation. Multiple entries allowed.

# Only allow your known workers
tightwad swarm seed ~/models/Qwen3-32B.gguf \
  --allowed-ips 192.168.86.0/24

# Multiple ranges
tightwad swarm seed ~/models/Qwen3-32B.gguf \
  --allowed-ips 192.168.86.28/32 \
  --allowed-ips 192.168.86.250/32 \
  --token mysecrettoken

Requests from non-matching IPs return 403 Forbidden. Token auth is checked first (if configured), then IP filtering.

Middleware Stack

Both security layers are implemented as Starlette ASGI middleware, wrapping the seeder app:

Incoming request → TokenAuthMiddleware → IPFilterMiddleware → Seeder App

If neither --token nor --allowed-ips is set, the seeder is open (backward compatible).

Usage

Distribute a model to your cluster

On the source machine:

# 1. Generate manifest (once per model)
tightwad manifest create ~/models/Qwen3-32B-Q4_K_M.gguf

# 2. Start seeding (with optional auth)
tightwad swarm seed ~/models/Qwen3-32B-Q4_K_M.gguf --port 9080 \
  --token mysecrettoken --allowed-ips 192.168.86.0/24

On each worker:

# 3. Pull the model (pass token if seeder requires it)
tightwad swarm pull ~/models/Qwen3-32B-Q4_K_M.gguf \
  --manifest http://192.168.86.36:9080/manifest \
  --peer http://192.168.86.36:9080 \
  --token mysecrettoken

# 4. Start seeding too (optional, helps other workers)
tightwad swarm seed ~/models/Qwen3-32B-Q4_K_M.gguf --port 9080 --token mysecrettoken

Multi-peer pull (faster):

# Worker 3 pulls from both the source AND worker 1
tightwad swarm pull ~/models/Qwen3-32B-Q4_K_M.gguf \
  --manifest http://192.168.86.36:9080/manifest \
  --peer http://192.168.86.36:9080 \
  --peer http://192.168.86.250:9080 \
  --parallel 8 \
  --token mysecrettoken

Check status

tightwad swarm status ~/models/Qwen3-32B-Q4_K_M.gguf

File Naming

All swarm files are appended to the model filename:

File Purpose
model.gguf.tightwad.manifest Piece hashes + metadata (JSON)
model.gguf.tightwad.pieces Bitfield tracking which pieces are present

Architecture

manifest.py
  ├── PieceInfo          — index, offset, size, sha256
  ├── SwarmManifest      — model metadata + piece list, save/load/find
  ├── PieceBitfield      — tracks which pieces are present, save/load
  ├── create_manifest()  — stream file, hash each piece
  └── verify_piece()     — read piece from disk, check SHA256

swarm_transfer.py
  ├── Seeder (Starlette) — /manifest, /bitfield, /pieces/{n}, /health
  ├── SwarmPuller        — async download engine
  │   ├── discover_peer_bitfields()  — query all peers
  │   ├── _select_piece_order()      — rarest-first, load-balanced
  │   ├── download_piece()           — fetch, verify SHA256, write
  │   └── run()                      — full pull loop
  ├── run_seeder()       — sync wrapper for CLI
  └── run_puller()       — sync wrapper for CLI

cli.py
  ├── tightwad manifest create  — generate manifest with progress bar
  ├── tightwad swarm seed       — start seeder server
  ├── tightwad swarm pull       — pull from peers with progress bar
  └── tightwad swarm status     — show completion status

Future: Phase 2

Phase 2 will add:

  • Auto-discovery: Seeders announce via mDNS/UDP broadcast, pullers find peers automatically
  • Tracker: Central tracker for peer coordination across subnets
  • Bandwidth limits: Rate limiting per peer to avoid saturating links
  • Integrity re-check: tightwad swarm verify to re-hash all pieces and report corruption

Clone this wiki locally