Repository navigation
Releases
SHARD v1.9.0 — Distributed AI Computing Platform
SHARD v1.9.0 — Distributed AI Computing Platform
Highlights
- Cluster Coordinator & Worker Lifecycle (
shard_engine/distributed/cluster.py): AddedClusterManager(CLUSTER) withIDLE -> FORMING -> LOADING -> VERIFYING -> READY / DEGRADEDstate machine supporting weak coordinators, explicit worker consent enforcement (HTTP 403without consent), cryptographic worker tokens (sk-shard-wrk-...), heartbeat monitoring, and automatic rebalance/degraded recovery. - PyTorch Pipeline Parallelism (
shard_engine/distributed/pipeline_*.py): Added authenticated binarySHPTTCP tensor transport,PipelineWorkerServerwith selectivesafetensorsshard loading per stage and per-stageDynamicCache, andPipelineCoordinatorsupportingllama,qwen2,qwen3,mistral,gemma,gemma2,phi3, andgpt2decoder architectures. - GGUF Distributed RPC Upgrades (
gguf_backend.py,lan.py): Added--parallel,--cont-batching,--slots,-sm layer,/slotsmonitoring, and worker-t max_threadsquota enforcement. - Partition Planner & Capability Discovery (
planner.py,capabilities.py,hardware.py): Added multi-GPUnvidia-smiprobing, CPU model/cores/utilization detection, backend capability probing, quota bounds (vram_pct,max_ram_gb,max_threads), and communication-aware layer placement. - Reverse Encrypted Worker Tunnel (
net_tunnel.py,net_identity.py,net_strategy.py): AddedMSG_WRK_OPEN/MSG_WRK_DATA/MSG_WRK_CLOSEloopback proxying over ChaCha20-Poly1305 encrypted tunnels for NAT-traversed workers. - Multi-User Async Scheduler & Dashboard/CLI (
scheduler.py,server.py,cli.py,dashboard.html): Non-blockingRequestScheduler(max_concurrent, bounded queue, per-client RPM rate limits, disconnect cancellation, live TTFT / tok/s / inter-worker transfer telemetry),/api/cluster/*,shard cluster ..., and theКластер и Распределениеtab.
v1.8.1: Hardened Private Session Security & PyPI Release
SHARD v1.8.1: Hardened Private Session Security & PyPI Release
Security Hardening
- Server-Side Session Registry: Issues cryptographically secure opaque participant tokens (32-byte urlsafe), stores SHA-256 hashes, enforces TTL expiration and instant revocation.
- Zero-Trust Participant Derivation: Participant identities are derived strictly from validated credentials (LAN pairing token, P2P tunnel peer identity, owner credentials, or validated session token); never trusts arbitrary X-Participant-Id or unknown X-Session-Token.
- NAT / Shared Wi-Fi Isolation: Eliminates silent IP fallback for unauthenticated requests, ensuring multiple guests behind the same NAT receive strictly isolated conversation files.
- Concurrency Protection: Wraps all private chat mutations in storage.file_lock to prevent race conditions during simultaneous participant activity.
- Endpoints: Added POST /api/server/session and POST /api/server/session/revoke.
- Test Suite: Added tests_phase_iii_chat_security.py with 10 dedicated security tests covering spoofing immunity, token revocation, cross-tenant isolation, and concurrent append safety.
Packaging
- Official release artifacts attached: wheel and sdist.
Shard v1.8.0 - server list and open networks
Shard v1.8.0 - server list and open networks (LAN sharding, Phase 1).
The headline flow: you create a network (optionally an open one) on your PC; every other shard instance on the LAN / Radmin VPN sees your machine in the "Серверы рядом" list on its dashboard - with name, IP, loaded model and an open badge - and clicks «Войти на сервер». It joins in one click, sharing its VRAM/RAM by default, and runs its part of the model through llama.cpp RPC. Your machines become one big server for one model.
Added
- Server list: every shard instance now listens for UDP beacons permanently (even with mode off), so other people's masters show up in the "Сеть" tab with name, IP:port, loaded model and an open/closed badge.
- Open networks ("открытая сеть"): create with the checkbox /
shard lan create --open/ toggle "Открыть/Закрыть сеть" - joining skips the pairing code entirely. Closed networks keep theXXXX-XXXXcode requirement (403 "Сеть закрыта: нужен код подключения"). - One-click join: green "Войти на сервер" button on open networks posts join with
share: true- your machine becomes a compute worker automatically. Ifggml-rpc-server.exeis missing, it gracefully falls back to plain member mode (warning shown). - Beacons announce the server's loaded model name; the server list survives
shard lan off. - 11 new LAN tests (80 total, all green).
Notes (honest limits)
- Same trust model as v1.7.0:
/v1/*requires member tokens from non-loopback clients;/api/*(dashboard + LAN management) stays trusted-LAN. Open networks lower the join barrier on purpose - enable them only on a network you trust (home LAN / your Radmin VPN room). - Discovery covers the same subnet and usually Radmin VPN / ZeroTier; Tailscale has no L2 broadcast - its users join by manual IP (that path always works). No NAT traversal.
Binaries
The zip contains the same prebuilt native GUI (shard.exe, shard-native.exe, shard_core.dll) as v1.6.0 - the native core is unchanged in v1.8.0 (all changes are in the Python side). Build from source if you prefer: python native/build_native.py.
Full changelog: CHANGELOG.md.
Shard v1.7.0 - LAN sharding: several PCs as one big server
Shard v1.7.0 - LAN sharding: several machines as one big server
Несколько машин в локальной сети работают как один большой сервер для одной модели: мастер держит API и модель, воркеры отдают VRAM/RAM через llama.cpp RPC. Модель, которую fit-advisor отказал на одной машине, может влезть в объединённый пул памяти. Для сетей через интернет дополнительный код не нужен (Radmin VPN / ZeroTier / Tailscale).
Added
- LAN sharding (Phase 1): the master hosts the API and the model; workers share their VRAM/RAM through llama.cpp RPC (
ggml-rpc-server.exe, bundled with the runtime already in~/.shard/runtime), so models the fit advisor refuses on one machine can fit the pooled memory (same per-device reserves as the advisor: 400 MB VRAM / 2048 MB RAM;nglforced to 999 when pooled). - UDP discovery (port
8001, announce every 2 s, peer TTL 8 s),XXXX-XXXXpairing codes (no ambiguous chars), member tokens (sk-shard-lan-+ 16 hex), idempotent re-pair per IP, 1 s worker heartbeats with auto-drop after 3 consecutive misses, and the kill switchshard lan off(drops every role, stops the RPC process and threads). - Dashboard tab "Сеть": create a network, join with IP + pairing code (+ "стать вычислительным узлом"), live status with member list (green/red dots), nearby peers, pairing-code copy, and the live
--rpc/-tsvalues the master will launch llama-server with. - CLI
shard lan [status|create|join|off]- HTTP proxy to a running Shard Server (SHARD_BASE_URLorhttp://localhost:8000), RU output. /api/lan/*endpoints (create, join, off, status, pair, heartbeat, leave) and an auth gate on/v1/*: while the machine is a LAN server, non-loopback clients must present a member token.- 19 LAN unit tests (69 total): device budgets, split weights, pairing/heartbeat state machine, token-leak prevention, pooled-fit integration.
Honest limits of this phase
/api/*(dashboard + LAN management) is trusted-LAN - only/v1/*requires member tokens from non-loopback clients.- No NAT traversal - for internet use run the machines in Radmin VPN / ZeroTier / Tailscale and use those IPs.
-tsdevice order follows documented llama.cpp behavior and is not yet verified against live weight placement (Phase 1.1).
Binaries
shard-v1.7.0-windows-x64.zip - same prebuilt Windows x64 binaries as v1.6.0 (shard.exe, shard-native.exe, shard_core.dll); the native core did not change in this release, only Python code (verified unchanged upstream). Extract and run shard.exe start.
Full details: docs/LAN_SHARDING_DESIGN.md
Shard v1.6.0 - first prebuilt release
Shard v1.6.0 — first prebuilt release
Windows x64 binaries are attached as shard-v1.6.0-windows-x64.zip. The desktop app auto-starts the local server and opens the dashboard in your browser. The repo plus pip install -e . are still required (the app drives a Python server); MSVC Build Tools are not.
RU: первый релиз с готовыми бинарниками — распакуйте shard.exe + shard_core.dll в корень клонированного репозитория, установите зависимости (pip install -e .) и запустите двойным кликом: программа сама поднимет сервер и откроет дашборд.
What's new
shard startnow opens the browser automatically (parity withpython main.py). Explicit--host/--portflags take priority overSHARD_HOST/SHARD_PORT;SHARD_DOCKER=1still disables auto-open.- Restored native sources (
shard_core.cpp,shard_gui.cpp) that were accidentally committed empty in v1.5.0. Verified: rebuilt binaries match the last good sizes byte-for-byte, the 98-kernel engine (v1.4.79 core) loads and benchmarks.
What's in the zip
| File | Size | Purpose |
|---|---|---|
shard.exe |
540 KB | Desktop app: auto-starts server + dashboard |
shard-native.exe |
540 KB | Same app under its build name |
shard_core.dll |
148 KB | C++20/MASM AVX2 native engine (98 kernels) |
How to run
git clone https://github.com/avelone123123/shard.git
cd shard
pip install -e .Unzip the download into the repo root and double-click shard.exe — or skip the GUI entirely and run shard start.
Honest notes
- Speed figures from the fit advisor are estimates computed from hardware profile, not measurements.
- The native core is optional; quantized models run through the GGUF (llama.cpp) backend.
- CLI output mixes English and Russian — known cosmetic issue, not a bug.
Full Changelog: v1.5.1...v1.6.0