Skip to content

Releases

SHARD v1.9.0 — Distributed AI Computing Platform

Choose a tag to compare

@avelone123123 avelone123123 released this 17 Nov 02:01

SHARD v1.9.0 — Distributed AI Computing Platform

Highlights

  • Cluster Coordinator & Worker Lifecycle (shard_engine/distributed/cluster.py): Added ClusterManager (CLUSTER) with IDLE -> FORMING -> LOADING -> VERIFYING -> READY / DEGRADED state machine supporting weak coordinators, explicit worker consent enforcement (HTTP 403 without consent), cryptographic worker tokens (sk-shard-wrk-...), heartbeat monitoring, and automatic rebalance/degraded recovery.
  • PyTorch Pipeline Parallelism (shard_engine/distributed/pipeline_*.py): Added authenticated binary SHPT TCP tensor transport, PipelineWorkerServer with selective safetensors shard loading per stage and per-stage DynamicCache, and PipelineCoordinator supporting llama, qwen2, qwen3, mistral, gemma, gemma2, phi3, and gpt2 decoder architectures.
  • GGUF Distributed RPC Upgrades (gguf_backend.py, lan.py): Added --parallel, --cont-batching, --slots, -sm layer, /slots monitoring, and worker -t max_threads quota enforcement.
  • Partition Planner & Capability Discovery (planner.py, capabilities.py, hardware.py): Added multi-GPU nvidia-smi probing, CPU model/cores/utilization detection, backend capability probing, quota bounds (vram_pct, max_ram_gb, max_threads), and communication-aware layer placement.
  • Reverse Encrypted Worker Tunnel (net_tunnel.py, net_identity.py, net_strategy.py): Added MSG_WRK_OPEN / MSG_WRK_DATA / MSG_WRK_CLOSE loopback proxying over ChaCha20-Poly1305 encrypted tunnels for NAT-traversed workers.
  • Multi-User Async Scheduler & Dashboard/CLI (scheduler.py, server.py, cli.py, dashboard.html): Non-blocking RequestScheduler (max_concurrent, bounded queue, per-client RPM rate limits, disconnect cancellation, live TTFT / tok/s / inter-worker transfer telemetry), /api/cluster/*, shard cluster ..., and the Кластер и Распределение tab.

v1.8.1: Hardened Private Session Security & PyPI Release

Choose a tag to compare

@avelone123123 avelone123123 released this 15 Nov 18:42

SHARD v1.8.1: Hardened Private Session Security & PyPI Release

Security Hardening

  • Server-Side Session Registry: Issues cryptographically secure opaque participant tokens (32-byte urlsafe), stores SHA-256 hashes, enforces TTL expiration and instant revocation.
  • Zero-Trust Participant Derivation: Participant identities are derived strictly from validated credentials (LAN pairing token, P2P tunnel peer identity, owner credentials, or validated session token); never trusts arbitrary X-Participant-Id or unknown X-Session-Token.
  • NAT / Shared Wi-Fi Isolation: Eliminates silent IP fallback for unauthenticated requests, ensuring multiple guests behind the same NAT receive strictly isolated conversation files.
  • Concurrency Protection: Wraps all private chat mutations in storage.file_lock to prevent race conditions during simultaneous participant activity.
  • Endpoints: Added POST /api/server/session and POST /api/server/session/revoke.
  • Test Suite: Added tests_phase_iii_chat_security.py with 10 dedicated security tests covering spoofing immunity, token revocation, cross-tenant isolation, and concurrent append safety.

Packaging

  • Official release artifacts attached: wheel and sdist.

Shard v1.8.0 - server list and open networks

Choose a tag to compare

@avelone123123 avelone123123 released this 11 Nov 05:58

Shard v1.8.0 - server list and open networks (LAN sharding, Phase 1).

The headline flow: you create a network (optionally an open one) on your PC; every other shard instance on the LAN / Radmin VPN sees your machine in the "Серверы рядом" list on its dashboard - with name, IP, loaded model and an open badge - and clicks «Войти на сервер». It joins in one click, sharing its VRAM/RAM by default, and runs its part of the model through llama.cpp RPC. Your machines become one big server for one model.

Added

  • Server list: every shard instance now listens for UDP beacons permanently (even with mode off), so other people's masters show up in the "Сеть" tab with name, IP:port, loaded model and an open/closed badge.
  • Open networks ("открытая сеть"): create with the checkbox / shard lan create --open / toggle "Открыть/Закрыть сеть" - joining skips the pairing code entirely. Closed networks keep the XXXX-XXXX code requirement (403 "Сеть закрыта: нужен код подключения").
  • One-click join: green "Войти на сервер" button on open networks posts join with share: true - your machine becomes a compute worker automatically. If ggml-rpc-server.exe is missing, it gracefully falls back to plain member mode (warning shown).
  • Beacons announce the server's loaded model name; the server list survives shard lan off.
  • 11 new LAN tests (80 total, all green).

Notes (honest limits)

  • Same trust model as v1.7.0: /v1/* requires member tokens from non-loopback clients; /api/* (dashboard + LAN management) stays trusted-LAN. Open networks lower the join barrier on purpose - enable them only on a network you trust (home LAN / your Radmin VPN room).
  • Discovery covers the same subnet and usually Radmin VPN / ZeroTier; Tailscale has no L2 broadcast - its users join by manual IP (that path always works). No NAT traversal.

Binaries

The zip contains the same prebuilt native GUI (shard.exe, shard-native.exe, shard_core.dll) as v1.6.0 - the native core is unchanged in v1.8.0 (all changes are in the Python side). Build from source if you prefer: python native/build_native.py.

Full changelog: CHANGELOG.md.

Shard v1.7.0 - LAN sharding: several PCs as one big server

Choose a tag to compare

@avelone123123 avelone123123 released this 11 Nov 05:14

Shard v1.7.0 - LAN sharding: several machines as one big server

Несколько машин в локальной сети работают как один большой сервер для одной модели: мастер держит API и модель, воркеры отдают VRAM/RAM через llama.cpp RPC. Модель, которую fit-advisor отказал на одной машине, может влезть в объединённый пул памяти. Для сетей через интернет дополнительный код не нужен (Radmin VPN / ZeroTier / Tailscale).

Added

  • LAN sharding (Phase 1): the master hosts the API and the model; workers share their VRAM/RAM through llama.cpp RPC (ggml-rpc-server.exe, bundled with the runtime already in ~/.shard/runtime), so models the fit advisor refuses on one machine can fit the pooled memory (same per-device reserves as the advisor: 400 MB VRAM / 2048 MB RAM; ngl forced to 999 when pooled).
  • UDP discovery (port 8001, announce every 2 s, peer TTL 8 s), XXXX-XXXX pairing codes (no ambiguous chars), member tokens (sk-shard-lan- + 16 hex), idempotent re-pair per IP, 1 s worker heartbeats with auto-drop after 3 consecutive misses, and the kill switch shard lan off (drops every role, stops the RPC process and threads).
  • Dashboard tab "Сеть": create a network, join with IP + pairing code (+ "стать вычислительным узлом"), live status with member list (green/red dots), nearby peers, pairing-code copy, and the live --rpc/-ts values the master will launch llama-server with.
  • CLI shard lan [status|create|join|off] - HTTP proxy to a running Shard Server (SHARD_BASE_URL or http://localhost:8000), RU output.
  • /api/lan/* endpoints (create, join, off, status, pair, heartbeat, leave) and an auth gate on /v1/*: while the machine is a LAN server, non-loopback clients must present a member token.
  • 19 LAN unit tests (69 total): device budgets, split weights, pairing/heartbeat state machine, token-leak prevention, pooled-fit integration.

Honest limits of this phase

  • /api/* (dashboard + LAN management) is trusted-LAN - only /v1/* requires member tokens from non-loopback clients.
  • No NAT traversal - for internet use run the machines in Radmin VPN / ZeroTier / Tailscale and use those IPs.
  • -ts device order follows documented llama.cpp behavior and is not yet verified against live weight placement (Phase 1.1).

Binaries

shard-v1.7.0-windows-x64.zip - same prebuilt Windows x64 binaries as v1.6.0 (shard.exe, shard-native.exe, shard_core.dll); the native core did not change in this release, only Python code (verified unchanged upstream). Extract and run shard.exe start.

Full details: docs/LAN_SHARDING_DESIGN.md

Shard v1.6.0 - first prebuilt release

Choose a tag to compare

@avelone123123 avelone123123 released this 11 Nov 03:36

Shard v1.6.0 — first prebuilt release

Windows x64 binaries are attached as shard-v1.6.0-windows-x64.zip. The desktop app auto-starts the local server and opens the dashboard in your browser. The repo plus pip install -e . are still required (the app drives a Python server); MSVC Build Tools are not.

RU: первый релиз с готовыми бинарниками — распакуйте shard.exe + shard_core.dll в корень клонированного репозитория, установите зависимости (pip install -e .) и запустите двойным кликом: программа сама поднимет сервер и откроет дашборд.

What's new

  • shard start now opens the browser automatically (parity with python main.py). Explicit --host/--port flags take priority over SHARD_HOST/SHARD_PORT; SHARD_DOCKER=1 still disables auto-open.
  • Restored native sources (shard_core.cpp, shard_gui.cpp) that were accidentally committed empty in v1.5.0. Verified: rebuilt binaries match the last good sizes byte-for-byte, the 98-kernel engine (v1.4.79 core) loads and benchmarks.

What's in the zip

File Size Purpose
shard.exe 540 KB Desktop app: auto-starts server + dashboard
shard-native.exe 540 KB Same app under its build name
shard_core.dll 148 KB C++20/MASM AVX2 native engine (98 kernels)

How to run

git clone https://github.com/avelone123123/shard.git
cd shard
pip install -e .

Unzip the download into the repo root and double-click shard.exe — or skip the GUI entirely and run shard start.

Honest notes

  • Speed figures from the fit advisor are estimates computed from hardware profile, not measurements.
  • The native core is optional; quantized models run through the GGUF (llama.cpp) backend.
  • CLI output mixes English and Russian — known cosmetic issue, not a bug.

Full Changelog: v1.5.1...v1.6.0