Repository navigation
Paragon Server v0.0.1 - Initial Release
A lightweight Go server for serving neural network inference using Paragon, a deterministic, cross-platform AI runtime. This example mounts a trained model once (onto GPU via WebGPU or CPU fallback) and handles concurrent forward passes efficiently. Includes an embedded web UI for load testing, latency monitoring, and session capture.
🚀 What's New in v0.0.1
- Full Server Implementation: Loads Paragon JSON models, initializes GPU/CPU backend, and serves inference via Fiber web framework.
- API Endpoints:
/infer: Single forward pass./infer-batch: Looped batch inference./blast: Concurrent burst testing (N goroutines)./save-session: Persist UI sessions to disk./health&/config: Monitoring endpoints.
- Embedded Web UI:
- Home/About pages.
- Load Test page with client/server modes, metrics (p50/p90/p99 latencies), bar charts, and session exports (JSON/CSV).
- Real-time inflight request polling.
- Concurrency & Performance:
- Semaphore-limited GPU submissions (configurable via
-maxgpu). - GPU mutex for non-reentrant backend.
- Graceful shutdown with cleanup.
- Semaphore-limited GPU submissions (configurable via
- Model Support: Auto-derives input shapes/activations from JSON; warmup with zeros.
- Developer Tools: Hot-reload templates, embedded static assets, session saving to
./data/sessions/.
📦 Installation & Quick Start
- Prerequisites: Go 1.21+, a Paragon JSON model (e.g., MNIST).
- Build: