Skip to content

Initial Release v0.0.1

Latest

Choose a tag to compare

@openfluke-dev openfluke-dev released this 08 Oct 09:17

Paragon Server v0.0.1 - Initial Release

A lightweight Go server for serving neural network inference using Paragon, a deterministic, cross-platform AI runtime. This example mounts a trained model once (onto GPU via WebGPU or CPU fallback) and handles concurrent forward passes efficiently. Includes an embedded web UI for load testing, latency monitoring, and session capture.

🚀 What's New in v0.0.1

  • Full Server Implementation: Loads Paragon JSON models, initializes GPU/CPU backend, and serves inference via Fiber web framework.
  • API Endpoints:
    • /infer: Single forward pass.
    • /infer-batch: Looped batch inference.
    • /blast: Concurrent burst testing (N goroutines).
    • /save-session: Persist UI sessions to disk.
    • /health & /config: Monitoring endpoints.
  • Embedded Web UI:
    • Home/About pages.
    • Load Test page with client/server modes, metrics (p50/p90/p99 latencies), bar charts, and session exports (JSON/CSV).
    • Real-time inflight request polling.
  • Concurrency & Performance:
    • Semaphore-limited GPU submissions (configurable via -maxgpu).
    • GPU mutex for non-reentrant backend.
    • Graceful shutdown with cleanup.
  • Model Support: Auto-derives input shapes/activations from JSON; warmup with zeros.
  • Developer Tools: Hot-reload templates, embedded static assets, session saving to ./data/sessions/.

📦 Installation & Quick Start

  1. Prerequisites: Go 1.21+, a Paragon JSON model (e.g., MNIST).
  2. Build: