deployment

Deployment

TokenPrint is two deployable pieces with very different needs:

• Frontend — a static-ish Next.js app. Deploys anywhere (Vercel, Netlify, Cloudflare Pages, a static host).
• Backend — FastAPI + PyTorch that loads a ~1GB model and streams generation over WebSocket. Needs a persistent process with enough RAM and long-lived connections. This does not fit serverless platforms (Vercel/Netlify functions) — size limits, execution timeouts, and no persistent WebSockets.

The frontend is already environment-driven: lib/api.ts reads NEXT_PUBLIC_API_URL (default http://localhost:8000) and lib/ws.ts derives the WebSocket URL from it (http→ws, https→wss). Set that one variable and the whole app points at a hosted backend.

Option A — frontend only (fastest)

Deploy just the Next.js app. Works standalone for the Architecture Explorer via .gguf drag-and-drop (parsed client-side, no backend) and, since #329, for browser GPT-2 generation (WebGPU/ONNX or CPU/WASM, greedy ≤ 32 new tokens). "Use live Qwen model", attention, and Walkthrough will show "Failed to fetch" until a backend is reachable (see the capability matrix).

Vercel

1. Import the GitHub repo.
2. Set Root Directory to frontend — the repo is a monorepo; this is required.
3. Build command next build, output auto-detected. Deploy.

Option B — full app (frontend + hosted backend)

Host the backend on a platform that supports long-running Python + WebSockets: Render, Railway, Fly.io, or a Hugging Face Space (or any VM/container host).

1. Backend

Run the same command the dev setup uses, bound to the platform's port:

pip install -r backend/requirements.txt
python -m uvicorn app.main:app --app-dir backend --host 0.0.0.0 --port $PORT

Notes:

• First boot downloads Qwen/Qwen2.5-0.5B-Instruct (~1GB) from Hugging Face; give it a persistent disk/cache or expect a cold-start download.
• CPU is fine at this model size; no GPU required.
• TOKENPRINT_MODEL / TOKENPRINT_DEVICE env vars override the model and device.

2. CORS

The backend restricts origins in backend/app/main.py (locals plus sudharsanselvaraj.github.io, tokenprint.in and subdomains). Add your deployed frontend origin:

allow_origins=[
    "http://localhost:3000",
    "https://your-app.vercel.app",   # <- your deployed frontend
    "https://sudharsanselvaraj.github.io",
    "https://tokenprint.in",
],

3. Frontend

Deploy as in Option A, and set the env var:

NEXT_PUBLIC_API_URL = https://your-backend-host

This drives both the REST calls and the WebSocket (wss://…/ws/generate).

Sanity checklist

• Backend GET /health returns {"status":"ok","model_loaded":true}.
• GET /architecture returns from the deployed backend origin without a CORS error in the browser console.
• NEXT_PUBLIC_API_URL has no trailing slash and uses https in prod.
• The deployed frontend origin is in the backend's allow_origins.
• A .gguf drag-and-drop works even with the backend down (client-side path).

---

Option C — static demo on GitHub Pages (no backend at all)

The most useful deployment for sharing: a fully static site that replays real recorded traces. No GPU, no Python, no running model — but every number on screen still came from an actual forward pass, because a trace is one frozen to disk (see api.md).

Config — frontend/next.config.mjs:

const repo = process.env.GITHUB_REPOSITORY?.split("/")[1];
{
  output: "export",
  trailingSlash: true,
  images: { unoptimized: true },
  ...(repo ? { basePath: `/${repo}`, assetPrefix: `/${repo}` } : {}),
}

basePath is derived from GITHUB_REPOSITORY at build time, so the same config works locally (no basePath) and on a project Pages site served from /<repo>/. Building writes to frontend/out/.

Workflow — .github/workflows/deploy.yml runs npm ci → npm run build (which includes the verify-data guard) → configure-pages → upload-pages-artifact (path frontend/out) → deploy-pages.

Important

Pages must be enabled once, out-of-band. The workflow's GITHUB_TOKEN cannot create a Pages site — configure-pages fails with Create Pages site failed / HttpError: Resource not accessible by integration, and enablement: true does not help, because creating a Pages site needs repo administration rights that GITHUB_TOKEN cannot be granted through the permissions: block.

Fix it once in Settings → Pages → Source: GitHub Actions, then re-run the job. Every subsequent push deploys automatically. (enablement: true is kept in the workflow so a fork whose owner has admin rights self-provisions.)

Bundled traces live in frontend/public/demo/ and are generated by scripts/generate-demo-trace.py. Snapshot URLs (lib/useSnapshotUrl.ts) encode mode / token / op / chapter, so a link can point at an exact moment in a replay.