v1.2.0

Speed and first-token estimates now count the prompts that keep arriving, are calibrated against published benchmarks, and come with a likely range. Sizing numbers change, most for long-prompt workloads. 1.1.0 was never released on its own; its changes are here.

Deploy on Kubernetes

kubectl apply -k "https://github.com/nvme101010/LLM-Inference-Estimator//deploy/k8s?ref=v1.2.0"

What changed since 1.0.3

• Speed under load. New prompts keep arriving while replies stream, and reading them takes a share of the engine's time, so speed per user is now the average under that load and the first token includes the wait for prompts ahead. Speed on this setup shows the request rate, how busy prompts keep the engine and the speed between prompt reads. RAG and long documents come out slower than 1.0.3 said; chat barely changes. (#12)
• Calibrated against published benchmarks. Board compute is each datasheet's dense BF16 figure, and the shares of bandwidth and compute that serving reaches, the NVLink all-reduce latency, FP8's realized gains and vLLM's fixed costs are fitted to 295 benchmark points from NVIDIA NIM and SemiAnalysis InferenceX (Llama 3.1 8B, Llama 3.3 70B and gpt-oss-120b on H200, B200 and L40S). On benchmarks the fit didn't see, speed per user lands at 1.02 × measured, 80% of points between 0.80 and 1.31. A factor for AMD's software stack, fitted to MI355X runs, applies to the MI350P. Method and every point: validation. (#13, #14)
• Likely ranges, and goals judged at the cautious end. Speed and first token show the range measured results fell in. A speed goal counts as met only at the low end of speed and the slow end of first token: held out, that over-promised by more than 25% on 3 of 295 benchmark points.
• Honest wording. The verdict says a goal is met on the estimate and to confirm with a load test, and capacity says what memory alone holds apart from what also reaches the goal.
• Versioned links. Shared links and summaries carry the estimator version. A link from an older version opens with what has changed since and the image that reproduces it, such as ghcr.io/nvme101010/llm-inference-estimator:1.0.3.
• First token includes the engine's ~37 ms to take a request in; only the network round trip is left out.
• RTX PRO 6000 memory bandwidth corrected to the Server Edition's 1,597 GB/s.

Container: ghcr.io/nvme101010/llm-inference-estimator:1.2.0 (also 1.2 and latest), linux/amd64 and linux/arm64. Hosted page: https://nvme101010.github.io/LLM-Inference-Estimator/

Model and board data as of October 2026. Estimates for planning, not a guarantee.

v1.0.3

Kubernetes polish. No sizing numbers change.

Deploy on Kubernetes

kubectl apply -k "https://github.com/nvme101010/LLM-Inference-Estimator//deploy/k8s?ref=v1.0.3"

What changed since 1.0.2

• Readiness probe waits two seconds before its first check. Pods started so fast that the first probe hit before nginx bound its port, logging one harmless "connection refused" Warning event per pod. First verified on a real cluster (Docker Desktop, Kubernetes 1.36): both replicas Ready in about two seconds, running as uid 101 with a read-only root filesystem and two nginx workers.

Container: ghcr.io/nvme101010/llm-inference-estimator:1.0.3 (also 1.0 and latest), linux/amd64 and linux/arm64. Hosted page: https://nvme101010.github.io/LLM-Inference-Estimator/

v1.0.2

Kubernetes deployment, and a smaller image footprint. No sizing numbers change.

Deploy on Kubernetes

kubectl apply -k "https://github.com/nvme101010/LLM-Inference-Estimator//deploy/k8s?ref=v1.0.2"

deploy/k8s holds a Deployment (2 replicas, non-root, read-only root filesystem, no capabilities, readiness and liveness probes on /healthz, 10m CPU and 16 Mi memory requested, 200m and 32 Mi limits), a Service on port 80, a PodDisruptionBudget, and an optional Ingress. Port-forward to try it: kubectl port-forward svc/llm-inference-estimator 8080:80.

Other ways to run it

• Hosted: https://nvme101010.github.io/LLM-Inference-Estimator/
• Container: docker run --rm -p 8080:8080 --read-only --tmpfs /tmp ghcr.io/nvme101010/llm-inference-estimator:1.0.2 then open http://localhost:8080. Tags 1.0.2, 1.0 and latest, for linux/amd64 and linux/arm64.

What changed since 1.0.1

• nginx runs two workers instead of one per node CPU. On a 24-core host the image used 24 workers and 19 MB; it now uses about 3 MB on any node, so the 32 Mi Kubernetes limit holds everywhere.
• deploy/k8s manifests and a README section for them.

Model and board data as of October 2026. Estimates for planning, not a guarantee.

v1.0.1

Wording and presentation fixes on top of 1.0.0. No sizing numbers change.

Use it

• Hosted: https://nvme101010.github.io/LLM-Inference-Estimator/
• Container: docker run --rm -p 8080:8080 ghcr.io/nvme101010/llm-inference-estimator:1.0.1 then open http://localhost:8080. Tags 1.0.1, 1.0 and latest.

What changed since 1.0.0

• First token, not first reply. The tile is now called "First token", which is what it measures. For a thinking model it also shows when the answer starts (first token plus the thinking budget at the per-user speed), and its goal line says plainly: "Meets the 2 s goal only if users see the thinking stream. If it is hidden, they wait for the answer." The tile turns orange in that case.
• Server-side caveat. The tile's hover text, the math section, the downloadable summary and the README say every speed figure is measured at the server in steady state and leaves out the network round trip, queueing before the engine admits a request, and tokenization, which can outweigh a short first token.
• Speed per user defined. The math section and the tile's hover text state that each decode step gives every active request one token, so everyone streams at the same rate: speed per user = 1 ÷ step time, output = users ÷ step time.

Model and board data as of October 2026. Estimates for planning, not a guarantee.

v1.0.0

First tagged release of the LLM Inference Estimator: a single-page tool that sizes one server for an open LLM, your user count, workload and speed goal.

Use it

• Hosted: https://nvme101010.github.io/LLM-Inference-Estimator/
• Container: docker run --rm -p 8080:8080 ghcr.io/nvme101010/llm-inference-estimator:1.0.0 then open http://localhost:8080. Images for linux/amd64 and linux/arm64; tags 1.0.0, 1.0 and latest.
• Single file: download index.html from the source below and open it in a browser.

What's in 1.0.0

• Layout. Fewer tooltip icons, numbers in tabular figures, one-line input hints. The suggestion panel shows three figures and explains capacity in a sentence. "Every board" is a table with the reasoning behind each row's GPU count behind a chevron. "Speed on this setup" lives under "Show the math".
• Boards. H200 NVL (1, 2, 4 or 8 cards per server; an NVLink bridge joins at most 4, so one copy of a model spans at most 4 and 8 cards run two copies), HGX H200, HGX B200 and HGX B300 (8-GPU systems joined through NVSwitch), L4, L40S, RTX PRO 4500 and 6000 Blackwell, AMD MI350P. The H100 is dropped as end of life.
• Not recommended layouts, where one copy would be split across PCIe-only cards, are marked in red and explained, but stay pickable.
• Explanations. Rows say what a split means ("both serve every user together") versus copies ("each serving ~300 of them"). The math section states that speed per user is 1 ÷ one decode step, with every active request getting one token per step.
• Container. nginxinc/nginx-unprivileged image, ~21 MB, non-root, read-only root filesystem, health check, gzip, security headers. A GitHub Actions workflow builds and smoke-tests every pull request and publishes to GHCR on merges and tags.

Model and board data as of October 2026. Estimates for planning, not a guarantee.