v1.2.0

Speed and first-token estimates now count the prompts that keep arriving, are calibrated against published benchmarks, and come with a likely range. Sizing numbers change, most for long-prompt workloads. 1.1.0 was never released on its own; its changes are here.

Deploy on Kubernetes

kubectl apply -k "https://github.com/nvme101010/LLM-Inference-Estimator//deploy/k8s?ref=v1.2.0"

What changed since 1.0.3

• Speed under load. New prompts keep arriving while replies stream, and reading them takes a share of the engine's time, so speed per user is now the average under that load and the first token includes the wait for prompts ahead. Speed on this setup shows the request rate, how busy prompts keep the engine and the speed between prompt reads. RAG and long documents come out slower than 1.0.3 said; chat barely changes. (#12)
• Calibrated against published benchmarks. Board compute is each datasheet's dense BF16 figure, and the shares of bandwidth and compute that serving reaches, the NVLink all-reduce latency, FP8's realized gains and vLLM's fixed costs are fitted to 295 benchmark points from NVIDIA NIM and SemiAnalysis InferenceX (Llama 3.1 8B, Llama 3.3 70B and gpt-oss-120b on H200, B200 and L40S). On benchmarks the fit didn't see, speed per user lands at 1.02 × measured, 80% of points between 0.80 and 1.31. A factor for AMD's software stack, fitted to MI355X runs, applies to the MI350P. Method and every point: validation. (#13, #14)
• Likely ranges, and goals judged at the cautious end. Speed and first token show the range measured results fell in. A speed goal counts as met only at the low end of speed and the slow end of first token: held out, that over-promised by more than 25% on 3 of 295 benchmark points.
• Honest wording. The verdict says a goal is met on the estimate and to confirm with a load test, and capacity says what memory alone holds apart from what also reaches the goal.
• Versioned links. Shared links and summaries carry the estimator version. A link from an older version opens with what has changed since and the image that reproduces it, such as ghcr.io/nvme101010/llm-inference-estimator:1.0.3.
• First token includes the engine's ~37 ms to take a request in; only the network round trip is left out.
• RTX PRO 6000 memory bandwidth corrected to the Server Edition's 1,597 GB/s.

Container: ghcr.io/nvme101010/llm-inference-estimator:1.2.0 (also 1.2 and latest), linux/amd64 and linux/arm64. Hosted page: https://nvme101010.github.io/LLM-Inference-Estimator/

Model and board data as of October 2026. Estimates for planning, not a guarantee.