Skip to content

v1.0.2

Choose a tag to compare

@nvme101010 nvme101010 released this 12 Oct 18:17
· 217 commits to main since this release
7647163

Kubernetes deployment, and a smaller image footprint. No sizing numbers change.

Deploy on Kubernetes

kubectl apply -k "https://github.com/nvme101010/LLM-Inference-Estimator//deploy/k8s?ref=v1.0.2"

deploy/k8s holds a Deployment (2 replicas, non-root, read-only root filesystem, no capabilities, readiness and liveness probes on /healthz, 10m CPU and 16 Mi memory requested, 200m and 32 Mi limits), a Service on port 80, a PodDisruptionBudget, and an optional Ingress. Port-forward to try it: kubectl port-forward svc/llm-inference-estimator 8080:80.

Other ways to run it

What changed since 1.0.1

  • nginx runs two workers instead of one per node CPU. On a 24-core host the image used 24 workers and 19 MB; it now uses about 3 MB on any node, so the 32 Mi Kubernetes limit holds everywhere.
  • deploy/k8s manifests and a README section for them.

Model and board data as of October 2026. Estimates for planning, not a guarantee.